Risk Adjustment In Neurocritical care (RAIN) - prospective validation of risk prediction models for adult patients with acute traumatic brain injury to use to evaluate the optimum location and comparative costs of neurocritical care: a cohort study

Search strategy

The electronic search strategy was based on that used by Perel et al. 9 (see Appendix 1) and was performed using EMBASE (incorporating MEDLINE) to identify eligible studies, published in English, from 2006 to 2008, which (1) gave an overall prognostic estimation by combining the predictive information from at least two variables – studies could develop new prognostic models (derivation studies) or evaluate previous ones (validation studies); (2) used variables collected before hospital discharge, which were therefore considered as predictors; (3) included patients of any age; (4) included patients with any type or severity of TBI; and (5) predicted any outcome, such as neurological impairment, disability, survival, etc. There was no time restriction for the evaluation of outcomes. Three search themes were combined: ‘traumatic brain injury’; ‘brain/coma/consciousness/craino/skull’; and ‘prognosis/predicts’. One reviewer (GPr) examined titles, abstracts and keywords of records identified by the electronic database searches for eligibility. The full text of all potentially eligible papers was obtained and independently assessed by two reviewers (GPr and DAH) for eligibility using the pre-defined inclusion criteria described above. Disagreement was resolved by a third reviewer (KMR). The reference lists of all full-text papers reviewed were checked for any additional potentially eligible studies.

The studies previously identified by Perel et al. 9 and by the updated searches were then independently assessed by two reviewers (GPr and DAH) for eligibility for the RAIN study. Studies were eligible for inclusion if they (1) were based on adult (> 16 years) populations; (2) had a sample size of greater than 500 patients in either the development or validation data set; (3) aimed to evaluate outcome regardless of care received during the hospital stay; and (4) were UK based or multicentre. Disagreement was resolved by a third reviewer (KMR).

Assessment of methodological quality

There is no gold standard tool for quality assessment of either RCTs or observational studies. Assessment of the methodological quality of studies eligible for the RAIN study was conducted using the same approach as Perel et al. 9 considering two domains: (1) internal validity, which refers to systematic error and is related to the study design and (2) external validity, which refers to the generalisability of the study and whether the results can be extrapolated to other populations and settings. Eighteen questions related to internal and external validity were considered for each of the studies, as follows:

Expert review

All studies eligible for the RAIN study were then reviewed by the RAIN Study Steering Group to identify any additional studies, either published or ongoing, of relevance, and to select the most appropriate models for validation in a UK critical care setting.

Description of eligible studies

Four studies,30,35,47,48 reporting a total of 17 risk prediction models, met the RAIN study eligibility criteria. A brief description of each is given below and a summary is provided in Table 1.

Assessment of methodological quality

The quality assessment of the four studies,30,35,47,48 using the criteria of Perel et al. ,9 is summarised in Table 2. Neither the MRC CRASH trial collaborators30 nor Steyerberg et al. 35 reported completeness of follow-up in their respective papers reporting development of the CRASH and IMPACT models. However, for the CRASH trial, a separate paper reporting the trial results indicated overall follow-up of 95% for GOS at 6 months,54 and a paper reporting the design of the IMPACT database indicated overall follow-up of 95% for GOS at 6 months (including last observation carried forward imputation of 18% of values from 3 months). 55

All of the studies30,35,47,48 provided some justification for the predictors included in the models, reflecting a combination of an existing established relationship with outcome and ease of collection and use. However, none of the investigators reported clear definitions for the predictors. All of the studies30,35,47,48 used valid outcomes (mortality and/or GOS) in the models. Handling of missing data varied. In two studies,30,47 complete case analyses were used. The final number of patients included in the model of Signorini et al. 47 was not reported; however, 20% of patients were missing CT assessment alone and, therefore, a maximum of 80% of the original sample of 365 patients can have been used in fitting the final model. The MRC CRASH trial collaborators30 cited low levels of missing data as justification for their complete case approach; however, this approach resulted in only 88% of the original sample being included in the Basic model for high-risk countries and 79% in the CT model. In contrast, for the remaining two studies,35,48 statistical imputation methods were used, resulting in all patients being included in the final models. Hukkelhoven et al. 48 used regression imputation to impute the 4.8% of missing values, acknowledging that such an approach would slightly underestimate the true variability and Steyerberg et al. 35 used the gold standard method of multiple imputation.

In all four studies,30,35,47,48 risk prediction models were developed using logistic regression, although the approach to variable selection varied. Signorini et al. 47 used a form of forward stepwise selection but with the order of variables being added to the model based on a combination of data completeness and clinical criteria rather than statistical significance alone. Hukkelhoven et al. 48 fitted a full model and used backward stepwise selection to remove variables with p > 0.2. The MRC CRASH trial collaborators30 included variables in the final models if they were significant at the 5% level in a full multivariable model. Finally, Steyerberg et al. 35 based inclusion in their final models on partial Nagelkerke R²-values from a previous multivariable analysis of the same database. 57 The only authors who reported evaluating interactions between predictors were Steyerberg et al. ;35 however, it was unclear which, or how many, potential interactions were examined. All of the studies30,35,47,48 used continuous modelling for continuous variables, and all either included or considered some degree of non-linearity. The criterion of at least 10 events per predictor variable included in the modelling process was met by three of the four studies. 30,35,48 Signorini et al. 47 reported approximately eight events per predictor variable included in the model.

A description of the sample population, including important case mix variables, was reported for all four studies. 30,35,47,48 In addition, simple methods for calculating predictions, including a nomogram, simple clinical scores, web calculators and a spreadsheet calculator, were provided. Measures of discrimination were reported in all papers, although only Hukkelhoven et al. 48 included a CI on the c-index [area under the receiver operating characteristic (ROC) curve]. In all papers, calibration was summarised graphically and tested for perfect calibration with the Hosmer–Lemeshow test, with CIs reported on model estimates.

All models were validated either internally and/or externally as described above and summarised in Table 1. None of the models was evaluated for its effect in clinical practice.

Internal validation was performed by Hukkelhoven et al. 48 and the MRC CRASH trial collaborators30 using bootstrap methods, and by Steyerberg et al. 35 using cross-validation across the 11 separate study data sets comprising the IMPACT database. The model of Signorini et al. 47 was externally validated using data from a further 520 patients admitted to the same single centre. The models of Hukkelhoven et al. 48 were externally validated using data from approximately 1500 patients from observational registries. Modified versions of the CRASH models were validated using the IMPACT database. 30 The IMPACT Core models and modified versions of the IMPACT Extended models were validated in the CRASH trial data set; however, it was not possible to externally validate the IMPACT Lab models, as the CRASH trial did not record the required laboratory values. 35

Expert review

The RAIN Study Steering Group did not identify any further studies (either published or ongoing) that would be potentially eligible for the RAIN study. The four studies identified by the systematic review reported development and validation of 17 risk prediction models. Of these, 11 were potentially eligible for validation in the RAIN study; the models developed by the MRC CRASH trial collaborators30 for low-income countries and predicting mortality at 14 days were excluded. Following review by the RAIN Study Steering Group, 10 risk prediction models were selected for external validation in the RAIN study. The model developed by Signorini et al. 47 was excluded because the model was developed using data from a relatively small single-centre study with mortality at 1 year as the outcome. The remaining three studies were large multicentre studies, which considered functional outcome as well as mortality at 6 months. There was also concern about the data burden associated with the ISS included in the Signorini et al. model. 47

Principal findings

The systematic review of the literature identified four studies30,35,47,48 reporting development and validation of 17 risk prediction models for TBI. Of the 17 models, 11 were eligible for the RAIN study. Following assessment of their methodological quality and review by the RAIN Study Steering Group, 10 models, developed and validated in three studies,30,35,48 were selected for external validation in the RAIN study. Four models were developed for mortality at 6 months35,48 and six models were developed for unfavourable outcome, using the GOS, at 6 months. 30,35,48

The variables included across the 10 selected models were age, GCS score, GCS motor score, pupil reactivity, presence of major extracranial injury, hypoxia, hypotension, glucose, haemoglobin, Marshall CT classification,56 presence of traumatic subarachnoid haemorrhage (SAH), presence of extradural haematoma, presence of petechial haemorrhages, obliteration of the third ventricle or basal cisterns, midline shift and non-evacuation of haematoma. The outcomes assessed were mortality and unfavourable outcome, using the GOS, at 6 months.

Strengths and weaknesses

A major strength of this systematic review was being built on a previous, rigorous systematic review. 9 One of the strengths of the Perel et al. 9 review is that there was no restriction on the types of patients, i.e. any age and any severity of TBI, or on the setting. Interestingly, although the burden of trauma is much greater in low-income countries, only 2% of the models identified included patients from these countries. The MRC CRASH trial collaborators30 identified significant interactions between the country's income level and several predictors and so developed separate models for low-/middle-income countries and for high-income countries. Given that the RAIN study was seeking to validate prognostic models for use in a UK NHS setting, models developed for low-/middle-income countries were excluded.

There are some limitations to the systematic review. The original literature search by Perel et al. 9 was restricted to 1990 onwards. However, this was on the basis that changes in management and diagnostic technology in recent years means that prognostic models developed before 1990 are unlikely to be relevant for the current medical care of patients with TBI. In addition, only studies that explicitly combined at least two predictors were included, which means that studies that used multivariable analysis to investigate individual predictors but did not report an overall estimation were excluded. Similarly, studies that assessed clinical prediction rules considering more than one variable were excluded if they did not combine them.

Methodological quality of the risk prediction models

The original systematic review by Perel et al. 9 and the recent update reveal that a large number of prognostic models for TBI have been published. However, their methodological quality is relatively poor. Limitations include, small sample sizes (fewer than 10 events per variable), loss to follow-up rates in excess of 10%, inappropriate handling of missing data (i.e. not using statistical imputation methods), and rarely being validated in external populations. Perel et al. 9 reported that of the 102 models (in 53 studies) identified, they considered only the three models developed by Signorini et al. 47 and Hukkelhoven et al. 48 to be clinically useful for patients from high-income countries. All three models fulfilled the eligibility criteria for inclusion in the RAIN study. The updated search identified an additional eight models (from two studies30,35) that were also eligible. In general, the methodological quality of these 11 models was good. However, eight models35,48 were developed using data from multiple sources and may therefore be limited by differences between data sets in eligibility criteria, definitions of variables and timings of measurements. Although all of the studies included discussion about the rationale for including specific predictors, none reported clear definitions for predictive variables. In addition, there was variation in how missing data were handled: two of the four studies used regression or multiple imputation and two used complete case analysis on the basis that there were few missing data; however, this meant that for at least one of the models, only 79% of the original sample was included. 30

Of the 11 eligible models, 10 risk prediction models were selected by the RAIN Study Steering Group for validation in the RAIN study. 30,35,48 All were developed using some or all data from RCTs, which may limit their external validity. Even in large pragmatic RCTs, such as the CRASH trial,30 external validity may be affected by self-selection of centres and patients to participate in the trial, as well as the potential for patients enrolled in a trial (in both the active and control arms) to receive better standard of care than in usual clinical practice. 58,59 To assess whether a model is generalisable to other populations, it is important to conduct external validation. Of the 10 models, all except two were validated on patients from different centres. However, for four of the remaining eight models, a limitation was that some variables had to be excluded from the models for validation as they were not available in the validation sample. Therefore, only four of the 10 models – those by Hukkelhoven et al. 48 and the Core models from Steyerberg et al. 35 – have been externally validated without undergoing any modifications.

Summary

In summary, three families of risk prediction models including 10 individual models were identified that are most likely to be applicable to a UK critical care setting. 30,35,48 These models require further prospective validation, and potentially recalibration, before they can be applied with confidence in neurocritical care in the NHS.

Research governance

The RAIN study was sponsored by ICNARC. The contract for the study was signed in December 2008 and, once the study co-ordinator was appointed, the process of completing research governance approvals commenced. Owing to the inclusion of adults with incapacity, two separate applications were made to the NHS Research Ethics Committee (REC) for Wales, covering sites in England and Wales under the Mental Capacity Act 2005, and to the Scotland A REC, covering sites in Scotland under the Adults with Incapacity (Scotland) Act 2000. Favourable opinions were received on 13 March 2009 (ref. 09/MRE09/10) and 30 March 2009 (ref. 09/MRE00/15), respectively.

In response to the unique problems faced by patients with acute TBI in critical care, we delayed the request for consent until 6 months after the TBI at the point of follow-up. At hospital/critical care unit admission these patients are often unconscious and their level of consciousness continues to vary during their stay in the critical care unit. Generally, treatment needs to be started urgently, so there is little time for health-care staff to adequately explain research studies to patients or their families. This is, of course, a stressful and emotional time for families. In view of these difficulties in gaining informed consent, an application was made to the National Information Governance Board (NIGB) Ethics and Confidentiality Committee (ECC) for support under Section 251 of the NHS Act 2006 for permission to hold sufficient patient identifiable data, prior to patient consent, in order for us to contact the patient at 6 months post TBI and gain their consent. Section 251 support, covering sites in England and Wales, was obtained on 4 August 2009 [ref. ECC 2–06(d)/2009]. For the two sites in Scotland, falling outside the remit of the NHS Act 2006, approval was sought from the Caldicott Guardian and was granted on 10 November 2009 and 25 March 2010.

Central Research and Development (R&D) approval was gained on 4 August 2009. Site-specific information (SSI) forms were submitted for each for each NHS Trust, with the last form submitted on 3 November 2009 and the final approval gained on 28 January 2010.

The National Institute for Health Research (NIHR) Clinical Research Network (CRN) Portfolio details high-quality clinical research studies that are eligible for support from the NIHR CRN in England. The RAIN study was adopted on to the NIHR CRN Portfolio on 27 August 2009 (ref. 7349).

Design and development of data set

The initial RAIN study data set was developed and produced by the RAIN study team from the original model publications. Definitions for some of the fields were refined through discussion and consultation with the clinical experts on the RAIN Study Steering Group. There were five elements to the data set: (1) characteristics of the patients and their injury; (2) risk factors for the selected risk prediction models; (3) location of care details, to describe and cost the patient journey in order to investigate the effect of location of neurocritical care; (4) short-term outcomes at discharge from critical care and acute hospital; and (5) contact details, to provide the information required for the 6-month follow-up.

To avoid duplication of data collection, the RAIN study was piggybacked on to the CMP in England and Wales and linked with data provided by the Scottish Intensive Care Society Audit Group (SICSAG) in Scotland. Both the CMP and SICSAG databases have been independently assessed to be of high quality against 10 criteria for coverage and accuracy by the Directory of Clinical Databases at docdat.ic.nhs.uk. Critical care units in England and Wales that were not participating in the CMP recorded all required fields within the RAIN study data set.

The publications reporting each risk prediction model, other associated publications and, where available, original study documentation relating to the data sources were examined for definitions for each field to be included in the data set. The definitions and, in particular, the time points for data collection were often not clearly defined and/or varied both between risk models and between different data sources used for the development of the same risk model. Clinical experts from the RAIN Study Steering Group also identified a small number of additional fields that they felt to be important predictors of outcome following TBI that were not included in any of the risk prediction models, the collection of which was considered valuable for informing future work in this area.

The full RAIN study data set and definitions are provided in Appendix 3. A brief summary is given below.

Sample size calculation

We performed a simulation study to assess the power to detect a difference in the c-index (area under the ROC curve) between two different risk prediction models applied to the same population. Simulations were based on the following assumptions: the rate of unfavourable outcome (death or severe disability) at 6 months in the population will be 40% (based on the observed rate of unfavourable outcomes in high-income countries in the CRASH trial30 and consistent with the results of a regional audit in East Anglia60); statistical tests will be based on a two-sided p-value of p = 0.05; and the ability to detect, with 80% power, a 10% relative difference in c-index from the value of 0.83 observed for the CRASH model in the development sample. 30 A total of 17,500 data sets were simulated at different sample sizes using a binormal model61 and the empirical power was assessed at each sample size as the proportion of data sets in which a statistically significant difference was detected (see Appendix 2).

Based on these simulations, a sample size of 3100 patients was required for model validation. To allow for 8% loss to follow-up [based on the observed follow-up rates from the CRASH62 and Randomised Evaluation of Surgery with Craniectomy for Uncontrollable Evaluation of Intra-Cranial Pressure (RESCUEicp) RCTs63], we aimed to recruit 3400 patients.

Using data from the CMP database, we anticipated the rate of admission of adult patients following acute TBI to be approximately eight per unit per month for dedicated neurocritical care units, six per unit per month for combined neuro/general critical care units, and 0.5 per unit per month for general critical care units outside a neuroscience centre. We therefore aimed to recruit at least 12 dedicated neurocritical care units, 13 combined neuro/general critical care units and 30 general critical care units outside neuroscience centres to complete recruitment within 18 months.

Recruitment of sites and patients

Data management

Six-month follow-up of neurological outcome and quality of life

Patients identified as not having died at 4.5 months using data from the critical care unit and from MRIS followed the process shown in Figure 3.

FIGURE 3.

Patient follow-up process.

Patient outcomes were collected centrally by the RAIN study team at ICNARC, using methods based on those undertaken in previous research studies, including the CRASH62 and RESCUEicp RCTs. 63 For patients registered with a general practice, their GP was sent a letter explaining the RAIN study and a form to complete to confirm that the patient had not died and to verify the patient's address. GPs could confirm the patient's status and address in a variety of ways: by returning the form by post or fax, completing a secure on-line response form or telephoning the RAIN study team. If no response was received, a follow-up telephone call was made to ensure the original information had been received. In cases where patients were not registered with a GP, REC approval allowed for them to be contacted directly. When informed that a patient was no longer registered with the GP contacted, attempts were made to ascertain the patient's current GP.

Patients were then sent, by post, an introductory letter, information sheet, consent form, questionnaires, freepost return envelope and pen (see Appendix 5), following best evidence-based practice to maximise response. 64 In instances where the patient was unable to consent, a boxed section on the letter addressing their carer asked them to offer what they feel would be the presumed will of the patient. Two questionnaires were included: the Your Health Questionnaire and the Health Services Questionnaire. The Your Health Questionnaire included the required questions to evaluate the European Quality of life (EuroQol) 5-dimension, 3-level version (EQ-5D-3L)65 and the Glasgow Outcome Scale – Extended (GOSE)66 measures. The EQ-5D-3L was included to enable the calculation of quality-adjusted life-years (QALYs) as the best available global measure of health outcome. 67 The GOSE questionnaire is the most widely used measure of functional outcome following acute TBI,68 and has been used in most of the large, recent and ongoing RCTs. 54,63 Use of a postal questionnaire to collect the GOSE questionnaire responses has been found to have high reliability. 69 The Health Services Questionnaire included questions about the patients use of health services following discharge from acute hospital and was used to cost subsequent use of health services (see Chapter 6). Patients were asked to complete the consent form and questionnaires if they wished to take part in the RAIN study, or to return the questionnaires blank to indicate that they did not wish to take part. Non-responders were followed up with a second letter after 4 weeks, including the same enclosures as the original.

If no postal response was received after a further 2 weeks following the second letter, then patients were telephoned if contact details were available. A telephone interview template was used to explain the RAIN study and to ask for informed consent. In order not to overburden the interviewee, the telephone interview included only the GOSE questionnaire as the primary outcome for the RAIN study. Telephone calls were made at various times from Monday to Saturday between 0900 and 2030 hours to maximise the chances of contacting the patient.

Follow-up ended when a postal questionnaire was returned, either complete or blank, or when a telephone interview was completed or refusal obtained. When post was returned (e.g. not known at this address, no longer at this address, address inaccessible, etc.) the critical care unit and GP were contacted to check the address details and to elicit any updates to the patient information.

When patients were identified either as discharged to or subsequently moved to a care home, rehabilitation centre or another hospital, these institutions were contacted to establish the status of the patient and the most appropriate way to proceed with follow-up. If a patient had capacity to consent but required assistance in reading and/or completing the questionnaire then health-care professionals would often assist the patient. For those unable to consent, institutions advised on the most appropriate person to contact to gain consent. In cases in which a patient did not have the capacity to consent and either there was no next-of-kin or the next-of-kin was also unable to consent, where possible, an Independent Mental Capacity Advocate was identified.

When attempts to make contact by telephone were repeatedly unsuccessful, for example where telephone numbers would ring through to an answering service or a mobile telephone was continually switched off, and attempts had been made on various days and at various times of day over at least a month, and no alternative contact information was available, if an answering service was available, a message with information to contact the RAIN study team was left and if the call was not returned then the patient was considered lost to follow-up.

Recruitment of sites

Recruitment of sites took place between December 2008 and December 2009. In total, 74 critical care units expressed an interest in taking part in the study and were sent an SSI form to complete. Of these, one neurocritical care unit was unable to take part owing to a conflicting, ongoing research study and a further four general critical care units outside a neuroscience centre did not reach the R&D submission stage. Local R&D approval was sought and gained for 69 critical care units. R&D approvals took a median of 68 days [interquartile range (IQR) 32 to 237] from submission of SSI form. R&D approval to start of RAIN study data collection took a median of 27 days (IQR 13 to 119). Two neurocritical care units withdrew, as they were unable to meet the study start date giving a total of 67 critical care units. This exceeded the recruitment targets with 13 dedicated neurocritical care units, 14 combined neuro/general critical units, four additional critical care units within a neuroscience centre (admitting overflow patients from the neurocritical care unit) and 36 general critical care units outside a neuroscience centre participating in the RAIN study (Table 3). One neurocritical care unit (and an additional critical care unit within the same neuroscience centre) withdrew from the study in August 2010 owing to research staffing shortages. All other critical care units collected data until March 2011 (Figure 4).

FIGURE 4.

Recruitment timeline.

Each critical care unit was represented at a start-up meeting/data set familiarisation course. In total, seven start-up meetings/data set familiarisation courses were held between May 2009 and March 2010.

Recruitment of patients

The first patient was recruited to the RAIN study on 19 August 2009 and patient recruitment continued until 31 March 2011. The final RAIN study data set contained a total of 3626 critical care unit admissions. After excluding multiple admissions of the same patient and patients who did not prove finally to have a TBI, 3210 patients remained (Figure 5). Of these, 28 patients were homeless, 61 were non-UK residents and for 12 patients (military) the address details were withheld, resulting in a cohort of 3109 patients (97%) that were followed up for 6-month survival by data linkage with death registrations.

FIGURE 5.

Patient flow. a, Paper questionnaire included full Your Health Questionnaire (GOSE and EQ-5D-3L) and Health Services Questionnaire; telephone questionnaire included GOSE only.

As a result of the regular feedback to critical care units participating in the CMP, four instances of a missed admission for TBI were identified after recruitment of patients had closed and these were, therefore, unable to be included in the study. There were also six admissions for which the research team at the unit was unable to confirm whether or not the patient had a TBI. Validation against figures from TARN indicated similar numbers of patients recruited to the RAIN study as reported through TARN. In some sites, the number of patients in the RAIN study exceeded the number reported to TARN by a small amount and, in other sites, the reverse was true, reflecting different definitions for TBI in the two projects.

Six-month follow-up of neurological outcome and quality of life

Of 2323 patients not reported as dead by MRIS, 242 (10%) refused follow-up and 247 (11%) were lost to follow-up (the first patient was recruited to the RAIN study on 19 August 2009 and patient recruitment continued until 31 March 2011).

A breakdown of the 242 patients for whom outcome data were refused is shown in Table 4. The majority of refusals (177, 73%) were by return of a blank questionnaire (from which no further information was available).

A breakdown of the 247 patients lost to follow-up is shown in Table 5. The largest number of patients lost to follow-up (122, 49%) were those with whom we were unable to make any contact, despite exhausting all options available to us. On four occasions, contact to a patient was blocked by their GP because the patient had not consented in advance, despite NIGB and REC approval for the follow-up process used in the study. Eight requests were received from GPs not to contact the patient or family, either on compassionate grounds or because the GP had spoken to the patient or family and they did not wish to be contacted, and on six occasions family members did not want us to contact their relative. All such requests were respected.

Questionnaires were completed a median of 199 days (IQR 166 to 239 days) after the TBI (Figure 6). Two questionnaires were received very early, owing to data entry errors in the date of TBI that were not identified until follow-up had commenced. There was an initial peak at around 160 days corresponding to postal questionnaires returned following the initial posting, a second peak at around 200 days corresponding to postal questionnaires returned following the second posting, and a subsequent heavy tail of the patients followed up by telephone that were often difficult to contact. A total of 30 patients were eventually contacted for the 6-month follow-up more than 1 year after their TBI.

FIGURE 6.

Distribution of days from TBI to completion of questionnaire. Vertical line indicates 6 months post TBI.

Principal findings

The RAIN study has provided case mix data from the time of injury and presentation at hospital and outcomes data at 6 months following injury on a highly representative sample of patients receiving critical care following acute TBI in the UK.

Challenges in conducting the study

Patients with TBI present a challenging population for obtaining reliable, longer-term outcome data. The reasons for this are wide ranging and must be considered when discussing those who were lost to follow-up. Identifying the current location of the patient is not straightforward because of the frequent movement between hospitals and health-care institutions, entry into rehabilitation programmes and moving back to live with family and friends after TBI, as well as a relatively high proportion of homeless patients and those from overseas. Other challenges include ascertaining whether a patient has the capacity to consent and to accurately answer the questions. For example, in some instances, we were informed by health-care professionals or family that while the patient had the capacity to consent following the TBI they had tendencies to confabulate and that answers provided may not have been accurate.

Throughout the follow-up process, every effort was taken to avoid causing unnecessary distress to patients or their families. Following MRIS confirmation, for those registered with a GP, it was necessary to send a letter to confirm the patient's status. There were a small number of cases where the GP blocked contact with the patient or refused to provide the information because prior patient consent had not been obtained. This was despite GPs being fully informed, by letter and telephone, about the study and the governance and ethics approvals. There were also instances where the GP contacted the patient and the patient requested not to be contacted by the Study Team. This raises possible questions whether such patients were fully informed about the study. However, the benefits of contacting GPs do appear to outweigh these possible missed follow-ups. In 28 cases, patients were not reported as having died by MRIS but, on contacting the GP, were found to have died. Despite this being a relatively small number in comparison with the resources and time required to collect GP responses, the prevention of distress to these families by this intervention cannot be ignored. In cases where GPs requested that we did not contact the patient or family for compassionate reasons, this was respected. Despite the dual safeguards of checking for reported deaths both with MRIS and GPs, there were seven cases in which, on making contact with the family, we were informed the patient had recently died. It is inevitable, when following up a high-risk population, that such situations will occur, but it is the responsibility of the researcher to do everything possible to keep these to a minimum.

The use of postal questionnaires may have been a factor in our response rate. Twenty-six patient questionnaires were reported to have been posted by the patient or family but were never received and, once it appeared that undelivered mail could potentially be influencing response rates and fluctuations in the returns were noticed, Royal Mail was contacted. The registered freepost envelope presented a possible issue. Considering this, and research indicating that stamped addressed envelopes may improve response rates over reply-paid envelopes,64 we switched to using stamps during the course of the study. Additionally, when patients indicated during telephone follow-up that they had already returned a questionnaire by post, we requested that they also complete a telephone interview so that in the event the postal questionnaire was not received an outcome would still be available.

Selection of patients

Despite a GCS score of < 15 being an inclusion criterion for the RAIN study, some patients with suspected TBI were recruited with no GCS score recorded, and others who had an initial GCS score of < 15, either pre-hospital or on presentation at hospital, subsequently improved prior to sedation. Patients from the RAIN study database were included in the analysis if their last GCS score prior to sedation/admission to critical care was < 15.

Statistical methods

Case mix, length of stay (LOS) and outcomes were summarised overall and for subgroups defined by the cause of TBI – road traffic accident (RTA), fall or assault. Patients with other or unknown causes of injury were included in the overall group but excluded from the subgroups. All analyses were descriptive and no statistical testing was undertaken.

Case mix

For presentation, case mix variables are categorised by causal factors; pre-injury status; date/time of TBI; hospital source of admission to critical care; age/sex; neurological dysfunction; and physiology.

Causal factors include the cause of TBI, categorised as above, with RTA details, height of fall (where relevant), presence and site(s) of major extracranial injury (defined as an injury that would require hospital admission in its own right) and presence of confirmed or suspected intoxication (with alcohol, drugs, etc.) at the time of injury.

Pre-injury status includes residence prior to admission to acute hospital, dependency prior to admission to acute hospital and deprivation status. Deprivation was assessed using the English Index of Multiple Deprivation (IMD) 201071 and the Welsh IMD 200872 for patients with a valid residential postcode. IMD quintiles of deprivation within Wales were assumed similar to the quintiles in England. Patients resident outside England and Wales were excluded from the assessment of deprivation, as sufficiently equivalent measures of deprivation were not available.

Date/time of TBI was categorised by the day of the week and hour of the day. Hospital source of admission to critical care was described by the location immediately prior to admission to the critical care unit. Age and sex were summarised both separately and by describing the overall age/sex distribution.

Neurological dysfunction includes the total GCS score, motor score component of the GCS score, pupil reactivity, Marshall CT classification56 and presence of traumatic SAH. GCS score, motor score and pupil reactivity were summarised for each time point at which they were recorded for all patients with an assessment available at that time point. Marshall CT classification and presence of traumatic SAH were assessed from the first CT scan following the injury.

Physiology includes the first recorded-at-hospital values of core physiology, arterial blood gases and laboratory parameters. Core physiological parameters recorded were temperature, systolic blood pressure, heart rate and oxygen saturation. Arterial blood gas parameters were partial pressure of oxygen (PaO₂), fraction of inspired oxygen (FiO₂), partial pressure of carbon dioxide (PaCO₂) and pH. Laboratory parameters were serum glucose, haemoglobin and platelet count.

Length of stay

Length of stay in critical care was summarised by the median and IQR of the total stay in critical care, including transfers between critical care units, and stratified by the survival status at final discharge from critical care. Similarly, LOS in acute hospital was summarised by the median and IQR of the total stay in acute hospital, including transfers between acute hospitals, and stratified by the survival status at final discharge from acute hospital.

Outcomes

The outcomes reported were mortality, neurological outcome and QOL. Mortality was reported at final discharge from critical care, final discharge from acute hospital, and at 6 months following the TBI. Survival over time to 6 months following the TBI was displayed using Kaplan–Meier plots. Neurological outcome was summarised by the GOSE category at 6 months, and dichotomised as favourable (GOSE category of good recovery or moderate disability) or unfavourable (GOSE category of death or severe disability; note that the questionnaire used for collection of GOSE responses did not distinguish vegetative state from lower severe disability). Quality of life at 6 months was described by the responses to the five domains of the EQ-5D-3L and by the patients' self-rated current health on a visual analogue scale from 0 (worst imaginable) to 100 (best imaginable).

Selection of patients

Of the 3210 patients in the RAIN study data set, 101 (3.1%) had no GCS score recorded, and 134 (4.2%) had a last pre-sedation GCS score of 15, resulting in a data set for analysis of 2975 patients.

Case mix

Causal factors

The most common causes of TBI were RTA (33%), fall (47%) and assault (12%), with 3% from other causes and 5% of unknown cause (Figure 7). RTA details are shown in Figure 8 and fall height in Figure 9.

FIGURE 7.

Cause of TBI.

FIGURE 8.

Road traffic accident details.

FIGURE 9.

Height of fall.

A major extracranial injury, sufficient to require hospital admission in its own right, was present in 41% and was particularly common for RTAs (70% compared with 28% for falls and 22% for assaults). Among assaults, major extracranial injuries were predominantly of the head and neck, whereas for falls and particularly for RTAs the sites of major extracranial injury were more widespread (Figure 10). Intoxication was either confirmed or suspected in almost half (45%), and varied by cause of TBI (26% for RTAs, 49% for falls and 74% for assaults; Figure 11).

FIGURE 10.

Major extracranial injury by site, overall and by cause of TBI. (a) Overall; (b) RTA; (c) fall; (d) assault.

FIGURE 11.

Confirmed or suspected intoxication at time of TBI, overall and by cause of TBI. (a) Overall; (b) RTA; (c) fall; (d) assault.

Pre-injury status

Table 6 reports the pre-injury status of patients, overall and by cause of TBI. The vast majority of patients were living independently prior to the TBI, indicated by 96% with a prior residence of ‘home’ and 95% able to live without assistance in activities of daily living. Figure 12 shows the distribution of deprivation by quintiles of IMD. There was a marked socioeconomic gradient that was strongest for assault (38% in the most deprived quintile vs 9% in the least deprived) and weakest for RTA (22% vs 19%).

Table 6 - Pre-injury status, overall and by TBI cause

Includes 95 patients with other causes of TBI and 152 with unknown cause.

Case mix factors	Overall (n = 2975a)	RTA (n = 976)	Fall (n = 1399)	Assault (n = 353)
Residence prior to admission, n (%) [N]	[2971]	[975]	[1398]	[351]
Home	2846 (95.8)	948 (97.2)	1345 (96.2)	330 (94.0)
No fixed abode or temporary abode	57 (1.9)	15 (1.5)	20 (1.4)	13 (3.7)
Residential place of work/education	27 (0.9)	8 (0.8)	3 (0.2)	4 (1.1)
Nursing home or equivalent	18 (0.6)	1 (0.1)	16 (1.1)	0 (0)
Health-related institution	13 (0.4)	2 (0.2)	10 (0.7)	0 (0)
Non-health-related institution	10 (0.3)	1 (0.1)	4 (0.3)	4 (1.1)
Prior dependency, n (%) [N]	[2895]	[944]	[1376]	[337]
Able to live without assistance in daily activities	2742 (94.7)	921 (97.6)	1258 (91.4)	333 (98.8)
Minor assistance with some daily activities	132 (4.6)	19 (2.0)	102 (7.4)	3 (0.9)
Major assistance with majority of/all daily activities	19 (0.7)	4 (0.4)	15 (1.1)	0 (0)
Total assistance with all daily activities	2 (0.1)	0 (0)	1 (0.1)	1 (0.3)

FIGURE 12.

Deprivation, overall and by cause of TBI. (a) Overall; (b) RTA; (c) fall; (d) assault.

Date/time of traumatic brain injury

Figure 13 shows the day of TBI, overall and by cause. TBI was more common at the weekend (37% overall), particularly for assaults (50%). TBI occurred throughout the day with a distribution that varied considerably by cause of TBI (Figure 14). For RTA, there were peaks corresponding to the morning and evening rush hours and a further peak in the late evening. Falls occurred throughout the day, particularly during waking hours. Assaults were much more common in the late evening and early hours of the morning, with almost half of all assaults occurring between 2300 and 0300 hours.

FIGURE 13.

Day of TBI, overall and by cause. (a) Overall; (b) RTA; (c) fall; (d) assault.

FIGURE 14.

Time of TBI, overall and by cause. (a) Overall; (b) RTA; (c) fall; (d) assault.

Age and sex

Patients were predominantly young (mean age 45 years overall) and male (76% overall; Table 8). The overall distribution of age was bimodal with peaks in the twenties and forties (Figure 15). This was, however, very much driven by different distributions by cause of TBI. For RTA, age of admissions peaked for males in their late teens and twenties, for falls the peak was among males in their forties, and for assaults, which were almost exclusively (95%) male, the peak was in the twenties.

Table 8 - Age and sex, overall and by TBI cause

SD, standard deviation.

Includes 95 patients with other causes of TBI and 152 with unknown cause.

Case mix factors	Overall (n = 2975a)	RTA (n = 976)	Fall (n = 1399)	Assault (n = 353)
Age [N]	[2975]	[976]	[1399]	[353]
Mean (SD)	44.7 (18.9)	36.3 (17.6)	53.4 (17.5)	33.9 (12.6)
Median (IQR)	44 (28 to 59)	31 (22 to 47)	54 (42 to 67)	30 (24 to 42)
Sex, n (%) [N]	[2975]	[976]	[1399]	[353]
Male	2263 (76.1)	727 (74.5)	985 (70.4)	334 (94.6)
Female	712 (23.9)	249 (25.5)	414 (29.6)	19 (5.4)

FIGURE 15.

Age and sex distribution, overall and by cause of TBI (a) Overall; (b) RTA; (c) fall; (d) assault.

Neurological dysfunction

Glasgow Coma Scale score was assessed and recorded pre-hospital for 2252 (76%) patients, including 9% with a pre-hospital GCS score of 15 that subsequently deteriorated such that the last pre-sedation GCS score was < 15. For 576 patients (19%), the pre-hospital GCS score was their last pre-sedation GCS score. The remaining 2399 patients all had a first at hospital GCS score recorded (within 12 hours following initial presentation at hospital), including 7% with a first recorded-at-hospital GCS score of 15 that subsequently deteriorated such that the last pre-sedation GCS score was < 15. For 1275 patients (43%), the first recorded-at-hospital GCS score was their last pre-sedation GCS score and the remaining 1124 (38%) patients had a subsequent last pre-sedation GCS score documented. The severity of TBI, as assessed by the GCS score, was generally worst for RTA, followed by assault and then fall (Figure 16). The motor score component of GCS score showed a U-shaped distribution with the highest proportions observed at the most extreme (low and high) values (Figure 17). Pupil reactivity was assessed pre-hospital, within 12 hours of initial presentation at hospital and on admission to the critical care unit for 52%, 90% and 97% of patients, respectively. For 20 (0.7%), 11 (0.4%) and 8 (0.3%) patients at each time point, respectively, neither pupil was able to be assessed, whereas the majority of missing data were not documented. Patients admitted following an RTA were more likely to have unreactive pupils than those admitted following either a fall or an assault (Figure 18).

FIGURE 16.

Glasgow Coma Scale score, overall and by cause of TBI. (a) Overall; (b) RTA; (c) fall; (d) assault.

FIGURE 17.

Glasgow Coma Scale motor score component, overall and by cause of TBI. (a) Overall; (b) RTA; (c) fall; (d) assault.

FIGURE 18.

Pupil reactivity, overall and by cause of TBI. (a) Overall; (b) RTA; (c) fall; (d) assault.

A first CT scan was available for 2817 (95%) patients and, of these, sufficient data were available to assign a Marshall CT classification for 2773 (98%; Figure 19). Very few patients (< 5%) had no abnormalities on the CT scan. The most common Marshall categories, each accounting for about one-third of patients, were diffuse injury II (basal cisterns present, midline shift 0–5 mm, no high- or mixed-density lesion > 25 ml) and evacuated mass lesion. Evacuation of a mass lesion was approximately twice as common among patients admitted following a fall (43%) or an assault (40%) than an RTA (20%). Overall, 56% of patients with a CT scan available had a traumatic SAH, and this was similar across the different causes of TBI (RTA 54%, fall 58%, assault 55%).

FIGURE 19.

Marshall CT classification, overall and by cause of TBI. (a) Overall; (b) RTA; (c) fall; (d) assault.

Physiology

Table 9 summarises the first recorded physiological parameters following presentation at hospital. Core physiological parameters were available for between 87% and 95% of patients, arterial blood gas parameters were available for between 72% and 80% of patients, and laboratory parameters were available for between 84% and 90% of patients. There was little difference in the degree of physiological derangement by cause of TBI.

Length of stay

Table 10 reports the LOS overall and by cause of TBI. The median total LOS in critical care was 7 days but this differed substantially between survivors (median 8 days) and non-survivors (median 3 days) of critical care. The critical care LOS distribution was similar for non-survivors across different causes of TBI, but for survivors was longer for patients admitted following an RTA (median 11 days) than for a fall or assault (median 7 and 6 days, respectively). The median total LOS in acute hospital was 30 days for survivors compared with 5 days for non-survivors. The LOS in acute hospital for survivors again varied substantially by cause of TBI, with the longest stay for patients admitted following an RTA (median 37 days), followed by for a fall (median 30 days), and for an assault (median 20 days).

Mortality

Mortality at final discharge from critical care was 18% and was highest for falls (21%) followed by RTAs (18%) and assaults (14%) (Table 11). At final discharge from acute hospital, mortality had increased to 24%, with the differential outcomes by cause of TBI maintained. Of the patients who survived to discharge from acute hospital, very few died before 6 months following the TBI, with 6-month mortality of 26%. The Kaplan–Meier survival curve (Figure 20) indicates that the majority of deaths occurred within the first 30 days following the TBI, with a slow decline in survival thereafter. The same was true across each cause of TBI, with the differential mortality by cause established by 30 days and maintained to 6 months.

Table 11 - Mortality at discharge from critical care and acute hospital and mortality and unfavourable outcome at 6 months following TBI, overall and by TBI cause

Includes 95 patients with other causes of TBI and 152 with unknown cause.

Denominator includes 24 patients who could not be assigned to a specific GOSE category, but where the questionnaire responses, although incomplete, were sufficient to indicate the patient did not have an unfavourable outcome.

Outcomes	Overall (n = 2975a)	RTA (n = 976)	Fall (n = 1399)	Assault (n = 353)
Mortality, deaths (%) [N]
At final discharge from critical care	546 (18.4) [2962]	171 (17.6) [972]	295 (21.2) [1394]	49 (13.9) [353]
At final discharge from acute hospital	705 (24.5) [2883]	197 (20.9) [944]	410 (30.2) [1358]	55 (15.9) [346]
At 6 months	748 (26.0) [2881]	204 (21.5) [948]	439 (32.0) [1370]	56 (16.8) [333]
Unfavourable outcome at 6 months,b n (%) [N]	1481 (61.2) [2422]	466 (57.3) [813]	776 (66.7) [1164]	137 (51.7) [265]

FIGURE 20.

Kaplan–Meier survival curves to 6 months following TBI. (a) Overall; (b) by cause of TBI.

Neurological outcome at 6 months

There was a substantial burden of disability at 6 months, with only 26% of surviving patients with a known GOSE category reporting a good recovery compared with 44% reporting severe disability (Figure 21). In addition to the 26% of patients who had died by 6 months, a similar proportion was known to have severe disability (Figure 22). When presented as a percentage of those with a known GOSE category, this corresponds to a rate of unfavourable outcome (death or severe disability) at 6 months of 61% (see Table 11). Outcomes were worst for falls, then for RTAs and assaults, with unfavourable outcome rates of 67%, 57% and 52%, respectively. The GOSE categories were reported for all patients with known survival status at 6 months (n = 2881) to avoid biasing the proportions towards deaths. Unfavourable outcome at 6 months is reported only for those with a known GOSE category (n = 2422) and, therefore, the reported rate of 61% is likely to represent an overestimate of the actual percentage with unfavourable outcome at 6 months because deaths will be over-represented owing to more complete follow-up. This is addressed in the following chapter using multiple imputation.

FIGURE 21.

Glasgow Outcome Scale – Extended at 6 months following TBI as a percentage of survivors with known GOSE category, overall and by cause of TBI. (a) Overall; (b) RTA; (c) fall; (d) assault.

FIGURE 22.

Glasgow Outcome Scale – Extended at 6 months following TBI as a percentage of all patients with known survival status, overall and by cause of TBI. Denominator includes 483 (17%) patients who were alive at 6 months following TBI but with unknown GOSE category. (a) Overall; (b) RTA; (c) fall; (d) assault.

Quality of life at 6 months

European Quality of Life-5 Dimensions (3-level version) responses were available for 1132 patients (53% of those known to be alive at 6 months). Figure 23 summarises the EQ-5D-3L responses for each of the five domains. The EQ-5D-3L responses indicated substantial issues with QOL at 6 months, including 70% of patients reporting problems with performing usual activities, around 60% reporting problems with pain or discomfort and anxiety or depression, around 50% reporting problems with mobility and around 35% reporting problems with self care. There was little difference across the EQ-5D-3L domains by cause of TBI, although those admitted following an RTA reported slightly higher proportions in the worst categories for physical functioning (mobility, self care and usual activities), whereas patients admitted following assault reported slightly higher proportions in the worst categories for pain or discomfort, and anxiety or depression.

FIGURE 23.

European Quality of Life-5 Dimensions (3-level version) responses by domain, overall and by TBI cause (a) Mobility; (b) self-care; (c) usual activities; (d) pain/discomfort; (e) anxiety/depression.

Figure 24 summarises the self-rated health from the visual analogue scale for the 1078 patients who completed this section of the questionnaire, overall and by cause of TBI. The overall median self-rated health was 70 (IQR 50 to 85). Self-rated health was lower for patients following assault (median 60, IQR 45 to 80) than for either RTA or fall (both 70, 50 to 85).

FIGURE 24.

Self-rated health at 6 months following TBI, overall and by cause (a) Overall; (b) RTA; (c) fall; (d) assault.

Principal findings

Patients with acute TBI admitted to UK critical care units participating in the RAIN study were predominantly admitted following RTA (33%), fall (47%) or assault (12%). Characteristics and outcomes varied markedly by cause of TBI. The RAIN study demonstrated a substantial burden of poor neurological outcomes and QOL 6 months after TBI for adult patients admitted to critical care. Mortality at discharge from acute hospital was 16% for assault, 21% for RTA and 30% for falls, rising to 17%, 22% and 32%, respectively, at 6 months following the TBI. Of survivors at 6 months with known a known GOSE catgory, 44% had severe disability, 30% had moderate disability, and only 26% had made a good recovery. When combined with the 26% mortality at 6 months, 61% of patients with known outcome had an unfavourable outcome (death or severe disability) at 6 months. In addition, between 35% and 70% of survivors reported problems across the five domains of the EQ-5D-3L. Despite this, self-rated health was remarkably good, with an overall median of 70 out of 100 and 22% of respondents rating their current health as 90 out of 100 or higher.

Comparison with other studies

Little literature exists reporting specifically on multicentre cohorts of patients with acute TBI receiving critical care. A previous study from the CMP reported on over 11,000 admissions to 169 general critical care units (including combined neuro/general critical care units) and two dedicated neurocritical care units between 1995 and 2005 with a primary reason for admission to the critical care unit of ‘primary brain injury’, ‘extradural haematoma’ or ‘subdural haematoma’. 6 Three-quarters of admissions were male and the mean age was 44 years, corresponding very closely with the overall figures in the RAIN study. Patients were followed up to ultimate discharge from acute hospital, with mortality of 33.5% – considerably higher than the 24.5% acute hospital mortality observed in the RAIN study. A prospective study from Australia and New Zealand reported on 635 patients admitted to the critical care units of 16 major trauma centres following acute TBI. 73 The age and sex distribution was again similar to that of the RAIN study (mean age 42 years, 74% male). However, there was a different distribution of cause of TBI with 61% RTA, 25% falls, 8% assault and 8% other/unknown; 57% of TBIs were severe (GCS score 3–8) based on the first at hospital GCS score compared with 46% of those in the RAIN study with a first at hospital GCS score recorded. A remarkably high proportion of patients had no visible pathology on CT scan (24% overall including 23% of severe TBI) and of those with abnormal CT scans, diffuse injuries predominated (63%), consistent with the higher proportion of patients admitted following RTA. Mortality was similar to that in the RAIN study (15.1% vs 18.4% at discharge from the critical care unit and 24.4% vs 26.0% at 6 months following TBI).

The proportion of survivors with severe disability at 6 months in the RAIN study (44%) is high compared with previous studies, for example 26% for patients with severe TBI from a study in the Netherlands74 and 27% among critically ill patients following TBI in Australia and New Zealand.73 Some of these patients will still be on a trajectory of improving neurological outcome; the proportion with a good recovery increased from 38% at 6 months to 46% at 12 months in the study from the Netherlands,74 and from 27% to 42% in the study from Australia and New Zealand. 73 However, for many patients, these are likely to represent long-term disabilities, for example a long-term follow-up study in Glasgow found that of 96 patients with severe disability at 1 year following TBI, 44 (46%) had died, 30 (31%) still had severe disability, 15 (16%) had moderate disability and seven (7%) had made a good recovery when followed up 5–7 years following injury. 75

Summary of risk prediction models

Following the updated systematic review (see Chapter 2) and review by the RAIN Study Steering Group, three families of risk prediction models were selected for validation in the RAIN study – the models of Hukkelhoven et al. (Hukkelhoven models);48 the MRC CRASH trial collaborators (CRASH models);30 and Steyerberg et al. (IMPACT models). 35 A total of 10 models were included in the RAIN study validation (Figure 25): four predicting mortality at 6 months (one Hukkelhoven and three IMPACT) and six predicting unfavourable outcome at 6 months (one Hukkelhoven, two CRASH and three IMPACT). Details of the development and validation of these models were reported in Chapter 2. A brief summary of the development data sources, inclusion/exclusion criteria and included variables is presented for each model below.

FIGURE 25.

Overview of risk prediction models.

Data sources and risk factors definitions

Patients were selected from the RAIN study database if their last pre-sedation GCS score was < 15. Definitions for fields in the RAIN study are provided in Appendix 3. Risk factor variables were calculated from the raw data as follows:

• Age Age in whole years at the time of TBI, calculated from the date of birth and date of TBI.

• GCS score The last recorded GCS score prior to sedation, which may have been either the pre-hospital GCS score, the first recorded-at-hospital GCS score or a subsequent measurement prior to sedation and admission to the critical care unit.

• Motor score The motor score component of the last pre-sedation GCS score.

• Pupil reactivity The first recorded-at-hospital pupil reactivity, if available, or the pre-hospital pupil reactivity if no first at hospital value was available. Pupil reactivity was recorded for the left and right eye separately and combined as either ‘both reactive’, ‘one reactive’ or ‘neither reactive’. If only one eye could be assessed, pupil reactivity was assigned as ‘both reactive’ if this eye was reactive, and ‘neither reactive’ if this eye was not reactive.

• Major extracranial injury Recorded as a specific field in the RAIN study data set, defined as an injury requiring hospital admission in its own right.

• Pre-hospital hypoxia Defined as either a pre-hospital oxygen saturation of < 90%, or pre-hospital hypoxia strongly suspected if there was clinical evidence of hypoxia (e.g. tension pneumothorax) but no pre-hospital oxygen saturation was recorded.

• Pre-hospital hypotension Defined as either a pre-hospital systolic blood pressure of < 90 mmHg, or pre-hospital hypotension strongly suspected if there was clinical evidence of hypotension (e.g. multiple trauma with massive blood loss) but no pre-hospital systolic blood pressure was recorded.

• Marshall CT classification Based on the first CT scan following the TBI, and defined as (1) diffuse injury I – no visible pathology seen on CT scan (first CT scan result normal); (2) diffuse injury II – basal cisterns present, midline shift of 0–5 mm, no high- or mixed-density lesion of > 25 ml; (3) diffuse injury III – basal cisterns compressed or absent, midline shift of 0–5 mm, no high- or mixed-density lesion of > 25 ml; (4) diffuse injury IV – midline shift of > 5 mm, no high- or mixed-density lesion > 25 ml; (5) evacuated mass lesion – any high- or mixed-density lesion of > 1 ml – surgically evacuated; (6) non-evacuated mass lesion of > 25 ml – high- or mixed-density lesion of > 25 ml not surgically evacuated. 56

• Traumatic SAH Recorded as a specific field in the RAIN study data set.

• One or more small petechial haemorrhages Recorded as a specific field in the RAIN study data set.

• Obliteration of the third ventricle or basal cisterns Third ventricle recorded as ‘obliterated’ or basal cisterns recorded as ‘absent’.

• Midline shift of > 5 mm Recorded as a specific field in the RAIN study data set.

• Non-evacuated haematoma Any high- or mixed-density lesion of > 1 ml – not surgically evacuated.

• Extradural haematoma Recorded as a specific field in the RAIN study data set.

• First at hospital glucose Recorded as a specific field in the RAIN study data set.

• First at hospital haemoglobin Recorded as a specific field in the RAIN study data set.

Handling of missing data

Missing data in the RAIN study data set were addressed with multiple imputation. 76,77 Multiple imputation aims to allow for the uncertainty about missing data by creating multiple copies of the data set with the missing values in each data set replaced by imputed values, sampled from their predicted distribution. 78 Analyses are conducted on each of the imputed data sets and combined together. As GOSE category is conditional on survival status, the imputation was conducted in two stages. In the first stage, imputation models were specified for risk factors and mortality at 6 months, according to observed covariates. In the second stage, for each of the previously imputed data sets, imputation models were specified for GOSE category for those patients who were either known to be alive at 6 months or were predicted to be alive by the first stage imputation model. Each multiple imputation model assumed that the data were ‘missing at random’, i.e. conditional on the variables included in the imputation models. 76 Each imputation model considered including, for example baseline characteristics (e.g. age) and resource use at 6 months (e.g. LOS in critical care). Five imputed data sets were generated for each stage of the imputation,78 giving 25 imputed data sets in total. All multiple imputation models were implemented in R (The R Foundation for Statistical Computing, Vienna, Austria: www.r-project.org) using Multivariate Imputation by Chained Equations (MICE). 79

Validation data sets

For the primary analyses, each model was validated in the multiply imputed data sets, using (1) all patients in the RAIN study with a last pre-sedation GCS score of < 15 and (2) only those patients meeting the original inclusion criteria for each model. As secondary analyses, and to reflect use of the models in real life settings, the validation was repeated in the original RAIN study data set prior to imputation using (3) all patients; (4) only those patients meeting the original inclusion criteria for each model; and (5) only those patients meeting the original inclusion criteria for each model and with complete data for all variables in the model. For analyses 3 and 4, missing values were assumed to be in the lowest risk category and laboratory data were singly imputed to a normal value.

As a sensitivity analysis, the analysis for models predicting unfavourable outcome in all patients in the RAIN study data set [data set (1) above] was repeated excluding those patients assigned to the GOSE category of severe disability that was either pre-existing or not due to the TBI.

Statistical analysis

The case mix and outcomes of patients reported for the original development sample for each family of models (Hukkelhoven, CRASH and IMPACT) was compared with those for the patients in the RAIN study data set that met the original inclusion criteria for the particular family of models, and with all patients in the RAIN study data set. The case mix factors included in these comparisons were those included in the papers reporting development of the models. Owing to the level of detail of reporting in the papers, it was possible to summarise the overall case mix and outcome for only each full development sample, and not specifically to the patients actually included in each development sample for each individual model.

The completeness of data for each risk factor included in each of the risk prediction models and the outcomes of mortality and unfavourable outcome at 6 months was assessed in the full RAIN study data set. Univariable analyses were conducted to assess the relationship between each risk factor and the outcomes.

Each risk prediction model was then validated in each of the five RAIN study validation data sets using measures of calibration, discrimination and overall fit, as described below. For the validation of risk prediction models for mortality at 6 months, the five data sets from the first stage imputation were used. For the validation of risk prediction models for unfavourable outcome at 6 months, the 25 data sets from the second stage imputation were used. In each case, measures of model performance were calculated in each imputed data set and combined using Rubin's rules,76 which recognise uncertainty both within and between imputations.

Discrimination describes the ability of the model to correctly separate the patients into different groups (survivors from non-survivors or favourable from unfavourable outcomes). Discrimination was assessed by the c-index,80 which is equivalent to the area under the ROC curve. 81 When c = 1 there is perfect discrimination between the groups (i.e. every patient that died has a higher predicted risk than every patient that survived, or every patient with an unfavourable outcome has a higher predicted risk than every patient with a favourable outcome) and when c = 0.5 the discrimination is no better than chance.

Calibration describes the degree of correspondence between the probability predicted by the model, and the observed proportion with the outcome. Calibration was assessed by graphical plots of observed against expected risk in 10 equal-sized groups by predicted risk. The Hosmer–Lemeshow test was used to test the hypothesis of perfect calibration. 82 However, the Hosmer–Lemeshow test does not provide a measure of the magnitude of miscalibration and is highly sensitive to sample size. 83 Therefore, the magnitude and direction of miscalibration was assessed using Cox's calibration regression. 84 Cox's calibration regression fits a logistic regression model to the predicted log odds from the risk prediction model to estimate the intercept, α, and slope, β. If the model is perfectly calibrated then α = 0 and β = 1. The value of α represents the calibration at a predicted risk of 0.5, and calibration more generally if β = 1. If 0 < β < 1, the predictions vary too much; if β > 1, the predictions ‘show the right general pattern of variation but do not vary enough’.

The overall fit (or accuracy) of the model describes how close the predictions are to the actual outcomes for individual patients. Overall fit was assessed using Brier's score, the mean squared error between the outcome and prediction. 85 Lower values of Brier's score represent better fit, with perfect predictions corresponding to B = 0 and a constant prediction of 0.5 for all patients corresponding to B = 0.25.

Inter-rater reliability of computerised tomography scan reporting

It is essential to know whether the data obtained from local reporting of CT scans are adequate for accurate risk adjustment, as this will have significant implications on the practicability of using any particular risk prediction model. Data collectors were therefore asked to record appropriate identifiers to allow access to CT scans for rereview at a later date, and were requested not to discard or destroy the films or digital imaging data for these patients until 5 years after entry into the RAIN study. Copies of the first CT scan following presentation at hospital were requested for a randomly selected sample of 10% of patients, weighted to include more patients from outside neuroscience centres, where patient throughput was lower, and including at least one patient from each site participating in the RAIN study (provided at least one patient had CT data available). Data collectors were requested to send anonymised CT scans to the WBIC. Images were transferred either electronically using the Image Exchange Portal or by recorded delivery or secure courier. The images were centrally viewed and assessed by four neurosurgical specialist registrars/research fellows experienced in assessing CT scans, working under the supervision of two of the clinical co-investigators (DKM, PJH). Each CT scan was assessed independently by two raters, working in two teams of two, and data were recorded using the same secure, web-based data entry system as for the original RAIN study data entry. Each rater was blinded to the site from which the CT scan originated, the original RAIN study data reported by that site, and the assessment of their co-rater.

Inter-rater reliability (reproducibility) was assessed by κ-statistics86 for each individual CT field in the RAIN study data set between:

1. each group of three ratings comprising the original RAIN study data and the two independent ratings from WBIC; and

2. each pair of ratings: RAIN compared with WBIC rater 1, RAIN compared with WBIC rater 2 and WBIC rater 1 compared with WBIC rater 2, separately for the two teams of two raters within WBIC.

Subgroup analyses were conducted to compare inter-rater reliability by the specialty and grade of assessor in the original RAIN study data, and by whether the scan was from a neuroscience centre or a non-neuroscience centre.

Confidence intervals for κ-statistics were calculated using the method of Donner and Eliasziw87 for two raters rating a binary outcome, the method of Zou and Donner88 for more than two raters rating a binary outcome, and using bootstrap resampling for more than two raters rating an outcome with more than two levels. 89

All statistical analyses were conducted using Stata/SE version 10.1 (StataCorp LP, College Station, TX, USA).

Comparison of model development samples with the RAIN study data set

Tables 12–14 describe the case mix and outcomes of the development samples for the families of Hukkelhoven, CRASH and IMPACT models compared with patients from the RAIN study, both those meeting the original inclusion criteria for the respective model and all patients in the RAIN study data set. Compared with patients in the Hukkelhoven development sample (see Table 12), patients in the RAIN study were, on average, older (mean 45 years vs 33 years), more likely to have had a fall (47% vs 17%) and less likely to have had an RTA (33% vs 58%) as the cause of their TBI, had worse GCS motor scores (32% vs 13% with motor score 1 or 2) and were more likely to have had either an evacuated or non-evacuated mass lesion (Marshall class 5/6, 50% vs 32%). They were, however, less likely to have either pre-hospital hypoxia (13% vs 21%) or hypotension (7% vs 18%). These differences remained, but were generally reduced, after applying the Hukkelhoven inclusion criteria to the RAIN study data set, with the exception of the GCS motor score in which the difference became more extreme (42% vs 13% with motor score 1 or 2). Mortality at 6 months was slightly higher in the RAIN study than in the Hukkelhoven development sample (26% vs 22%) but was similar (23% vs 22%) after applying the inclusion criteria. Unfavourable outcome at 6 months was substantially higher in the RAIN study both before (61% vs 40%) and after (60% vs 40%) applying the inclusion criteria.

Compared with patients in the CRASH development sample (see Table 13), patients in the RAIN study were, on average, slightly older (mean 45 vs 41 years), more likely to have had a fall (47% vs 20%) and less likely to have had an RTA (33% vs 50%) as the cause of their TBI, had worse GCS scores (59% vs 44% severe TBI, GCS score 3–8; 15% vs 33% mild TBI, GCS score 13–14), were more likely to have unreactive pupils (13% vs 9%) and major extracranial injury (41% vs 23%), and had higher rates of all risk factors on CT. The rate of unfavourable outcome at 6 months was substantially higher in the RAIN study (61% vs 39%). There was very little change after applying the CRASH inclusion criteria to the RAIN study data set, as the majority of patients in the RAIN study met CRASH inclusion.

Compared with patients in the IMPACT development sample (see Table 14), patients in the RAIN study were, on average, older (median 44 vs 30 years), more likely to have GCS motor scores at the extremes of the range (24% vs 16% with motor score 1 and 47% vs 30% with motor score 6), and were less likely to have unreactive pupils (13% vs 21%), hypoxia (13% vs 21%) and hypotension (7% vs 18%). The distributions of CT findings and laboratory results were broadly similar. Mortality at 6 months was similar (26% vs 28%) but the rate of unfavourable outcome at 6 months was higher in the RAIN study (61% vs 48%). There was little change after applying the IMPACT inclusion criteria to the RAIN study data set.

Data completeness

Table 15 reports the data completeness for risk factors and outcomes in the RAIN study data set. Age and GCS score were 100% complete (due to their requirement for meeting RAIN study inclusion criteria), and major extracranial injury was missing for only one patient (and assumed absent). All other risk factors and outcomes were included in the multiple imputation. The core risk factors were available for between 92.8% (pupil reactivity) and 97.7% (motor score) of patients. The first CT scan was available for 2817 patients (94.7%) and the majority of these had all CT fields recorded. Overall, 93.2% of CT scans could be assigned to a Marshall class; of the remaining 6.8%, 5.3% could not be assigned as no CT data were available and 1.5% were due to individual missing CT fields. Laboratory variables were the least complete with 83.9% completeness for glucose and 90.2% for haemoglobin. There was 96.8% follow-up for mortality at 6 months (being unknown only for patients who were homeless or a non-UK resident and could not be linked with the MRIS database; see Chapter 3. There was 81.4% follow-up overall for unfavourable outcome at 6 months (see Chapter 4). Following multiple imputation, the overall mortality at 6 months was estimated to be 25.7% and the rate of unfavourable outcomes 57.4% (note that this is lower than the observed rate owing to being more likely to observe outcomes for non-survivors).

Univariable associations between risk factors and outcomes

Tables 16–18 report the univariable associations between risk factors and outcomes for core, CT and laboratory variables. The majority of core risk factors demonstrated similar patterns of associations with outcomes as in the risk prediction models (see Table 16). Mortality at 6 months was relatively flat below the age of 40 years and increased from the age of 40 years (reflecting the structure used in the CRASH models), whereas unfavourable outcome increased across the whole range of age (reflecting the structure used in the IMPACT models). The range of outcomes across categories of GCS motor score (16–40% for mortality and 45–75% for unfavourable outcome) was almost as wide as that seen across the total GCS score (15–43% and 44–78%), reflecting the consensus that the majority of prognostic information in the GCS score, for patients with TBI, comes from the motor score component. Non-reactive pupils and pre-hospital hypoxia and hypotension were associated with substantially worse outcomes. However, major extracranial injury was not associated with any increase in mortality or unfavourable outcome at 6 months.

For CT variables (see Table 17), mortality increased substantially across the four categories of diffuse injury in the Marshall CT classification, was lower for category 5 (evacuated mass lesion) and substantially higher for category 6 (non-evacuated mass lesion > 25 ml). This pattern does not appear to support the modelling approach taken in both the Hukkelhoven and IMPACT models of combining categories 5 and 6, although the relationship may be different once adjusted for other variables (e.g. the separate variable for extradural haematoma included in the IMPACT models). All other CT factors were associated with increased mortality and unfavourable outcome except small petechial haemorrhages and extradural haematoma. There was no univariable association between small petechial haemorrhages and outcomes. However, it should be noted that in the CRASH CT model, this variable was associated only with a small, and not statistically significant, increase in risk in high-income countries and was retained in the model because of the more substantial effect observed in low- and middle-income countries. Extradural haematoma was associated with a decreased risk of mortality and unfavourable outcome, consistent with the negative coefficient for this variable in the IMPACT Extended and IMPACT Lab models.

For the laboratory variables (see Table 18), there was a substantial increasing risk of both mortality and unfavourable outcome at 6 months associated with both increasing glucose and decreasing haemoglobin, consistent with the relationships in the IMPACT Lab models.

External validation of the risk prediction models

For the primary analysis, following multiple imputation, all 2975 patients were included in the validation of the risk models among all patients in the RAIN study data set and between 1868 (63%) and 2711 (91%) were included in the validation among patients eligible for each model (Table 19). Owing to multiple imputation of missing data, the sample sizes were the same for the validation data sets for mortality and for unfavourable outcome for each model, where relevant.

The discrimination (c-index) of the four models predicting mortality at 6 months is reported in Table 20. Discrimination was similar for the Hukkelhoven, IMPACT Extended and IMPACT Lab models (c-index 0.78) and higher than that for the IMPACT Core model (0.75). Inspection of the calibration plots (Figure 26) showed good calibration for the Hukkelhoven and IMPACT Lab models, whereas the IMPACT Core and Extended models tended to overestimate the risk of mortality at 6 months (i.e. observed mortality was lower than predicted). This visual interpretation is supported by the measures and tests of calibration (Table 21). Brier's score (Table 22), summarising the overall fit of the predictions, was best for the Hukkelhoven model and worst for the IMPACT Core model. There was little difference in any measures of model performance between assessment among all patients and among those eligible for the model.

FIGURE 26.

Calibration plots: mortality at 6 months. Each point represents the observed mortality (with 95% CI) plotted against the predicted mortality from the model for 10 equal-sized groups based on the predicted mortality. Bars at the foot of each figure (‘rug plots’) illustrate the distribution of predictions for survivors and non-survivors.

Each point represents the observed mortality (with 95% CI) plotted against the predicted mortality from the model for 10 equal-sized groups based on the predicted mortality. Bars at the foot of each figure (‘rug plots’) illustrate the distribution of predictions for survivors and non-survivors.

The discrimination of the models for predicting unfavourable outcome at 6 months was worse than for mortality at 6 months, with c-index values ranging from 0.69 to 0.71 (Table 23). The IMPACT Lab model had slightly better discrimination (0.714), followed by the models making use of CT findings (Hukkelhoven, CRASH CT and IMPACT Extended; c-index 0.708) with the models using only core variables (CRASH Basic and IMPACT Core) having the worst discrimination (c-index 0.69–0.70). All models were poorly calibrated, substantially underestimating the risk of unfavourable outcome at 6 months, particularly at low predicted risk (Figure 27 and Table 24). The lack of calibration resulted in very poor values for Brier's score – little better than (and, in the case of the CRASH Basic model, worse than) the value of 0.25 that would be observed from predicting a constant risk of 0.5 for all patients (Table 25). The sensitivity analysis excluding patients with severe disability – either pre-existing or not due to the TBI – resulted in very small improvements in discrimination and no discernible change to calibration or overall fit (Table 26). Results were very similar when the analyses were repeated for the secondary analyses in the data sets prior to multiple imputation (see Appendix 6).

FIGURE 27.

Calibration plots: unfavourable outcome at 6 months. Each point represents the observed proportion with unfavourable outcome (with 95% CI) plotted against the predicted risk of unfavourable outcome from the model for 10 equal-sized groups based on the predicted risk. Bars at the foot of each figure (‘rug plots’) illustrate the distribution of predictions for patients with favourable and unfavourable outcomes.

Each point represents the observed proportion with unfavourable outcome (with 95% CI) plotted against the predicted risk of unfavourable outcome from the model for 10 equal-sized groups based on the predicted risk. Bars at the foot of each figure (‘rug plots’) illustrate the distribution of predictions for patients with favourable and unfavourable outcomes.

Inter-rater reliability of computerised tomography scan reporting

Computerised tomography scans were available for 312 (91%) of 342 patients randomly selected for inclusion in the substudy evaluating inter-rater reliability of CT scan reporting, including at least one scan from 64 of the 66 critical care units that recruited at least one patient to the RAIN study. The main reason cited for failure to provide a CT scan was the need to obtain the scan from a referring hospital. κ-Statistics ranged from 0.19 to 0.78 across the individual CT fields (Table 27). The majority of fields used in the risk model were found to have moderate to good inter-rater reliability (κ-statistic 0.4–0.8). The worst inter-rater reliability was found for the presence of small petechial haemorrhages, which may partly explain why no association was found between this factor and outcomes. Agreement between the WBIC assessors (WBIC1 vs WBIC2 and WBIC3 vs WBIC4; median κ-statistic 0.70, IQR 0.58 to 0.76) was generally better than between the original RAIN study data and the WBIC assessors (RAIN vs WBIC1, RAIN vs WBIC2, etc.; 0.52, 0.37–0.62). κ-Statistics for subgroup analyses are reported in Tables 28 and 29. There was little discernible variation in inter-rater reliability of CT scan assessments according to the specialty and grade of the assessor for the original RAIN study data (see Table 28). CT scans that originated from RAIN study critical care units within neuroscience centres had similar inter-rater reliability compared with CT scans that originated from non-neuroscience centres (see Table 29).

Principal findings

The RAIN study has demonstrated that the Hukkelhoven, CRASH and IMPACT risk prediction models for mortality and unfavourable outcome at 6 months have acceptable levels of discrimination when applied in a UK cohort of adult patients admitted to critical care units following acute TBI. Although the Hukkelhoven and IMPACT Lab models for mortality at 6 months had good calibration, the other IMPACT models for mortality at 6 months overpredicted risk, and all models for unfavourable outcome at 6 months underpredicted risk particularly at the lower end of the risk spectrum.

Strengths and weaknesses

The main strengths of the RAIN study are the large, representative sample with high levels of data completeness and the rigorous statistical methods. The RAIN study included 84% of all neuroscience centres in the UK plus representation from the important, but often overlooked, cohort of critically ill patients with acute TBI managed outside neuroscience centres. Although the completeness of follow-up for unfavourable outcome at 6 months (81%) was lower than ideal, this reflects the difficulties inherent in following up such a population (see Chapter 4). Furthermore, consent was obtained at the point of follow-up rather than prior to enrolment (as would be the case in an RCT), so patients who declined follow-up were still included within the initial study population. The low follow-up rate was addressed by implementing a two-stage multiple imputation process – first imputing risk factors and mortality, and subsequently, for those alive or imputed to be alive at 6 months, imputing the GOSE categories.

Strengths and weaknesses in comparison with other studies

The RAIN study is the first prospective external validation of these risk prediction models, with all previous validations having been undertaken using existing data sources from either RCTs or trauma registries. Consequently, although many previous validation studies have been restricted to validating modified and refitted versions of the models due to differences between the available data for validation and the data used for model development, the RAIN study was able to define variables in advance to accurately implement all risk models as originally reported.

The discrimination of the risk prediction models in the RAIN study was generally within the range observed in previous development and validation populations. The Hukkelhoven models had c-indices of 0.78 and 0.71 for mortality and unfavourable outcome at 6 months, respectively, in the RAIN study compared with 0.78 and 0.80 in the original development sample. 48 The Hukkelhoven models have been validated in four external existing data sources. The model development paper reported validation of the model for mortality at 6 months in observational data sets from the EBIC and the TCDB with a c-index of 0.87 and 0.89, respectively. 48 The model for unfavourable outcome at 6 months was validated in the EBIC data set only (as the timing of GOS reporting in the TCDB data set was variable) with a c-index of 0.83. Subsequently, Hukkelhoven et al. 90 presented validation data from one further data source, the Selfotel RCT, with a c-index of 0.74 for both mortality and unfavourable outcome at 6 month. A recently reported study from a 10-year cohort of patients admitted to a single UK neuroscience centre reported a c-index around 0.83 for both mortality and unfavourable outcome at 1 year. 91

The CRASH Basic model and CT model for unfavourable outcome at 6 months had c-indices of 0.70 and 0.71, respectively, in the RAIN study compared with 0.81 and 0.83 in the original development sample. 30 The model development paper reported validation of modified versions of both models (excluding major extracranial injury and small petechial haemorrhages) in the IMPACT database with a c-index of 0.77 for both models. The CRASH models have subsequently been validated in three further existing data sources from previous RCTs in a joint study by the IMPACT, CRASH and TARN groups. 92 These were, again, modified versions of the original models, refitted using only patients in the CRASH trial with GCS score of ≤ 8 and replacing total GCS with motor score and the CRASH CT variables with Marshall classification, resulting in a model structure substantially more similar to the Hukkelhoven and IMPACT models. The c-index for the modified CRASH Basic model in these three data sets was 0.68, 0.74 and 0.76, respectively; the c-index for the modified CRASH CT model was 0.71 in the only data set for which this could be calculated.

The IMPACT Core, Extended and Laboratory models for mortality at 6 months had c-indices of 0.75, 0.78 and 0.78, respectively, in the RAIN study compared with values of 0.66–0.84, 0.71–0.87 and 0.72–0.80 in cross-validation among the original data sets used for model development. 35 The corresponding figures for the models for unfavourable outcome at 6 months were 0.69, 0.71 and 0.71, respectively, in the RAIN study compared with 0.74–0.82, 0.73–0.84 and 0.75–0.82 in cross-validation among the original development data sets. The model development paper reported validation of the IMPACT Core model and a modified version of the Extended model (the ‘Core + CT’ model, excluding hypoxia, hypotension and extradural haematoma) in the CRASH trial data set with c-indices of 0.78 and 0.80, respectively, for mortality at 6 months, and the same results for unfavourable outcome at 6 months. The IMPACT models have subsequently been validated in six further data sets – a study from a single trauma centre in the USA,93 and the joint study from the IMPACT, CRASH and TARN groups using data from three RCTs (as for the validation of the CRASH models) and two observational studies. 92 The single-centre US study reported c-indices of 0.78, 0.83 and 0.83 for mortality at 6 months, and 0.76, 0.79 and 0.76 for unfavourable outcome at 6 months for the Core, Extended and Laboratory models, respectively. 93 The IMPACT/CRASH/TARN study reported a c-index for the IMPACT Core model ranging from 0.65–0.83 for mortality at 6 months across the five data sets, and 0.66–0.76 for unfavourable outcome at 6 months across four data sets in which this outcome was available. 92 The best discrimination for mortality was from the UK TARN data set; however, the outcome assessed in this data set was mortality at discharge from hospital not mortality at 6 months. The IMPACT Extended model had a c-index of 0.86 for hospital mortality in the TARN data set, and 0.71 for both mortality and unfavourable outcome at 6 months in the one RCT data set in which it could be applied. The IMPACT Lab model had c-index of 0.71 and 0.70 for mortality and unfavourable outcome at 6 months, respectively, in the same data set.

In the RAIN study, the presence of major extracranial injury was found not to be associated with an increased risk of either mortality or unfavourable outcome at 6 months and this is at variance with some previous studies. A recent meta-analysis on the effect of major extracranial injury on mortality in TBI found an inverse relationship between major extracranial injury and mortality according to the severity of the TBI with odds ratios of 2.14, 1.46 and 1.18 in mild (GCS score of 13–14), moderate (GCS score of 9–12) and severe (GCS score of 3–8) TBI, respectively. 94 Although there are patients with a range of severity of TBI within the RAIN study, approximately two-thirds had GCS scores of 3–8 and all patients were admitted to a critical care unit, which may also represent an additional marker of severity. One may therefore anticipate that the effect of major extracranial injury within the RAIN study would be small. It is also possible, however, that the lack of effect of major extracranial injury in the RAIN study may be due to a lack of consistent application of the definition, taken from the CRASH trial, which was somewhat subjective. Collection of more detailed injury severity scoring, although time consuming, would permit more detailed investigation of the effect of major extracranial injury both more consistently across sites and according to severity.

Advantages and disadvantages of the alternative risk prediction models

In terms of the statistical assessment of model performance, there was very little to choose between models of similar complexity from Hukkelhoven, CRASH and IMPACT. The best discrimination overall was from the IMPACT Lab model – the only one of the models to include laboratory parameters – however, the improvement in performance over the models of the next level of complexity (Hukkelhoven, CRASH CT, IMPACT Extended) was very small. There was a larger difference in performance between these models and the simplest models using core data only (CRASH Basic and IMPACT Core), suggesting that there is important prognostic information within the CT scan and the presence/absence of pre-hospital hypoxia/hypotension. The substudy on inter-rater reliability of CT scan reporting suggested that the CT findings included in the models could be assessed with acceptable reliability. The CRASH models included two variables – major extracranial injury and small petechial haemorrhages – that did not demonstrate any association with outcomes in the RAIN study. This may be seen as a reason to prefer the Hukkelhoven and IMPACT Extended models over the CRASH CT model; however, despite the inclusion of these apparently non-prognostic fields, the overall performance of the CRASH CT model was similar. For the subsequent analyses within the RAIN study, we therefore selected the IMPACT Lab model as the primary model for risk adjustment in the base-case analyses, with the CRASH CT model used for sensitivity analyses. The CRASH CT model was chosen over the Hukkelhoven model for sensitivity analyses as it included more substantially different predictor variables from the IMPACT Lab model.

Alternative care pathways in the NHS for adult patients with traumatic brain injury

There are many possible pathways for patients admitted to NHS critical care units following acute TBI, but the vast majority of patients can broadly be classified into one of four pathways (Figure 28). Patients may either present at a non-neuroscience centre (pathways A, B and C), or directly to a neuroscience centre (pathway D). Patients who present to a non-neuroscience centre may either receive all of their critical care within that centre (pathway A) or transfer to a neuroscience centre either directly from the emergency department (pathway B), or critical care within the non-neuroscience centre (pathway C). (Note that pathway C will include both patients for whom the initial decision is to transfer to a neuroscience centre but who receive a period of critical care within the non-neuroscience centre either for stabilisation prior to transfer or while a suitable bed is located, and also patients for whom the initial decision is to manage within the non-neuroscience centre but a subsequent decision is made to transfer, e.g. owing to deterioration.) Patients who present directly to a neuroscience centre are generally managed within a neurocritical care unit in that neuroscience centre (pathway D), which may be either a dedicated neurocritical care unit or a combined neuro/general critical care unit. (Occasionally, when beds are unavailable patients may be transferred to a general critical care unit in a neuroscience centre.)

FIGURE 28.

Simplified patient pathways.

Comparators for addressing research objective 1

Research objective 1 considered alternative ways of organising and delivering neurocritical care services within a neuroscience centre for patients admitted following acute TBI (Figure 29). The population of interest was defined by all RAIN study patients who were admitted to critical care within a neuroscience centre, regardless of whether they initially presented at a neuroscience centre (pathway D) or were transferred from a non-neuroscience centre (pathways B or C). Patients who received all their critical care within a non-neuroscience centre were excluded (i.e. pathway A). The decision problem contrasted care in neuroscience centres with a dedicated neurocritical care unit (Figure 29a) compared with a combined neuro/general critical care unit (see Figure 29b). Hence, patients within each neuroscience centre were only included in one of the comparator groups, analogous to a cluster randomised trial in which individuals within a cluster are all randomised to the same ‘treatment’.

FIGURE 29.

Comparators for research objective 1: organisation of care within neuroscience centres. (a) Dedicated neuro unit; (b) combined neuro/general unit.

Comparators for addressing research objective 2

The second research objective considered whether, for those patients who presented at a non-neuroscience centre, the intention to undertake an early transfer to a neuroscience centre was cost-effective. This population was characterised by all RAIN study patients who presented at a non-neuroscience centre (i.e. pathways A, B and C) apart from those who underwent neurosurgery for evacuation of a mass lesion within 24 hours following presentation. These comparators were selected to contrast an initial decision to transfer the patient to a neuroscience centre (Figure 30a) compared with an initial decision to manage the patient within the non-neuroscience centre (see Figure 30b). The ‘early’ transfer group comprised patients transferred directly to a neuroscience centre from an emergency department in a non-neuroscience centre, and transfers from a critical care unit that were within 18 hours of initial hospital presentation. The ‘no or late’ transfer group comprised patients who received all their critical care within a non-neuroscience centre, and also patients from critical care units in non-neuroscience centres, who were transferred to a neuroscience centre more than 24 hours after initial hospital presentation. To ensure separation between the comparator arms, we excluded patients transferred between 18 and 24 hours following initial hospital presentation. The cut-off times were chosen by consensus among the clinical experts on the RAIN Study Steering Group. Patients within each participating critical care unit could be included within either comparator group.

FIGURE 30.

Comparators for research objective 2: ‘early’ vs ‘no or late’ transfer to a neuroscience centre (a) Early transfer; (b) no or late transfer.

As part of research objective 2, three sets of prespecified subgroup analyses were undertaken: the first according to age, the second according to presence of major extracranial injury and the third by GCS score at baseline. The justification for reporting results by age group was that several recent studies have reported worse outcomes following TBI for older patients, defined as those above an age threshold ranging from 65 to 80 years. 121–124 This raises the hypothesis that ‘early’ transfer for an older subgroup may not be cost-effective. We categorised age groups according to a cut-off of 70 years, which is in the middle of the range of age thresholds used previously, and was anticipated to ensure sufficient sample sizes in each subgroup.

Major extracranial injury frequently accompanies TBI and can increase mortality. 16,94,125–127 However, it is unclear whether the presence of major extracranial injury will modify the effect of an early transfer to neuroscience centres on mortality, morbidity or cost. Hence, we reported the costs, consequences and cost-effectiveness of ‘early’ compared with ‘no or late’ transfer for subgroups with and without major extracranial injury, defined as an injury that would require hospital admission in its own right.

The third subgroup analysis was motivated by NICE guidance, which recommended transfer to a neuroscience centre for all patients with severe TBI, defined as a TBI with GCS score of ≤ 8, regardless of the need for neurosurgery. 11 We therefore reported the costs, consequences and cost-effectiveness of ‘early’ compared with ‘no or late’ transfer for subgroups with a last pre-sedation GCS scores of 3–8 and 9–14.

Settings, inclusion criteria and measurement of case mix

Each critical care unit within the RAIN study was classified according to whether they were in a neuroscience centre and, if so, whether they were a dedicated neurocritical care unit or a combined neuro/general critical care unit (Table 30). (The one ‘major injuries unit’ participating in the RAIN study was classified as a dedicated neurocritical care unit due to the specialist nature of their case mix and the presence of a separate general critical care unit within the same hospital.) All dedicated neurocritical care units in the UK participated in the RAIN study, along with 74% of combined neuro/general critical care units and 16% of general critical care units in non-neuroscience centres. Dedicated neurocritical care units had a similar distribution of bed numbers to general critical care units in non-neuroscience centres, whereas combined neuro/general critical care units were larger. The throughput of TBI cases was similar for dedicated neurocritical care units and combined neuro/general critical care units, and substantially higher than for general critical care units in non-neuroscience centres.

Data were extracted from the RAIN study data set for all patients with acute TBI and a last pre-sedation GCS score of < 15. For research objective 1, all patients with RAIN study data submitted from a neuroscience centre were included in the analysis. All patients who met the inclusion criteria within an eligible critical care unit were then assigned to either comparator group according to whether the neurocritical care unit met the criteria for a dedicated critical care or combined neuro/general critical care unit. 13,14

For research objective 2, all patients initially presenting at a non-neuroscience centre were included in the analysis. Patients included in this analysis came from RAIN study data submitted either from non-neuroscience centres or from neuroscience centres where the location prior to admission field indicated that the patient had been transferred from another acute hospital. Patients were assigned to either comparison group according to the time of transfer to a neuroscience centre. Patients transferred to a neuroscience centre between 18 and 24 hours following initial presentation at an acute hospital were excluded from the analysis.

Risk factor data for the IMPACT Lab35 and CRASH CT30 models were taken from the time of injury and initial presentation at acute hospital, and were defined in the same way as for the external validation of the risk prediction models (see Chapter 5).

Resource use before 6 months

For research objective 1, resource use was considered from the first admission to critical care within the neuroscience centre, i.e. any resource use before this critical care admission, was excluded. For the second research objective, resource use was considered from the first critical care admission following the TBI. For both research objectives, the RAIN study prospectively recorded the LOS in critical care for each admission, according to days in a dedicated neurocritical care unit within a neuroscience centre, days in a combined neuro/general critical care unit within a neuroscience centre, and days in a general critical care unit within a non-neuroscience centre. The RAIN study also recorded whether or not the patient had intracranial neurosurgery for evacuation of a mass lesion. We recorded the LOS following transfer to other wards, both within the same acute hospital or to other hospitals including rehabilitation centres (all termed LOS on general medical wards). Any readmissions to critical care within the RAIN study were recorded. We also extracted data on readmissions to critical care units that were not in the RAIN study via data linkage with the CMP database. Each day during critical care was assigned to the appropriate Healthcare Resource Group (HRG) using daily organ support data recorded for the CCMDS.

The 6-month follow-up in the RAIN study (see Chapter 3) included a Health Services Questionnaire to ascertain use of other hospital and personal social services up to 6 months following the TBI. These postal questionnaires asked patients or their carers to report any hospital and personal service use between discharge from hospital and 6 months. We adapted a questionnaire128 to report service use following a TBI. The items of service use included subsequent hospital readmissions (excluding critical care), visits to hospital outpatients, and community health services (contacts with the GP, health visitor, district nurse, physiotherapist, occupational therapist, psychologist and speech therapist) (see Appendix 5). The patients eligible for this questionnaire were those known to be alive, and discharged from hospital by the date of the 6-month follow-up.

Health outcomes at 6 months

Data on 6-month outcomes were collected centrally as detailed in Chapter 3. EQ-5D-3L profiles were combined with health-state preference values from the UK general population, to give an EQ-5D-3L utility index score on a scale anchored at 0 (death) and 1 (perfect health). 129 QALYs at 6 months were then reported for each patient by combining information on survival and utility score at 6 months. Patients who died before 6 months were assigned zero QALYs. For patients alive at 6 months who completed an EQ-5D-3L questionnaire, their utility score at 6 months was multiplied by 0.5 life-years (6/12 months) to give their 6-month QALY.

Unit costs

For all critical care admissions, each bed-day was costed with the corresponding unit cost per bed-day from the ‘Payment by Results’ database. 130 Each set of unit costs distinguished between the alternative care locations; for research objective 1, separate unit costs were extracted for neuroscience centres with dedicated neurocritical care units compared with neuroscience centres with combined neuro/general critical care units (Table 31). For research objective 2, separate unit costs were applied to critical care in non-neuroscience centres (general critical care) compared with neuroscience centres (dedicated neurocritical care or combined neuro/general critical care). The unit cost of transfer to a neuroscience centre was taken as the cost of an emergency transfer. 130 For those admissions that included an intracranial procedure, to recognise the additional costs of theatre time and consumables, we included an additional unit cost corresponding to neurosurgery for cases with complications or comorbidities. 130 Each set of unit costs was reported as an average across the centres in the RAIN study. Unit costs for general medical bed-days were taken across all categories of elective, excess bed-days, weighted for their relative prevalence. 131 Other unit costs of health and personal social services were taken from the literature (Table 32). 132 All unit costs were reported in 2010–11 prices.

Each item of resource use was combined with the appropriate unit cost to report a cost per patient for each cost category (inpatient, outpatient, community and total costs). Resource use was estimated at the individual patient-level for each specific location of care. The unit costs were average unit costs across the centres in the RAIN study specific to each location of care (see Table 32).

Analysis of 6-month costs and consequences

For both research objectives, we described baseline characteristics, unadjusted outcomes, resource use and costs for each comparator. For each end point we employed regression analysis to adjust for baseline differences between the comparators using the individual risk factor variables from the IMPACT Laboratory model. 35 For the mortality and unfavourable outcome end points, we report odds ratios with logistic regression. For research objective 1, where each patient within a critical care unit was by definition within the same comparison group, we specified multilevel models that included random effects for centre to recognise any between-centre differences. 133,134 For the EQ-5D-3L, QALY and cost end points, the multilevel model reported incremental effects of the alternative care locations, after adjusting for case mix. The multilevel models assumed that both individual and centre level residuals were drawn from normal distributions. For research objective 2, where patients within each critical care unit could be assigned to either comparison arm, we repeated the above analysis but with single-level regression models estimated by maximum likelihood. These models reported incremental effects of the alternative care locations, together with 95% CIs, assuming that individual level residuals followed normal distributions. For research objective 2, we undertook prespecified subgroup analyses to report: incremental effects according to age (aged ≤ 70 years or > 70 years), major extracranial injury (yes or no) and GCS score (3–8 or 9–14). All analyses were implemented in R using the General Linearized Models (GLM) package. 135

For those patients who did not complete the 6-month follow-up questionnaires, data were unavailable for GOSE (approximately 20%), EQ-5D-3L and health service use (both approximately 50%). For a minority of patients, information was missing for baseline characteristics (between 2% and 17%) and mortality (3%). We addressed missing data with multiple imputation following a similar approach to that applied in Chapter 5. 76,78,136 The EQ-5D-3L, GOSE and cost end points were conditional on survival status, so we conducted the imputation in two stages. In the first stage, we specified imputation models for baseline characteristics and mortality according to observed covariates. In the second stage, for each of the previously imputed data sets, we specified imputation models for GOSE, EQ-5D-3L and health service use for those patients who were either known to be alive at 6 months, or were predicted to be alive by the first stage imputation model. (Owing to the high number of health service use categories, we specified multiple imputation models only for three aggregated categories: inpatient, outpatient and ‘other’ service use. In order to present disaggregated descriptions of mean service use across all the patients sampled in each arm, we also undertook mean imputation. Here we replaced missing observations in each category with the unconditional mean resource use for each category by treatment arm, among those patients with the relevant resource use observed.)

Each multiple imputation model assumed that the data were ‘missing at random’, that is conditional on the variables included in each imputation model. 76 Each imputation model considered including baseline covariates (e.g. risk factor variables from the IMPACT Lab model), resource use (e.g. LOS in critical care), and end points (e.g. we used data for those patients who completed the GOSE questionnaire to predict missing EQ-5D-3L). For research objective 1, we developed multilevel imputation models to recognise the hierarchical structure of the data. 137 For research objective 2, we used standard multiple imputation. 77 As the missingness pattern may differ across treatment groups, we specified separate imputation models for each comparator, and five imputed data sets were generated for each stage of the imputation, giving a total of 25 imputed data sets for each arm. After imputation, we applied the analytical models to each imputed data set, and combined the resultant estimates with Rubin's rules,76 which recognise uncertainty both within and between imputations. All multiple imputation models were implemented in R using MICE. 79

Research objective 1: Dedicated neurocritical care unit compared with combined neuro/general critical care unit

The baseline characteristics of the patients were broadly similar between the dedicated neurocritical care units and combined neuro/general critical care units (Table 33). In particular, the median predicted risk of unfavourable outcome, according to the IMPACT Lab model, was 40% for those dedicated critical care units compared with 41% in the combined units.

Table 34 describes the main items of hospital resource use recorded from the RAIN study and the CMP database. Although the overall mean hospital LOS was similar between the groups, the dedicated neurocritical care unit group had a higher average LOS in critical care.

Table 35 shows the flow of patients through the study. As the EQ-5D-3L and the Health Services Questionnaire were administered only by post, and not by telephone, the completion rates were lower for these questionnaires.

Mortality at 6 months was similar between the groups (Table 36). The dedicated neurocritical care unit group had a higher mean EQ-5D-3L utility index score for survivors, a lower proportion of patients with unfavourable outcomes and higher mean QALYs, although none of these differences was statistically significant, after case mix adjustment.

Table 37 reports the results from the Health Services Questionnaire for all patients sampled in each comparison group. The mean numbers of contacts were similar between the groups. For both groups, relatively low use, on average, was made of community health services.

Table 38 reports the mean total costs at 6 months for each comparison arm. On average, critical care costs were higher for the dedicated neurocritical care units, and this led to a positive incremental cost compared with combined neuro/general critical care units.

Research objective 2: ‘early’ compared with ‘no or late’ transfer to a neuroscience centre

There were substantial differences in case mix between patients in the ‘early’ transfer group and the ‘no or late’ transfer group (Table 39). The patients in the ‘early’ transfer group were on average younger but were generally of less severe case mix than the ‘no or late’ transfer group. The ‘no or late’ transfer group had a higher proportion of patients with a GCS score of 3–8, for whom neither pupil was reactive, and who had a Marshall CT classification of ‘non-evacuated mass lesion’. The net effect was that the median predicted risk of death at 6 months was higher for the ‘no or late’ transfer group than for the ‘early’ transfer group.

Figure 31 presents a histogram of the time from initial presentation at hospital to transfer to a neuroscience centre, for all patients transferred. Each bin on the histogram represents a time period of 1 hour; the vertical line at 18 hours represent the maximum time to transfer for the ‘early’ group, and the line at 24 hours, the minimum transfer time for the ‘late’ group. In the ‘no or late’ transfer group, 94% of patients were not transferred to a neuroscience centre. The majority of transfers were between 3 and 10 hours from hospital presentation. For 77 patients transferred directly from an emergency department a time of transfer was not available; these patients were defined as in the ‘early’ transfer group.

FIGURE 31.

Distribution of time from initial presentation at hospital to transfer to a neuroscience centre. Note: 10 transfers beyond 48 hours are not shown; 77 transfers direct from the emergency department had missing transfer times and were assumed to be early.

The ‘early’ transfer group had mean LOS in critical care, on general wards and in total that was approximately double that of the ‘no or late’ transfer group (Table 40). In addition, a higher proportion of patients in the ‘early’ transfer group had an intracranial procedure for evacuation of a mass lesion, at least 24 hours after initial presentation.

Table 41 shows the flow of patients through the study, and highlights that around half the eligible patients completed the postal questionnaires for the EQ-5D-3L and service use.

The proportion of patients who died before 6 months was substantially lower in the ‘early’ compared with the ‘no or late’ transfer group (odds ratio after case mix adjustment 0.52, 95% CI 0.34 to 0.80; Table 42). Although the CI on the odds ratio for unfavourable outcome included 1, the ‘early’ transfer group reported a higher mean EQ-5D-3L utility index score for survivors, and higher mean QALYs after case mix adjustment.

Table 43 shows the hospital and community health service use reported by the patients at 6 months, averaged across the whole sample including those who died. The ‘early’ transfer group reported higher average LOS after readmissions to general medical wards, and had higher mean contacts with outpatient and community health services.

The higher proportion of patients surviving in the early transfer arm is reflected in the estimates of the total costs at 6 months (Table 44); the mean costs of critical care and the total costs were twice as high in the early transfer group, with a positive incremental cost of approximately £15,000, after case mix adjustment.

Table 45 presents the results of the subgroup analyses for the 6-month end points, after case mix adjustment. Patients aged > 70 years had, on average, a higher odds of death in the ‘early’ compared with the ‘no or late’ transfer group, but the CIs around the odds ratios were wide and included 1. For older survivors, the mean odds ratio for death at 6 months was < 1 whether or not patients had a major extracranial injury but the subgroup with major extra cranial injury had very low odds of death in the ‘early’ transfer group.

Overview

For each research objective, we reported incremental cost-effectiveness of the alternative care locations over the lifetime. Such lifetime CEA requires information on long-term survival, QOL and cost of alternative interventions. Hence, it was necessary to make assumptions about the long-term prognosis of patients with TBI based on the 6-month data from the RAIN study, but also drawing on evidence from the literature. We tested whether the results were robust to alternative assumptions in extensive sensitivity analyses.

To calculate lifetime QALYs, long-term survival for each patient was calculated from observed survival for each patient within the first 6 months, and from their predicted survival after 6 months. Survival after 6 months was predicted for each patient by applying age-/sex-adjusted excess death rates for the RAIN study patients compared with those for the general population. This was undertaken in two steps. First, we calculated the excess death rates for RAIN study patients who survived after 6 months, compared with the age-/sex-matched UK general population. Second, these excess death rates were applied to predict life expectancy for each RAIN study patient alive at 6 months according to their age and sex.

Long-term survival

Patients in the RAIN study had their vital status followed up until September 2011, and dates of all cause death before that cut-off were provided by MRIS. For each patient the total survival time was calculated up to death or September 2011. We plotted Kaplan–Meier survival curves showing the number of patients who died over time until the last available time point, which was 800 days after the initial admission. For each comparator, the majority of deaths were within the first 30 days, as shown in Figure 32 for research objective 1.

FIGURE 32.

Kaplan–Meier survival curves for research objective 1.

We followed methodological guidance and considered alternative parametric approaches for extrapolating the observed survival. 138 When considering alternative parametric specifications we excluded the first 30 days of follow-up, as during this during this time period, patients had high risks of death; it was judged inappropriate to use this portion of the data for predicting the long-term probability of death. Figure 33 shows alternative parametric extrapolations for research objective 1, for the two comparator arms combined.

FIGURE 33.

Comparison of alternative parametric extrapolations of survival for research objective 1.

We compared the relative goodness of fit of the alternative parametric survival functions, for the remaining period of observed survival according to the Akaike information criterion (AIC) and Bayesian information criterion (BIC), the fit was similar with the logistic, log-normal or Weibull models (Table 46).

We compared the survival predicted by the alternative extrapolation approaches at 6–12 months with that for the age-/sex-matched general population. 139 We used each parametric extrapolation to report the predicted probability of death at 6–12 months for all the 30-day survivors included in RAIN for research objective 1, and report the ratio of these probabilities of death compared with those for the age-/sex-matched general population, also over a 6-month period. Table 47 shows, for the older age group each of the parametric survival curves tended to predict lower mortality compared with the general population.

In the base case we therefore decided that the most plausible approach was to apply age-/sex-matched general population death rates after 6 months for each comparator. Hence, in each of the comparator arms for research objective 1, a similar proportion of patients were assumed to die each year. Of the alternative parametric functions, the Gompertz was judged the most plausible and so we applied this extrapolation as a sensitivity analysis.

For research objective 2, the Kaplan–Meier survival curves (Figure 34) highlight that differences in survival between the comparator groups were observed within the first 30 days, and were maintained over time. Here when the alternative parametric specifications were applied to the RAIN data (excluding the first 30 days after TBI), the fit was similar (Table 48). Each of the extrapolation approaches predicted death rates lower than those for the general population for 6–12 months for the younger age group (Table 49). In the base case, we again applied age-/sex-matched general population death rates after 6 months, and used the Gompertz extrapolation in a sensitivity analysis. For this decision problem, both approaches assumed that the differences between the comparison groups observed within the RAIN study were maintained over time.

FIGURE 34.

Kaplan–Meier survival curves for research objective 2.

Long-term quality of life

The lifetime CEA required estimates of QOL over time according to the initial locations of critical care. Ideally, CEAs are populated with longitudinal estimates of QOL reported using a generic, preference-based utility measure such as the EQ-5D-3L. We reviewed the literature to find appropriate evidence on QOL for this research objective. However, we did not find any studies that reported QOL over time with a preference-based measure. Our review did find two studies that reported QOL assessed with the Short Form questionnaire-36 items (SF-36) at approximately 10 years after TBI compared with the general population. Jacobsson et al. 140 collected data for 67 individuals in Sweden 10 years after acute TBI. This sample included 50% of patients defined as having had a mild TBI (GCS score of 13–15 at admission to the emergency department) and 50% who had either a moderate (GCS score of 9–12) or severe (GCS score of 3–8) TBI. However, the study did not find any differences in QOL according to initial severity. The main finding was that for the whole sample of patients with TBI, general health assessed by the SF-36 was approximately 15% lower than that for the age-/sex-matched general population (n = 1224). Andelic et al. 141 described QOL (SF-36) for a sample of 62 survivors 10 years after moderate (GCS score of 9–12; 52%) to severe (GCS score of 3–8; 48%) TBI in Norway. The main finding was that the study patients had systematically lower scores on each dimension of the SF-36 than the age-/sex-matched general population (n = 2323). The mean SF-36 score for general health was similar to that for a moderate to severe TBI sample in the USA (n = 228), and approximately 15% less than that of the general Norwegian population. A similar decrement to the general populations was reported for the moderate and the severe TBI groups. Other studies have considered disability over time for TBI survivors,75 and found that in patients admitted to hospital 5–7 years after a TBI, rates of disability were high (53%) and similar to those reported at 1 year (57%). Our review did not find any studies that estimated long-term QOL or disability for TBI patients managed in alternative locations of neurocritical care.

In light of the lack of evidence on long-term QOL following alternative locations of critical care, our estimates of QOL over time for each comparator were based on the 6-month data from the RAIN study. However, rather than assuming that the differences in QOL between the comparators at 6 months were maintained, we made the conservative assumption that the differences in QOL attenuated over time. For each comparator we predicted the mean QOL at 6 months for patients aged 40 years (approximately the median age of RAIN patients) with the same risk adjustment applied as in the cost–consequence analysis. We predict mean QOL for each treatment arm between year 1 and year 10 with a linear interpolation, such that after 10 years the mean for each treatment arm was 15% lower than that for the age-matched general population. After 10 years we assumed that for each comparator, the 15% decrement compared with the age-matched population was maintained. As Figure 35 shows, this implies that the relative gains in QOL at 6 months attenuate over time (see also Sensitivity analyses).

FIGURE 35.

Attenuation of relative QOL over time for (a) research objective 1 and (b) research objective 2.

For each individual, we combined their predicted life expectancy (see Long-term survival), with their predicted QOL over time to give their projected lifetime QALY.

Long-term costs

The lifetime CEA also required costs to be projected for each individual for those surviving beyond 6 months. We reviewed the literature but did not find any relevant information on the relative costs of alternative care locations for patients who were alive 6 months after TBI. We therefore considered long-term costs based on the costs estimated in the RAIN study at 6 months. The costs considered fell into four categories: inpatient costs incurred after admission to critical care, inpatient costs on general wards, outpatient costs and community care costs.

We calculated inpatient costs incurred after critical care admission by recording ongoing admissions and readmissions to critical care units that participated in the RAIN study between 6 and 12 months after the TBI. We also considered admissions to critical care units that did not participate in the RAIN study but were included in the CMP, and to critical care units that participated in the CMP after patient recruitment to the RAIN study ended. The mean costs for each comparator group were then calculated for those patients who survived at least 6 months and were not censored between 6 and 12 months. These mean costs were used to impute mean costs between 6 and 12 months for the censored observations. For each comparator, these mean costs were relatively small (Tables 50 and 51). After 1 year it was assumed that there were no further readmissions to critical care that were attributable to the original TBI.

For the other cost categories, we took each individual's inpatient, outpatient and community costs up to 6 months estimated from the Health Services Questionnaires. For each comparator arm we then reported the mean cost between initial hospital discharge and the 6-month time point but only for those surviving up to 6 months. We then applied these 6 monthly costs for each survivor to each subsequent 6-month period up to 3 years. After 3 years we assumed that there were no further costs attributable to the acute TBI (see Sensitivity analyses).

Base-case analysis

We reported mean lifetime costs (£) and QALYs per patient for each comparison group by predicting costs and outcomes for each individual as described above. Each incremental effect was reported using the risk factors from the IMPACT Lab risk prediction model to adjust for case mix. For research objective 1, incremental QALYs and costs were estimated with bivariate multilevel models to allow for clustering and any correlation between the end points. 133,134 For research objective 2, the correlation between the end points was recognised with Seemingly Unrelated Regression. 142

For each research objective, incremental net monetary benefits (INBs) were estimated by valuing incremental QALYs at a threshold of £20,000 per QALY and subtracting from this the incremental costs. For each lifetime end point (costs, QALYs and INBs) we used bivariate regression analysis to recognise correlation between individuals' costs and QALYs.77 Each of these regression models was applied to each of the 25 imputed data sets from the analysis of the 6-month end points (see Analysis of 6-month costs and consequences). We again combined the resultant estimates of the incremental effectiveness, costs and cost-effectiveness with Rubin's rules. Hence, CIs for incremental costs, QALYs and INBs again recognised the within- and between-imputation variation. Cost-effectiveness acceptability curves (CEACs) were calculated by reporting the probability that each alternative was the most cost-effective (i.e. had a positive INB) at different levels of willingness to pay for a QALY gain (£0−50,000 per QALY gained). For research objective 2, analyses were repeated for the previously defined subgroups using stratified analyses.

Sensitivity analyses

The base case made the following assumptions that, a priori, were judged to be potentially important: (a) costs attributable to the TBI were only incurred for up to 3 years; (b) the differences in unit costs across care locations represented genuine differences in resource inputs across settings; (c) QOL from the 6-month follow-up applied irrespective of the actual time of follow-up (which ranged from 4 to 10 months after initial presentation); (d) the QOL for survivors at 10 years post TBI was 15% of that of the general population; (e) transfers up to 18 hours following initial hospital presentation were categorised as ‘early’; (f) costs were normally distributed; (g) all-cause death rates applied from 6 months following hospital presentation; and (h) the IMPACT Lab risk prediction model was the most appropriate for case mix adjustment. The sensitivity analyses tested whether the base-case results were robust if the following alternative standpoints were taken:

1. Extending the period over which costs were attributable to the acute TBI To test whether the results were sensitive to the duration over which costs were attributable to the acute TBI, we repeated the analysis assuming that (i) inpatient costs in general medical wards, outpatient and community costs were incurred for 5 years, and (ii) outpatient and community costs continued for up to 10 years.

2. Using the same unit costs across all critical care locations We assumed that all critical care units in the RAIN study had the unit costs for combined neuro/general critical care units.

3. Limiting use of QOL data from the RAIN study to those patients whose QOL was measured at close to 6 months To assess the impact of only using QOL information from those patients in the RAIN study followed up between 150 and 220 days, we assumed that those patients who had QOL measured outside these time points had missing 6-month QOL and GOSE questionnaire data. We then applied a multiple imputation model to impute 6-month end points for these patients.

4. Assuming that the QOL decrement observed in RAIN was maintained Some studies have suggested that the level of residual disability reported at 12 months is maintained for 5–7 years. 75 Here we assumed that the decrement in QOL reported in the RAIN study at 6 months compared with the general population was maintained over the patients' lifetime.

5. Taking a stricter definition of ‘early’ transfer for research objective 2 To test whether or not the results for research objective 2 were robust to the arbitrary definition of ‘early’ transfer, we repeated the analyses defining an ‘early’ transfer as within 8 hours of hospital presentation and excluding patients transferred between 8 and 18 hours.

6. Assuming that individual patient costs were drawn from a gamma distribution compared with a normal distribution The assumption that costs are normally distributed may not be plausible,143 so here we allow costs to follow a gamma distribution. 144

7. Using a Gompertz survival function To test whether the results were robust to the choice of extrapolation approach we applied age-stratified death rates using a Gompertz survival function.

8. Undertaking risk adjustment with CRASH CT risk prediction model We reran the analytical models estimating incremental costs and QALYs using the variables from the CRASH CT model30 rather than the IMPACT Lab model35 to adjust for case mix.

Base-case results for research objective 1

The base-case lifetime cost-effectiveness results for the first research objective are shown in Table 52. Dedicated neurocritical care units had higher mean lifetime QALYs at small additional mean costs, with an incremental cost-effectiveness ratio (ICER) of approximately £14,000 per QALY. At a ceiling ratio of £20,000 per QALY the INB was positive (approximately £1300). The CEAC suggested that at ceiling ratios of £20,000 to £30,000 per QALY, the probability that dedicated compared with combined neurocritical care units are cost-effective for patients following acute TBI, is around 60% (Figure 36).

FIGURE 36.

Probability that care following acute TBI is more cost-effective in a dedicated neurocritical care unit vs a combined neuro/general critical care unit at alternative levels of willingness to pay for a QALY gain.

Sensitivity analysis results for research objective 1

The sensitivity analysis shows that the base-case finding, that dedicated critical care units were on average more cost-effective, is robust to the alternative approaches taken (Figure 37). Where the higher mean QOL for the dedicated neurocritical compared with the combined units observed in the RAIN study is maintained for the lifetime, the positive incremental QALY and INB in favour of the dedicated units increased. Here, sampling uncertainty is greater than in the base case as this approach recognises more fully individual-level variation in QOL across the sample. However, as for the other scenarios considered, the point estimate of the INB is consistent with that reported in the base case.

FIGURE 37.

Sensitivity analyses reporting mean (95% CI) INB comparing care for dedicated neurocritical care units vs combined neuro/general critical care units. Vertical dashed line indicates incremental net benefits in the base case analysis. Solid vertical line indicates no difference in net monetary benefits between comparator groups.

Base-case results for research objective 2

Table 53 reports the base-case results for the lifetime CEA comparing ‘early’ transfer compared with ‘no or late’ transfer to a neuroscience centre.

The results show that, after adjusting for differences in observed baseline characteristics the ‘early’ transfer group reported higher lifetime QALYs, at an additional cost, with an ICER of approximately £11,000 per QALY. The CEAC suggested that at the standard thresholds of £20,000–30,000 per QALY, the probability that early transfer was cost-effective is close to 100% (Figure 38).

FIGURE 38.

Probability that ‘early’ vs ‘no or late’ transfer is cost-effective at alternative levels of willingness to pay for a QALY gain.

Sensitivity analysis results for research objective 2

The sensitivity analysis shows that the base-case cost-effectiveness results are relatively robust to the alternative methodological standpoints considered; the INB is broadly similar across scenarios (Figure 39). The additional scenario considered for this objective was to limit the sample for the ‘early’ transfer group to those patients who were transferred within 8 hours of presentation. Tightening the criteria for ‘early’ transfer led to slightly higher incremental QALYs and increased the cost-effectiveness of ‘early’ transfer.

FIGURE 39.

Sensitivity analyses reporting mean (95% CI) INB comparing care following TBI for ‘early’ vs ‘no or late’ transfer to a neuroscience centre. Vertical dashed line indicates incremental net benefits in the base case analysis. Solid vertical line indicates no difference in net monetary benefits between comparator groups.

Subgroup analysis results for research objective 2

The results for the subgroup analysis according to age group suggest that ‘early’ transfer has a very low probability of being cost-effective for patients aged > 70 years (Figure 40).

FIGURE 40.

Probability that ‘early’ vs ‘no or late’ transfer is cost-effective, by age group, at alternative levels of willingness to pay for a QALY gain.

The probability that ‘early’ transfer is cost-effective is lower for the subgroup of patients defined as without major extracranial injury. Here the probability that ‘early’ transfer is cost-effective is around 60% compared with 100% for the subgroup defined as having a major extracranial injury (Figure 41).

FIGURE 41.

Probability that ‘early’ vs ‘no or late’ transfer is cost-effective, by presence or absence of major extracranial injury, at alternative levels of willingness to pay for a QALY gain.

‘Early’ transfer appears most cost-effective for patients with severe TBI according to the baseline GCS score; the probability that ‘early’ transfer is relatively cost-effective is between 60% and 80% at ceiling ratios of between £20,000 and £30,000 per QALY (Figure 42).

FIGURE 42.

Probability that ‘early’ vs ‘no or late’ transfer is cost-effective, by baseline level of TBI severity, at alternative levels of willingness to pay for a QALY gain.

Principal findings

This aspect of the study had two research objectives. The first of these was to compare the relative costs, consequences and cost-effectiveness of management in a dedicated neurocritical care unit compared with a combined neuro/general critical care unit for critically ill adult patients with acute TBI who present at, or are transferred to, a neuroscience centre. The second objective was to compare the relative costs, consequences and cost-effectiveness of ‘early’ (within 18 hours of hospital presentation) transfer compared with no transfer or ‘late’ (after 24 hours) transfer, for critically ill patients with acute TBI who initially present at a non-neuroscience centre and do not require surgery for evacuation of a mass lesion within 24 hours. The main findings are that, following case mix adjustment using the variables from the IMPACT Lab model,35 dedicated neurocritical care units lead to higher mean QOL than combined units but considerable statistical uncertainty surrounds this result; and dedicated units are associated with additional costs. The cost-effectiveness results suggest that it is highly uncertain that dedicated neurocritical care units are more cost-effective than combined units; the probability that dedicated units are more cost-effective is around 60%. For patients presenting at non-neuroscience centres who do not require immediate neurosurgery, we find that, after risk adjustment, early transfer is associated with gains in survival, and improvements in mean QOL for survivors. Although early transfer is also associated with higher costs, these appear to be justified by the relatively large QALY gains, with an incremental cost per QALY of around £11,000. Hence, at the £20,000 per QALY threshold typically used by NICE, the probability that ‘early’ transfer is cost-effective is close to 100%. Although this finding is robust to all the alternative assumptions considered, the potential role for unobserved confounding must be recognised.

Meaning of the study and comparison with previous studies

This study extends the previous literature on the relative costs and consequences of alternative locations of neurocritical care. 107–109 The finding that there is, on average, an improvement in risk-adjusted QOL if patients with acute TBI are managed in a dedicated neurocritical care unit rather than a combined neuro/general critical care unit has several possible explanations. First, a dedicated multidisciplinary team may provide more effective and immediate rehabilitation as part of acute care; they may also improve access to specialised neurorehabilitation after discharge from the unit, which can lead to improved outcomes. 145 Second, dedicated units may provide more aggressive monitoring and management, which have been suggested to reduce morbidity. 16 Although concerns have also been raised that more invasive approaches may increase mortality,116 the RAIN study did not find any differences in mortality at 6 months after TBI between patients managed in dedicated and combined units. A third explanation is that the differences in the mean QOL between the settings may simply reflect chance variation, and indeed the CIs include a difference of zero, and so the null hypothesis that there is no difference in QOL between the settings cannot be rejected. That said, we followed standard methods guidance for CEA, and the lifetime analysis incorporated the non-significant difference in the mean QOL observed at 6 months. We also made the conservative assumption that the gain in mean QOL for the dedicated neurocritical care units attenuated over time. Fourth, the improvement in average QOL for TBI patients managed in dedicated neurocritical care units rather than combined units could reflect unmeasured confounding between the settings. For this comparison, the baseline characteristics of the patients were similar between the groups. However, the mean improvement in QOL in favour of dedicated units is fairly small, and could be overturned if a factor associated with poor prognosis was somewhat more prevalent in the dedicated compared with combined units.

In this context, it is useful to consider the differences in patterns of care between specialist neurocritical care units and neuro/general critical care units that were perceived to exist at the study outset. Then, the published literature suggested there were between-unit differences in the ability to deliver protocol driven critical care according to expert guidelines. 13 However, since then, the dissemination of perceived best practice (e.g. through NCCNet) may have reduced clinical practice variation among neuroscience centres. This may explain why any differences observed in costs and consequences according to dedicated compared with combined neuro/general critical care are small.

The finding that early transfer to a neuroscience centre for TBI patients appears cost-effective is driven by the gains in survival and QOL observed in the RAIN study at 6 months after hospital presentation. Previous work has found that there are benefits from early transfer for patients who have a space-occupying haematoma with worsening mass effect,115 but this study extends the evidence base to other critically ill patients with TBI. Previous research raised conflicting hypotheses, with some studies suggesting that, for non-surgical patients, early transfer and subsequently more aggressive management may lead to increase risks and costs that outweigh any gains,119 whereas other studies suggested that, delayed transfer may lead to worse outcomes. 120 In the RAIN study, only a small proportion (6%) of the ‘no or late’ transfer group had a delayed transfer. Hence, any detrimental effects from delayed transfer would be unlikely to explain the findings, and the main contrast is between an intention to transfer the TBI patient to a neuroscience centre compared with continuing management at a non-neuroscience centre.

The definition of an ‘early’ transfer is clearly arbitrary, and may differ across contexts. For the RAIN study it was judged important to define the time point at which an ‘early’ transfer was made according to what was broadly appropriate for the NHS context. Hence, in advance of any analysis, the RAIN Study Steering Group defined an ‘early’ transfer as being within 18 hours of hospital presentation. This pragmatic choice of cut-off time recognised that there may be several alternative care pathways that patients may take from initial hospital presentation to transfer to a neuroscience centre. The chosen cut-off time also recognised that even if an early decision to transfer is made, there may be local logistical barriers to an immediate transfer. Hence, an 18 hours time point was chosen to be conservative and included all admissions where there was ‘an intention’ to made a rapid transfer. A potential concern is that the relative cost-effectiveness of an ‘early’ transfer policy could be highly sensitive to the cut-off point, and indeed previous evidence from trauma networks encourages a 4-hour time window, but for neurosurgical interventions. 101 Our sensitivity analysis suggested that applying an 8-hour rather than an 18 hours time window from hospital presentation, reduced the incremental cost per QALY for early transfer from £11,000 (base case) to £9000. Hence, the overall finding that ‘early’ transfer appears cost-effective is not sensitive to the choice of time threshold.

Our evaluation considered the relative costs and consequences of ‘early’ transfer according to alternative pre-defined subgroup analyses. The results suggest that ‘early’ transfer is relatively cost-effective, both for patients defined as having mild or moderate TBI (GCS score of 9–14) as well as for patients with severe TBI (GCS score of 3–8) at admission. Patients aged > 70 years have a higher odds ratio of death in the ‘early’ transfer group. For this subgroup the most cost-effective strategy appears to be to keep those patients presenting at a non-neuroscience centre within that setting. The effect of ‘early’ transfer on mortality appears to differ according to whether or not patients have a major extracranial injury. We observed large and significant gains from ‘early’ transfer in the subgroup of patients with major extracranial injury. However, for patients without a major extracranial injury, the improvements in mortality and QOL following ‘early’ transfer did not reach statistical significance, and we are, hence, unable to conclude that ‘early’ transfer of patients in this subgroup is cost-effective.

The relative gains from ‘early’ transfer in the subgroup with major extracranial injury are somewhat counterintuitive as it might be expected that the risks of physiological derangement associated with transfer would be greater in this subgroup. One possible explanation is that the presence of significant extracranial injury may represent a marker of a disease mechanism or injury type that predisposes patients to develop the pathophysiological derangements (such as severe intracranial hypertension) that most benefit from the specialist critical care available at a neuroscience centre. For example, high-speed RTAs may not only tend to cause more extracranial injury, but also may predispose to traumatic axonal injury and subdural haemorrhage, and enhance the benefits of specialist critical care. Another explanation is that the differential gains may reflect the broad definition of major extracranial injury adopted in the RAIN study. Specifically, patients who would not have met a more rigorous definition of major extracranial injury may have been misclassified as having significant extracranial injury. These patients may be at relatively low risk of death following transfer. This could have been coupled with the retention of unstable patients with truly significant extracranial injury in referring hospitals, which might be undetected by the RAIN study, as we had no metric of the severity of extracranial trauma. The net effect of such misclassification and inadequate severity characterisation may be to overstate the cost-effectiveness of early transfer in those defined as having major extracranial injury and understate the gains in the subgroup defined as not have major extracranial injury. A final explanation is that there may be residual confounding in the major extracranial injury stratum. This subgroup may have included patients who, according to unmeasured prognostic factors, were ‘too unstable for transfer’. Further research would ideally record such potential confounders and use them as a basis for excluding such patients from the decision problem. That is, the population of interest should only include patient groups where there is equipoise as to the costs and benefits of ‘early’ transfer.

Strengths and weaknesses

Lifetime CEAs requires assumptions to be made about the long-term mortality, cost and QOL following alternative interventions – in this case, care locations. As the RAIN study was only designed to measure survival and QOL for all eligible patients up to 6 months, we carefully considered alternative ways of extrapolating from these data to the lifetime. There is no consensus from the literature on the size or the duration of the impact that an acute TBI has on mortality, QOL or costs. More specifically, estimates from the RAIN study provided the strongest basis for projecting the relative effects of alternative care locations on mortality, QOL and health and personal social service costs. Hence, in the base case, information from the RAIN study was used to support assumptions about the long-term prognosis for patients following a TBI, according to the location of critical care.

A potential issue raised by the results of the RAIN study is that few deaths were reported between 30 days and 1 year after hospital presentation; in general, the death rates reported were either similar to or lower than the age-/sex-matched general population. Rather, than just choosing the extrapolation approach that appeared to best fit the observed data, we considered which extrapolation approach would be most plausible according to the previous literature. Other studies have reported excess deaths for patients at 5–7 years following a TBI,146–149 whereas a more relevant study that included patients admitted to critical care following TBI observed only one death between 6 and 12 months out of over 400 survivors at 6 months. 73 In the base-case analysis we judged that the most reasonable extrapolation approach was to apply all-cause death rates from 6 months onwards. Then, to examine whether the results would be robust to an alternative approach we ran sensitivity analyses in which we applied the most plausible and conservative of the parametric extrapolations (using the Gompertz function). This sensitivity analysis led to small differences in the ICER – for example, for research objective 2, the base-case ICER reduced marginally from £10,704 to £10,691. More, generally, the sensitivity analyses suggested that the findings did not appear to change substantively when alternative assumptions were made about the time period over which costs attributable to the TBI were applied, or whether the decrement in QOL observed at 6 months was maintained over the lifetime.

Although the RAIN study has added to the literature on alternative care pathways following TBI, it does have some limitations. First, the RAIN study only measured costs and outcomes for up to 6 months; however, the sensitivity analysis suggests that results appear robust to the choice of alternative assumptions made in extrapolating the data over the lifetime. Second, the study followed methodological guidance and took a health and personal social services perspective, and so excluded any broader societal costs, for example from productivity losses or ongoing costs of rehabilitation or nursing home care borne by the patient. Third, about half of patients eligible for follow-up at 6 months were missing data on health service use and QOL, and these data were not missing completely at random. We tackled this issue by using state-of-the-art approaches to handling missing data, in that we used multiple imputation models that respected the hierarchical structure of the data. That said, such approaches assume that the data are missing conditional on baseline factors and other end point and process measures that are observed, hence if missingness is driven by unobserved prognostic factors this could have led to biased estimates.

Finally, the major concern in any such non-randomised comparison is residual confounding. Although our sensitivity analysis that used variables from the CRASH risk prediction model suggested that the results were robust to the choice of risk adjustment method, unobserved confounders may explain the differences in costs and consequences between the comparator groups. Specifically, the comparison of observed baseline characteristics between the ‘early’ and ‘no or late’ transfer groups suggests that the patients transferred early were of much less severe case mix according to observed factors. This raises the concern that unobserved confounders associated with worse outcome may also be more prevalent in the ‘no or late’ transfer group. Further research is required to consider alternative approaches for handling the potential impact of unobserved confounders. Such approaches may include collecting data on additional baseline measures anticipated to be important, considering more flexible risk prediction and matching methods, formal sensitivity analyses that investigate whether the findings are robust to unobserved confounding, and alternative approaches as instrumental variable estimation that purport to handle unobserved confounding.

Summary

In summary, this evaluation finds that for critically ill patients with acute TBI who present at, or are transferred to, a neuroscience centre, management in a dedicated neurocritical care unit rather than a combined neuro/general critical care unit is associated with small gains in QALYs, at additional costs, but it remains highly uncertain whether dedicated units are more cost-effective. Overall, for adult patients with acute TBI who present at a non-neuroscience centre and do not require neurosurgery within 24 hours, ‘early’ transfer appears more cost-effective than ‘no or late’ transfer to a neuroscience centre, after risk adjustment. However, further research is required that considers alternative approaches for handling differences in baseline characteristics between settings. Such research could draw on the framework and data presented in the RAIN study to address alternative comparisons, for example contrasting management in high-volume settings with management in low-volume settings, or comparing settings with relatively high proportions of patients with TBI with those with relatively low proportions.

Recommendation 1: Further research should be undertaken to explore the potential to improve on the current risk prediction models for acute traumatic brain injury

Although the existing risk prediction models demonstrated acceptable discrimination for mortality at 6 months and, less so, for unfavourable outcome at 6 months, there appears to be considerable scope for further improvement. As a minimum, the risk prediction models for unfavourable outcome all require recalibration for use in a UK critical care setting. However, additional areas that may be explored, either within the RAIN study data set, by data linkage (e.g. with the TARN database), and/or by new prospective data collection include: consideration of mechanism of injury; incorporation of pupil size in addition to reactivity; addressing the severity and/or site(s) of major extracranial injury; incorporating additional data on the physiological response to injury; and consideration of genetic factors in the host response to injury. Development of new risk prediction models may consider both classical statistical methods and machine learning approaches, such as artificial neural networks. The results of the RAIN study should be revisited in light of any improvements to risk modelling. A new risk prediction model may also be incorporated into a web-based prediction tool, as has been done for the CRASH and IMPACT models, to facilitate its use in practice.

Recommendation 2: Further research is required to consider alternative approaches for handling the potential impact of unobserved confounders on the RAIN study results

In addition to improving the risk prediction models used to underpin the non-randomised comparisons, alternative methodological approaches to complement risk adjustment include matching methods, such as propensity matching or GenMatch. 150 Such approaches would be complementary to the risk adjustment methods described, in that matching could be used to pre-process the RAIN study data before applying risk adjustment. 151 To consider unobserved confounding, future research could also be undertaken using instrumental variable estimation152 and formal sensitivity analyses that investigate whether the findings are robust to unobserved confounding. 153

Recommendation 3: Options should be explored for continuing to follow up the RAIN study cohort to obtain data on long-term mortality, functional outcomes and quality of life

The RAIN study identified substantial gaps and discordance in the literature regarding the long-term outcomes of patients with acute TBI. The RAIN study database provides a large, representative cohort of critically ill adult patients following acute TBI. Continuing follow-up of the RAIN study cohort would provide much valuable information, both to strengthen the lifetime CEA and to address additional issues, such as accelerated late cognitive decline. However, this would require changes to the current research ethics and governance approvals for the RAIN study and, therefore, these should be addressed urgently.

Recommendation 4: Further research is required to better understand the alternative pathways of care for patients following acute TBI and the impact of these on costs and outcomes

The RAIN study addressed certain specific aspects of the alternative pathways of care for critically ill adult patients following acute TBI and many other aspects remain unexplored. The simple comparison of dedicated neurocritical care units compared with combined neuro/general critical care units may be expanded to model a volume–outcome relationship with the absolute number of patients with TBI admitted or a dose–response relationship with the proportion of admissions that are patients with TBI to any given neurocritical care unit (or neurosciences patients more generally), or to give consideration to variation across centres in the use of treatment protocols or particular interventions. Further analyses for patients presenting outside neuroscience centres may consider the effect of time to transfer on costs and outcomes – both for patients who require neurosurgery (for whom current guidelines often recommend transfer within 4 hours101) and those who do not. An alternative model of service delivery, not currently implemented in most regions of the UK, is bypass of local hospitals to transport patients with significant TBI directly to a neuroscience centre. A feasibility study for a cluster RCT on this intervention – the Head Injury Transportation Straight to Neurosurgery (HITS-NS) trial (www.hta.ac.uk/project/2223.asp) – is currently ongoing.

Recommendation 5: Further research should explore equity of access to post-critical care support for patients following acute traumatic brain injury

Improved access to specialist neurorehabilitation services has been identified as a potential driver of better outcomes reported from neuroscience centres. 145 However, provision of and access to such services varies and questionnaire responses in the RAIN study indicated much dissatisfaction with the continuity of care and provision of rehabilitation and follow-up after discharge from critical care. Future research should identify the regional and local variation in provision of post-critical care support for patients following acute TBI and consider the impact of such provision on subsequent recovery.

Finally, the RAIN study should inform all future research studies in the neurocritical care of adult patients following acute TBI through provision of reliable data for sample size calculations and exploratory analyses, and informing the choice of risk adjustment methods and data set design.