Developing an intervention around referral and admissions to intensive care: a mixed-methods study

Assessing the clinical situation

A patient’s acute clinical condition and the severity and nature of their past medical history are important considerations when making a decision about admitting him or her to ICU. Our previous scoping review found several studies confirming that these factors have an impact on the decision. 9–14 However, there is often uncertainty about prognostic indicators and the extent of chronic illness in an acute situation. Other, more value-laden factors also appear to influence the assessment of the clinical situation, including the age of the patient13,15–17 or the doctor’s assessment of the patient’s functional status or quality of life,14,18–20 whether a patient has a medical or a surgical diagnosis3,12,21 and whether they are assessed by a junior or a senior physician. 10,16,22–24

Evidence and prognostic indicators

Although prognostic information from observational studies is available to guide decision-making, the likelihood of death for a critically ill individual remains difficult to predict with an acceptable degree of certainty. Several prognostic tools for application to critically ill patients have been developed, the most widely used in the UK being the Acute Physiology and Chronic Health Evaluation (APACHE-II) score. 25 However, this is calculated after the patient has been admitted to ICU, so it is of limited value for referral or admission decisions. Most prognostic models predict either ICU or hospital mortality, rather than long-term survival (or quality of life). Therefore, prognostic models are primarily used to assess ICU quality and performance rather than as a framework for decision-making in individual cases. 26

Some individual patient-related factors have been found to be associated with increased mortality in ICU patients after hospital discharge, namely severity of illness, age and presence of comorbidities. 5 This information can inform the risk–benefit analysis when considering admission, but it does not provide a definitive prognosis for an individual patient.

There is evidence to suggest that, even if clinicians are provided with patient-specific prognostic information about end of life at the time of decision-making, this does not materially alter their clinical decision-making. 27

Patient’s values and wishes

The ethical principle of respect for patient autonomy is reflected in legal and professional regulation of clinical decision-making. 28,29 A person should be provided with relevant information and given an appropriate time to contribute to a decision about their treatment. However, when the decision is about whether to refer or admit a person to ICU, that person is often unable to take part in the discussion, and in most cases will lack the capacity to consent to, refuse or request treatment because of the severity of their illness. Guidance on how to use a shared decision-making model in the context of critical illness has been produced;30 however, it is still the clinician’s responsibility to operationalise the shared decision-making model within a specific context, including the urgency of the situation and the extent to which the patient and/or their family or surrogate decision-maker are able to participate. In the UK, if a patient lacks capacity, it is the responsibility of the clinician caring for them to first determine if there is any relevant advance statement or legal proxy to make decisions on the patient’s behalf, and then to make a decision that is in the patient’s best interests, consulting where possible with the patient’s family and those who know the patient well in order to understand what the patient’s wishes might be. 28

Critical illness often develops with little warning and the decision whether or not to admit a patient to ICU often needs to be made in an emergency situation or when a patient’s condition is deteriorating rapidly. Family members may not be present and, even if they are, there may be little time for extensive discussion with them. When families are consulted about the patient’s values and wishes, they are not always accurate in predicting what the patient might want in this situation. There is evidence that families acting as surrogate decision-makers poorly predict what the patient would wish for themselves, and may prioritise their own values and wishes, rather than those of the patient, when asked for their views on future therapy. 31

Resources and external influences

The safe delivery of intensive care treatments requires a high concentration of staff and resources. In an ICU, the nurse-to-patient ratio is 1 : 1 for a ‘level 3’ patient or 1 : 2 for a ‘level 2 patient’,1,32 compared with a standard ward where one nurse will look after nine patients. 33 In 2006 in Europe the cost of an intensive care bed was between €1168 and €2025 per day. 34 Delivering this level of nursing for all patients would quickly overwhelm the resources of any health-care system, and therefore access must be limited. 35 The UK has relatively few intensive care beds compared with many other countries,36 and pressure on the UK’s available ICU resources is a regular occurrence. A survey by the Faculty of Intensive Care Medicine in 201837 showed that 21% of ICUs regularly moved patients to other hospitals because of a lack of local intensive care resources. Surveys and cross-sectional studies10,38–43 of patients admitted to ICU have previously shown that the availability of ICU resources influences whether or not a patient is admitted.

Decision-making within intensive care is set in the context of wider organisational policies and priorities. Some hospitals have specific priority programmes, such as transplant surgery or major trauma, and institutional policy may prioritise these patient pathways for intensive care admission. 44,45 The behaviour of other clinicians (e.g. failure to specify end-of-life treatment plans or to secure an ICU bed before elective surgery) and family demands for life support can also create a perceived pressure to admit a patient to ICU. 46

These organisational and situational pressures add a further ethical dimension to the decision-making, in addition to the difficulty of balancing benefits and burdens of ICU treatment for a particular patient.

The views and values of the decision-maker

As evidence-based prognostic indicators are seen to be of limited value, and as the patients’ views may not be known, it is likely that other values and perceptions will have a bearing on the decision about whether or not to admit a patient to ICU. Previous experience of treating patients with similar conditions may influence the clinician’s perception of a patient’s potential to benefit from a particular treatment. A phenomenon known as ‘prognostic pessimism’ has been demonstrated in ICU clinicians assessing patients with chronic obstructive pulmonary disease (COPD). Patients with the most severe disease were found to benefit most from ICU treatment and valued the resultant extension to their life after hospital discharge, contrary to the expectations of clinicians who predicted that they would fare badly. 47 Similar findings have also been seen in patients with heart failure. 48,49 A large European study50 of decisions to admit to ICU found that, although older people were less likely to be admitted to ICU, the mortality benefit (mortality without ICU compared with that with ICU) was greater for older patients than for their younger counterparts. The authors concluded that ‘physicians should consider changing their intensive care triage practices for the elderly’. 50

Personal moral values and religious values have also been shown to influence clinicians’ decisions about withholding or withdrawing life-prolonging treatment, of which intensive care can be considered an example. 51

Project design

Before submitting the application for funding, we held a patient and public engagement meeting. Attendees were recruited from a national ICU patient support charity, ICUsteps; patients from a NHS post-ICU clinic; and the University of Warwick’s Teaching and Research Action Partnership (UNTRAP). The purpose of this meeting was to seek input from the group on framing and prioritising the research questions and identifying appropriate methods for conducting the research that would be acceptable to patients and their families. During the meeting, the participants were presented with background information about referrals and admissions to intensive care and why there was a perceived need for a study. The presentation concluded with the proposed broad aim of the research. The group was asked to identify what they saw as important issues for the research to address, what the specific research questions might be and how to conduct the research in a way that was sensitive to patients and their families.

The group thought that this was an important area of research as it was crucial for patients and families in an extremely vulnerable situation to be able to trust both the professionals making these decisions and the decision-making process. Key questions identified by the group included how patients and families were involved in these decisions and communicated with; who was involved in making the decisions; how decisions about quality of life were made; and if ICU resources were important in the decision-making process. The group also offered suggestions on the timing and the number of interviews with family members to balance the needs of data collection with reducing family distress. These fed into the final project design.

Investigator team

Our investigator team included two PPI co-investigators (CW and SS), who were involved in the design and development of the project, provided guidance on the acceptability of the methods, commented on and amended all patient and family information materials (both study and intervention documents) and contributed to the writing and editing of reports from the different WPs.

Advisory group

We convened a patient and public involvement advisory group (PPIAG) of six members, some of whom had either been patients themselves or had experience of a relative being in an ICU. The group met 6-monthly throughout the project for updates from the project team and to provide advice on the conduct and findings of the project as it progressed. Individual members of the group also contributed more directly to specific WPs.

Project oversight

The project’s independent steering committee included two PPI members, one of whom had been a patient in an ICU. The committee met at 6-monthly intervals throughout the project to provide support to and oversight of the project.

Project development

The PPI co-investigators worked closely with the project team on developing information for patient and family participants and recruitment and consent processes in the ethnographic study. Two members of the PPIAG participated in the development of the draft decision-support intervention (DSI), attending project meetings and commenting on each stage of the process. One member led on initial drafting of patient and family information leaflets to be used as part of the intervention. Members of patient representative groups and the PPIAG attended the stakeholder conference and participated in focus groups to refine the content of the DSI. One of our PPI co-investigators (CW) chaired a session at the conference. Following the stakeholder conference, further extensive revision of patient and family information leaflets was overseen by the PPI co-investigators. Translated versions of these documents were checked for cultural acceptability among native speakers of the languages through our PPIAG contacts.

We were unable to obtain sufficient representation at the stakeholder conference for patients with mental health disorders and for elderly patients. We therefore attended local advocate group meetings to seek feedback on the DSI and its implementation.

Analysis

Two members of the PPIAG and one PPI co-investigator (CW) contributed to the analysis of the qualitative data in our ethnographic study. They attended data analysis meetings, read a selection of interview transcripts and contributed to the refinement of interview schedules and the development of an analysis framework.

Dissemination

Patient and public involvement co-investigators, members of the PPIAG and patient organisation representatives who attended the stakeholder conference were invited to the dissemination event at the end of the project. One PPI co-investigator (CW) gave a response to and reflection on the project findings from the patient and family perspective.

Study identification

Because both reviews focused on the process of decision-making about referral and admission to ICU, we used the same search strategy and abstract screening process to avoid duplication between them. At the full-paper screening stage, we identified studies relevant to (1) the factors review, (2) the experiences review and (3) both reviews. The search strategy was informed by an initial scoping review of the literature, and had three broad areas using a combination of the following MeSH (medical subject heading) headings and keywords: (1) critical and intensive care, intensive care units and critical illness; (2) patient admission, transfer, triage and refusal to treat; and (3) professional decision-making and judgement, professional–family relations, choice behaviour and medical futility. Papers that referred to paediatrics or neonates were excluded. The following databases were searched: MEDLINE, EMBASE, Applied Social Sciences Index and Abstracts (ASSIA), all sections of The Cochrane Library, Cumulative Index to Nursing and Allied Health Literature (CINAHL), PsycINFO and Web of Science. ‘Grey literature’ (Dissertation Abstracts Online, Index to Theses, OpenGrey) and references from key papers were also screened. We used forward and backward citation tracking from our full-text papers to identify further studies that our initial search had missed. Our full searches are presented in Appendix 1 (see Tables 25 and 26). We included papers published between 1980 and 2015 describing empirical research that focused on the process of decision-making about referral or admission of adult patients to ICU, and that investigated either factors affecting decision-making or the experiences of clinicians, patients or families. The search was run on 11 May 2015.

We identified 47,674 records, 34,343 after duplicates were removed. Abstracts were double-screened by a team of 13 reviewers [three members of the study team (CB, AS and HH) and 10 medical students trained in the process], and 552 records went forward for full-text retrieval and formal inclusion in/exclusion from the review (Figure 2). Full-text papers were also double screened by six members of the research team (CB, AS, ZF, HH, JT and MB).

FIGURE 2.

The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flow diagram for the main systematic review. a, We were unable to source these five full texts for a variety of reasons, including being unable to make contact with the author even after internet searching, e-mail-based invitation and telephoning, where possible; the author of two related conference abstracts also did not follow up in the end with further details on their unpublished full study findings (as promised earlier). Reproduced with permission from Rees et al. 61

Reproduced with permission from Rees et al. 61

In March 2018, we completed a brief update review to identify any relevant studies that had been published since our initial searches. A re-run of the original searches yielded over 10,000 returns. Given time constraints, we chose not to repeat the full review process. Instead, we adopted a pragmatic approach using the following method:

• We searched PubMed for papers published between 1 May 2015 and 31 December 2017 using the search terms Critical care/CCU or intensive care/ICU AND decision-making AND admissions OR referrals.

• We hand-searched the contents of the six journals that had provided more than one included paper in our original review. We searched all issues published from 1 May 2015 to 1 January 2018.

• We searched the reference lists of any identified papers to check for further papers, as well as the reference list of a published review. 62

Ten papers were identified at abstract screening stage for full-text assessment by two reviewers (AS/KR or AS/CB). Three further papers were identified from the reference list of the published review. Eight studies (seven for factors and one for experiences) were added to the total number of studies included for analysis in the main systematic review.

Methodological quality assessment

Cohort studies were assessed using the Newcastle–Ottawa Scale. 63 The majority of studies were cross-sectional, and we used an adapted version of the Newcastle–Ottawa Scale to assess quality for this study design. 64 Clinical trials were assessed with the Cochrane Risk of Bias tool65 and qualitative studies were assessed using May and Pope’s qualitative evaluation criteria. 66 Each included study was scored for quality by two reviewers (AS/HH, CB/JT or ZF/MB) and any discrepancies were referred to a third reviewer for final decision (KR or FG) (see Appendix 2, Tables 27–29).

Analysis

The majority of data for the factors review were quantitative, but a small number of studies had descriptive qualitative data. When possible, we combined studies statistically using a meta-analysis. Owing to the potential confounding effects of each of the factors examined on the others, we focused on studies reporting multivariate analyses of independent factors affecting admission decisions. Where these were lacking, we explored descriptive associations but were cautious in our interpretation because of biases and confounding.

If there were sufficient studies, effect sizes from multivariate analyses for each factor were pooled using the generic inverse variance method using RevMan software (version 5.3; The Cochrane Collaboration, The Nordic Cochrane Centre, Copenhagen, Denmark). 67 The results from cohort studies were pooled together with the results from cross-sectional studies as we were concerned not with longitudinal associations but with decisions to admit and the factors affecting these that occur concurrently. The remaining studies were described narratively.

For the experiences review, a thematic analysis of any relevant qualitative data from the identified studies was conducted. Two research team members (SR and AS) read all of the data and developed initial codes from which themes were developed during discussion. The themes were tested in a further research meeting with a third member of the team (FG).

Factors review

The characteristics of the 88 included studies and the factors each study investigated are documented in Appendix 3 (see Table 31). The vast majority were observational.

The findings are reported under factor headings grouped as patient-related medical, patient-related non-medical, clinician-related and organisation-related (Table 1). For each of the factors analysed, the results are presented first for multivariate analyses. A summary of multivariate analysis results is presented in Appendix 4 (see Table 32). For factors for which there are no multivariate analyses, we report findings of descriptive studies. If multivariate analyses are present, we note the presence of descriptive studies.

Patient-related medical factors

Type of acute illness

Nine studies12,14,16,41,68–70,72,73 reported multivariate analyses of type of acute illness as a factor affecting decisions about patient admission to ICU. The types of acute illnesses considered between studies varied enormously and precluded meta-analyses. Distinct groupings were possible in the following categories: respiratory, cardiac/vascular, neurological, infections and emergency surgery. Three studies14,16,69 reported respiratory diagnoses as a factor associated with reduced likelihood of refusal or increased odds of admission, whereas one study70,71 found respiratory diagnoses to be associated with an increased odds of refusal. Cardiac or vascular diseases were reported in six studies,12,16,41,69,72,73 with increased odds of admission reported in four studies. 12,16,69,73 One study72 showed less likelihood of refusal to ICU with a diagnosis of cardiac failure than with a diagnosis of respiratory failure. In another study,41 cardiac disease was associated with refusal to admit. Four studies16,41,69–71 reported on neurological disease, with inconsistent results. Two of the studies16,69 showed that admission was more likely with a diagnosis of neurological disease, whereas the other two studies41,70,71 showed that the same diagnosis was associated with refusal to admit. Three studies68,69,72 reported that infections were an independent factor associated with ICU admission, and two studies12,72 reported independent effects of emergency surgery on admission decisions (see Appendix 4, Table 32).

Other diagnoses as independent predictors of admission decisions in multivariate models included shock and coma, which reduced the likelihood of refusal. 14 Haematologic etiology,68 injuries/poisonings/toxic effects of drugs,69 trauma and haematological malignancy12 and ‘worried’16 were all associated with increased likelihood of admission.

One study13 reported diagnosis to be a significant factor in multivariate analyses affecting refusal to admit to ICU, but it is unclear which individual diagnoses had the most impact. Another study20 reported univariate associations between categories of acute illness and odds of ICU refusal in an elderly cohort, but type of acute illness was not an independent predictor in the multivariate model. Fourteen further studies10,11,18,19,24,74–82 reported descriptive associations between types of acute illness and decisions about admission to ICU.

Severity of acute illness

Eight studies12,13,41,76,82–85 reported multivariate analyses of severity of acute illness as a factor. A number of different scales were used to assess severity of acute illness (see Appendix 4, Table 32). Results from individual studies were plotted, but the studies were not combined statistically because of considerable heterogeneity (I² = 96%) and inconsistent direction of effect (Figure 3). There were no clear effects of severity of acute illness and decisions to admit patients to ICU (see Appendix 4, Table 32). It is possible that the heterogeneity and inconsistent direction of effect relates to questions around patients being too ill or too well to benefit from ICU care.

FIGURE 3.

Forest plot of studies reporting severity of acute illness scales in multivariate analyses. The pooled effect estimate is suppressed as heterogeneity is considerable (I² = 97%) and the direction of effect is inconsistent between studies. a, APACHE II score; b, APS II score (Simplified Acute Physiology Score II without age, comorbidity and type of admission); c, MPM-0 (mortality predicted model at admission); d, All Patient Refined Diagnosis Related Group (APR DRG) grouper (measures chronic and acute disease severity); e, APACHE II (SE imputed); f, EWS; and g, severity of illness stage 2 (as defined by the authors): selective population – patients with AIDS with pneumonia. AIDS, acquired immunodeficiency syndrome; EWS, Early Warning Score; IV, instrumental variable; OR, odds ratio; SE, standard error.

One further study20 reported a reduced odds of refusal of admission with increasing SOFA (Sequential Organ Failure Assessment) score [odds ratio (OR) 0.93, 95% confidence interval (CI) 0.87 to 1.0] in a univariate analysis, although this was not an independent predictor in the multivariate model. Thirty-six further studies10,19,21,22,24,42,44,72,73,75–77,80,81,86–107 reported descriptive associations between severity of acute illness and decisions about admission to the ICU.

Presence of chronic illness

Seven studies10,14,18,70,71,76,108 reported multivariate analyses of presence of chronic illness as a factor. Chronic illnesses investigated included dementia,108 metastatic cancer,10,18,76 mental disorder (psychotropic drug use),18 a combined category of chronic respiratory or heart failure or metastatic cancer without hope of remission,14 and a general category of ‘underlying chronic disease’70,71 (see Appendix 4, Table 32).

One study20 reported univariate associations between prior cognitive status and odds of ICU refusal, where refusal was associated with worse cognitive status, but this was not an independent predictor in the multivariate model. Twenty-one studies11,12,15,17,22,41,74,78,80,88–91,93,98,101,105,106,109–111 reported descriptive associations between the presence of chronic illness and decisions about admission to the ICU.

Severity of chronic illness

Five studies16,19,20,69,112 reported multivariate analyses for severity of chronic illness as a factor. Four studies16,20,69,112 assessed the severity of chronic illness using the Charlson Comorbidity Index. Results from individual studies have been plotted but not combined statistically because of considerable heterogeneity (I² = 95%) and inconsistent direction of effect (Figure 4). Sensitivity analyses combining scales of the same type and removing selective populations had no effect on the very high level of heterogeneity. There are no clear effects from multivariate studies of severity of chronic illness and decisions to admit patients to the ICU. Fourteen further studies10,24,41,74,80,81,86,87,91,92,95,100,101,113 reported descriptive associations between the severity of chronic illness and decisions about admission to the ICU.

FIGURE 4.

Forest plot of studies reporting severity of chronic illness scales in multivariate analyses. The pooled effect estimate is suppressed as heterogeneity is considerable (I² = 95%) and the direction of effect is inconsistent between studies. a, Elixhauser scale; b, Quan’s adaptation of Charlson Index; c, Charlson Comorbidity Index, selective population – diabetic ketoacidosis; d, Charlson Comorbidity Index, selective population – elderly cohort; and e, Charlson Comorbidity Index; IV, instrumental variable; OR, odds ratio; SE, standard error.

Functional status/quality of life

Fourteen studies10,12,14,15,17–21,68,70,76,82,114 reported multivariate analyses for functional status/quality of life as a factor using a number of different scales and measures, so we were unable to pool data statistically. Most studies reported on patient data from real-world settings but three reported simulation studies,17 surveys to health-care professionals using clinical vignettes114 or the theoretical future need of ICU in elderly patients admitted to emergency departments (EDs). 15

Measures of dependency/self-caring status related independently to admission decisions in the 11 studies10,12,14,18–21,68,70,76,82 in real-world settings; in most studies, increased dependency was associated with reduced odds of admission (see Appendix 4, Table 32).

Twenty-five further studies11,22,24,42,46,74,76,78,80,81,87^-91,99–105,111,114,115 reported descriptive associations between functional status/quality of life and decisions about admission to the ICU.

Nutritional status

Only one study18 reported multivariate analyses of nutritional status on admission decisions. In this elderly cohort of patients aged > 80 years, nutritional status was independently associated with eligibility for ICU admission (see Appendix 4, Table 32). One descriptive study76 in patients aged > 80 years found no statistically significant differences in nutritional status between those referred and those not referred to ICU.

Pre-admission length of hospital stay

One study16 reported multivariate analyses for pre-admission length of hospital stay as a factor. One study84 reported univariate analyses where length of hospital stay of > 7 days increased the odds of ICU transfer compared with 1 or 2 days, but this was not an independent predictor in the multivariate model. Five additional descriptive studies12,18,50,68,110 reported previous length of hospital stay and admission decisions to the ICU with conflicting results.

Trajectory of illness

Three studies16,17,76 reported multivariate analyses of trajectory of illness as a factor. Previous hospitalisation in the past year was associated with reduced odds of admission to the ICU in one study,17 but hospitalisation in the past 6 months showed no difference in another. 76 Previous ICU admission during the hospitalisation was associated with increased odds of admission within 2 hours of medical emergency team (MET) activation16 (see Appendix 4, Table 32). Four further studies90,91,105,111 reported descriptive associations between trajectory of illness and ICU admission.

Presence of do-not-resuscitate order (variously described in studies as DNR/DNAR/DNACPR)

Three studies19,68,97 reported multivariate analyses of do not resuscitate orders and admission decisions to the ICU. All three studies showed that patients were less likely to be admitted with a do not resuscitate order.

Six descriptive studies78,89,92,98,116,117 reported associations between presence of do not resuscitate order and admission decisions. Three studies found that it resulted in significantly fewer ICU admissions. 89,116,117 In the others, do not attempt resuscitation status was seen as important98 or clinicians said that they would comply with a do not attempt resuscitation order. 78,92

Patient-related non-medical factors

Age

Eighteen studies12–18,20,21,69–73,76,84,97,106,112 examined age as a factor in multivariate analyses. Data on age were inconsistently reported. Six studies12,15,16,18,20,84 reported admission decisions per year increase in age, and data from these studies were pooled statistically (Figure 5). There was considerable heterogeneity (I² = 99%), but magnitude and direction of effect between studies were similar, showing an increased odds of refusal to ICU with increasing age (OR 0.95, 95% CI 0.91 to 1.0; p = 0.05). Five studies12,16,18,20,84 were conducted in real-world settings and one15 was theoretical.

FIGURE 5.

Forest plot of studies reporting age in multivariate analyses, for each year increase in age and by age bands. IV, instrumental variable; OR, odds ratio; SE, standard error.

Other studies reported specifically on elderly cohorts or similar cut-off points and, where possible, we have pooled these statistically (see Figure 5). All multivariate analyses are summarised in Appendix 4, Table 32.

A further 35 studies (39 publications)10,11,19,22,24,41,68,74,75,77,78,80–83,87–91,93–95,99–101,105,107,109–111,114,118–124 looked at the association between age and admission to ICU in descriptive analyses.

Sex

Seven studies69,72,73,84,112,125,126 examined sex as a factor in multivariate analyses. Results were inconsistent between studies, with different age groups and ethnicity also playing a role.

Three studies73,84,112 reported no difference in admission decisions by sex when controlling for other covariates. One study72 found that being female reduced the odds of refusal to ICU compared with being male (the reference), but this did not reach statistical significance. Three studies69,125,126 found that men were more likely to be admitted than women (see Appendix 4, Table 32).

One study20 reported univariate associations between sex and ICU admission decisions and found no differences between men and women (OR 0.83, 95% CI 0.52 to 1.43). Twenty-two further studies10–13,18,19,21,24,41,50,68,70,74–76,79,81,89,94,106,110,127 reported descriptive associations between sex and decisions around admission to the ICU.

Ethnicity

Four studies73,85,112,125 reported multivariate analyses for ethnicity as a factor, with inconsistent findings. All studies were conducted in the USA.

Five additional studies11,68,75,109,110 reported the effects of ethnicity on admission decisions in descriptive analyses. Ethnic origin did not have an impact on admission decisions in these studies.

Patient preference

Three studies16,17,114 reported multivariate analyses of patient preferences and admission decisions to the ICU. Two studies16,17 of patient cohorts found increased odds of ICU admission when patient preferences for ICU care were considered (resuscitative vs. comfort care, and I accept vs. I refuse ICU care) (see Appendix 4, Table 32). A survey of health-care professionals,114 using a clinical vignette to determine decisions about admission to the last available ICU bed, found that the prior wishes of the patient were not an independent predictor for either physicians or nurses in predicting choice of patient for admission.

No further studies reported the effects of patient preference in univariate analyses, but 15 additional studies22,71,78,80,81,87,90,100,101,105,107,111,116,128,129 reported its effect on admission decisions in descriptive analyses, the majority being questionnaire studies of physician attitudes.

Family preference

One study114 reported multivariate analyses of family preferences and admission decisions to the ICU. This study was regarded as being of low methodological quality and did not show family considerations as an independent factor in affecting admission decisions in a clinical vignette study.

Thirteen additional studies38,46,71,78,80,81,91,96,100,101,105,130,131 reported the effects of family preference on admission decisions in descriptive analyses. The majority of these were questionnaire studies of physician attitudes and perceptions that variably reported the effect of family wishes on decision-making.

Health insurance

Two cohort studies73,112 reported multivariate analyses of health insurance and admission decisions to the ICU, one of which was in a population of patients with diabetic ketoacidosis. Both were US studies and explored differences between types of health insurance (see Appendix 4, Table 32).

Four additional studies22,68,91,98 reported the effects of health insurance on admission decisions in descriptive analyses.

Experiences review

We identified 12 studies (14 publications), presented in Table 2. Eleven44,46,92,115,128,130,141–145 used qualitative and one100 used quantitative methods. Interviews were most commonly used for data collection. Two studies used questionnaires and four used multiple methods (including interviews, focus groups, observation, and analysis of documents). Five of the studies44,46,128,143,144 were carried out in North America and seven92,100,115,130,141,142,145 were carried out in Europe (including three92,141,142 in the UK). In general, the quality of the studies was mixed (see Appendix 2 for a review of quality assessment of studies). All but two of the qualitative studies were single-site or single-participant studies.

Of the 12 studies, two looked at patients’ or relatives’ experiences (having only one participant each): Fulbrook et al. 142 and Todres et al. 141 The other 10 explored the experiences of health-care professionals, including ICU and general ward doctors and nurses, CCOR nurses, medical directors, respiratory technicians, hospital administrators, social workers and bioethicists. One study92 contained a detailed reflection of how a CCOR nurse made her clinical decisions based on one case, rather than a reflection on the wider process. We did not find any papers that focused solely on experiences of the process of referral and admission to ICU, but in these 12 we were able to identify data relevant to our research question.

Participants and decisions observed

In total, 55 decisions to admit or not to ICU were observed on wards or in EDs (Table 3) over 28 days, of which 6 were weekend days. In 18 of these decisions the referral was received after 17.00 and before 09.00 Monday to Friday, and in nine decisions the referral was received at the weekend (of which one was received after 17.00 and before 09.00). These decisions involved 46 patients (27 female; mean age 61 years; age range 19–94 years); nine of these patients had two decisions within 2 days. In these nine cases, the first decision was not to admit.

In total, 42 different ICU doctors were observed, all of whom were interviewed. Thirty-one referring doctors were interviewed about referrals related to observed decision-making. Twenty-eight further specialist doctors who refer to ICU were interviewed (Table 4) (see Appendix 5, Table 33, for further information regarding interviews by site). The acute medical conditions of the patients, as initially reported, are summarised in Table 5.

From our observations, family members of 45 patients were identified as potentially eligible for interview. Seven were not asked for an interview as the clinical team thought that an approach would cause undue distress, and 13 were not available to ask or did not respond to the initial information. Fifteen declined to be interviewed. Therefore, 10 family members were interviewed soon after a decision had been made to admit or not to admit to ICU (two parents, four spouses/partners, five daughters or sons and one daughter-in-law).

We gained consent to approach 13 family members of patients for a late-stage interview approximately 3 months after the observed decision. Of these, four family members did not wish to be interviewed; contact was lost with one family; one patient was readmitted to ICU and the family were not re-contacted; and two patients, after regaining capacity, did not consent for their family to be contacted. One family member was not re-interviewed as their consent for further contact by the research team could not be verified from documentation. Four family members were interviewed about three patients. One family member was interviewed at the time of the decision and 3 months later.

Of the 45 patients for whom we have observation data, 11 agreed to interview 3 months after the decision, but only three were interviewed; the remainder declined or contact was lost. Of the 34 from whom we did not obtain consent for interview, 15 died in hospital, two did not regain capacity while in hospital so were not asked, four declined to participate and 13 were not invited to participate.

Analysis to inform the design of the choice experiment

When designing the choice experiment, we were specifically interested in factors that influenced the decision-making process. Our analysis questions were as follows.

Analysis to inform the design of the decision-support framework, accompanying training and patient and family support

Our analysis questions were designed to capture the whole process of referral and decision-making, including the difficulties encountered and the facilitators of a good decision-making process.

What is considered an ideal process for decision-making?

In this section we report what doctors told us during interviews about what they thought would make an ideal process for decision-making. Data for this analysis come from both comments doctors made when discussing a referral and what they said when asked how the process of admitting patients to ICU could be improved. We mostly summarise what doctors said, limiting illustrative quotations to those relating to issues not already covered in this chapter.

Improved understanding of what is involved

Ideally, there would be understanding among the population generally about what ICU can achieve and what it cannot:

If the population understood that intensive care isn’t magic and that even if you survive intensive care you may not go back to your old self . . . there might not be so much of a push to refer for intensive care.

ICU consultant, interview, index event 2, hospital 6

Many ICU doctors said that they would like greater understanding among other clinical staff about what ICU can achieve and the burden of treatment:

Sometimes there is a misunderstanding about the rigours of intensive care, what it costs a patient to be put through a level 3 stay and the consequences of that for the patient and the family.

ICU consultant, interview, index event 2, hospital 2

A few referring doctors wanted more understanding among ICU doctors about which treatments can be delivered on wards:

In an ideal situation more could be offered outside ITU but we as physicians who don’t work in ITU are very well aware of the limitations of high-level care in any other part of the hospital.

Referring doctor, interview, index event 1, hospital 5

Several doctors said that decisions about care plans and treatment escalation should be made as soon as possible after admission.

Good communication between teams with relevant information

Both referring doctors and ICU doctors saw good communication between them as important:

As long as I can have a discussion with the intensivist on the whole I am pretty happy with the outcome.

Referring doctor, hospital 5

Consultant-to-consultant referral was considered ideal for cases that were not straightforward. However, doctors warned that insisting on this could delay referrals. They were also realistic about what was possible given the system in which they worked:

Very rarely do you have a consultant-to-consultant referral, particularly from the medical teams . . . it is impossible to achieve because of the way the physicians have their work organised.

ICU consultant, interview, index event 3, hospital 5

All doctors noted the importance of having information about the patient available as and when this was needed, but several acknowledged that good information recording did not always occur:

Sometimes they are supplying history that’s just not all the history. It’s not because they’re hiding facts, it’s only because they don’t have the time or they don’t have the resources.

ICU registrar, index event 7, hospital 3

Involvement of patients and their family

Most doctors thought that the ideal process would involve patients and their families and that the doctor would maintain an open mind about what the patient might want:

It’s a very interesting and when you start putting it to patients ‘Do you really want to come down to the intensive unit and have all these tubes and lines?’ And actually they generally don’t because if they’re dying, they know it. But you know sometimes they do [want admission] and sometimes I’ll be persuaded by a family [member]. You have to keep an open mind.

ICU consultant (second interview, admission), interview, index event 1, hospital 4

The patient’s own values regarding quality of life would be taken into account by the doctor making the decision, even when the doctor had assessed that patient as having poor functional status:

We can’t really say that [a patient’s] quality of life would be poor on the simple grounds that people put different value on what quality is. It’s very subjective . . . I had a patient a few months ago who said, ‘If I can walk from my bed to the end of the bed I would think I have a good quality of life’. And there’s another one who’d say, ‘If I can’t go swimming every day I don’t think there’s any reason [to live]’.

ICU registrar, interview, index event 1, hospital 1

Doctors wanted to provide their best possible prediction of outcome for patients and family members, yet they also recognised uncertainty:

What you’ve got to do is refine the probabilities to an extent that you feel you can make a decision but always acknowledge that there is a chance . . . that it’s wrong . . . I say to the family, ‘I think it is likely they’re going to die,’ . . . if they remarkably make a recovery . . . ’I’m very happy!’ That’s fine but I suspect I won’t be wrong.

ICU consultant, interview, index event 4, hospital 2

Several doctors commented on the importance of ensuring that the family receives a consistent message from the clinical team after the decision is made and that families should feel involved in the decision but not responsible for it:

We’ll know what the family’s wishes are but we won’t involve them in the decision-making until we’ve got a medical consensus. There’s no point saying to the family, ‘We would allow it but the neurologists don’t want us to, what do you want us to do?’ that doesn’t work. You’ve got to come to the point where you say, ‘We’re all of the opinion that we continue treatment for 24 hours longer, then review’ . . . it makes it more difficult if the family almost feel like they’ve got to decide.

ICU consultant, interview, index event 7, hospital 2

Holistic assessment of a patient’s needs and willingness to respond to a changing clicinal situation

Doctors commented that, ideally, each deteriorating patient would receive a joint assessment involving the ward and ICU teams before a crisis is reached, although they also reflected that cases are complex and a patient’s illness trajectory is uncertain. Doctors with more experience were more confident in their decision-making, considering the whole person and feeling more able to predict outcome while recognising the uncertainty:

The more experienced you get, the more you’ll tend to look at the whole picture so it’s not just a case of pneumonia . . . Greater experience allows you to predict with more confidence what the likely outcomes are likely to be but not always. So I often think the more I know the less I know because I don’t see the exceptions . . . Generally I think have a better ability to predict.

ICU consultant, interview, index event 2, hospital 1

The ideal process was seen as including a willingness and opportunity to review and change a decision, providing a ‘safety net’ for patients:

The most important thing with intensive care is to not go, ‘No we’re never going to take them’ because the situation changes . . . I’ve been wrong with my gut thinking that someone is on their way to getting better only to see them 24 hours later in a worse state.

ICU consultant, interview, index event 4, hospital 1

Doctors talked about ideally balancing the benefits and burdens of admission to ICU for each patient, including considering other treatment options such as palliative care, although, as we have noted, this explicit balancing was almost never observed during our field work.

Adequate resources, monitoring systems and support

All doctors recognised that ICU doctors described decision-making as easier when resources (staff, beds, equipment) are available and that this has an effect on patient safety. ICU doctors described that ideally ward teams would be willing and able to offer to help with step-down from ICU when the unit is full. Doctors recognised that the downside of the ideal situation of always having sufficient ICU beds is that some patients would be admitted who are borderline and may not need ICU treatment:

Patients are sometimes being picked up who may be wouldn’t have been brought up if the unit was full . . . people who are borderline, who could probably be OK on the ward.

ICU registrar, interview, index event 6, hospital 3

Some doctors suggested that a system to record referrals would improve patient care by facilitating appropriate review of patients who are not admitted:

There’s a ITU log book which works well . . . every patient we get called about goes into a log book so can refer back to that book . . . who’s been referred up and why . . . [What] I’ve offered them [on ward], I’ll review this patient tomorrow and Sunday if required.

ICU registrar, interview, index event 6, hospital 3

Doctors suggested that decisions should be made with another doctor, particularly when cases are borderline or more complex. They noted that collegiate decision-making reduced variation:

During the daytime there’s always two of us working together so we know how each other works and in the daytimes we discuss cases very freely. So I think there isn’t a huge amount of variation.

ICU consultant, interview, index event 2, hospital 1

However, they recognised the cost implications of this approach, particularly out of hours:

If we could have two people of appropriate seniority around at night it would be great but it’s far too expensive.

ICU consultant, interview, index event 2, hospital 6

Another suggestion for improving the consistency of decision-making was to hold regular consultant (or, indeed, wider ICU team) meetings to discuss cases and decisions:

All intensive care consultants meet once in a month . . . even though I have come from different hospitals my practice won’t be any different from the people here because we all sit every month and interact with each other, discuss cases . . . what’s right, what is wrong.

ICU consultant (interview 1, non-admission), interview, index event 1, hospital 5

Both consultants and registrars described the importance of consultants providing support for junior doctors during the decision-making process. This may include providing advice and support on the telephone, speaking directly to the other team’s consultant when there is a misunderstanding or disagreement, or stepping in to take over from their juniors:

If I don’t feel the junior doctor has got enough experience, or that they don’t quite grasp what it is that I am concerned about in the decision-making process, then I’ve got to come in and do it myself.

ICU consultant, interview, index event 4, hospital 5

Figure 6 illustrates the components of the ideal decision-making process as described in our data.

FIGURE 6.

The ideal decision-making process.

What is the range of values associated with the decision-making process?

The analysis reported so far describes the ICU decision-making process and identifies what makes a good, poor or ideal process. In this section we add to this analysis the explicit and implicit values informing and framing the decision-making process identified through our empirical ethics approach. Figure 7 summarises the current ICU admission decision-making process (central column), the values supporting a good decision-making process (left column) and the flaws contributing to a poor process (right column). We identify intrinsic moral values, such as respect, honesty and trust, and the instrumental values that enable the realisation of intrinsic moral values, such as time, analytical skills, communication and mentorship.

FIGURE 7.

The ICU admission decision-making process.

Strengths and limitations

This project has several strengths. We had well-defined research questions and a field researcher familiar with the observation context, which can be an advantage for focused ethnography. 146 Coming from a different European country gave the researcher some degree of an outsider’s perspective considered important for more classical ethnography. 146 The field researcher’s expertise in ethics meant that she was able to draw on relevant frameworks while in the field to inform her conversations and interviews. Field work occurred in a range of sizes of NHS hospitals, and at different times of the day/week, providing a variety of contexts for the ICU admission decisions. Most NHS hospitals work within similar policy guidance and resource constraints, so our findings are likely to be transferable to most NHS hospitals that have an ICU. Most doctors involved in observed decisions were willing to be interviewed. Analysis was undertaken by a team with diverse academic, clinical and research backgrounds, who were able to challenge each other’s assumptions.

Although we achieved some patient/family interviews, we were disappointed not to achieve more, and this could be seen as a key limitation of the project. This was despite taking extensive advice from the PPIAG. Taking part in research in this context is extremely difficult for patients and families. The patient is usually too sick to engage in a discussion about research and their family is too worried about their loved one or too preoccupied with tasks associated with the crisis. More focus on training the clinical and nursing staff in how to support the families to participate in research at such a difficult time may have improved recruitment. 158 We attempted to recruit patients and families from post-ICU clinics to increase our number of participants, but this was also unsuccessful.

Doctors who did not refer during the observation period were interviewed by three different members of the team. This may mean that the data from these doctors were not collected as consistently as other data. However, the team members worked on the interview schedule development together and read each other’s interviews to minimise inconsistency. All members of the analysis team, and the researcher, had a clinical background, so we lacked the perspective that a non-clinical social scientist or ethicist would have brought to the data.

Step 1: identifying patients’ features

A planned interim analysis of data from the initial systematic review (see Chapter 3) and data from two of the six hospitals in our ethnographic study (see Chapter 4) were used to identify relevant patient-related factors. The systematic review identified 88 studies that reported empirical research investigating the factors associated with decisions to admit or refuse admission to ICU. From these studies, we identified a list of factors to map to the list identified from the ethnographic study. We excluded factors that were not relevant to a UK NHS context (e.g. presence of health insurance). For the ethnographic study, we observed 15 ICU referrals and interviewed the consultants, as well as referring clinicians, about their decision-making. We interrogated the observation field notes and interview transcripts for data where we could discern an influence on the decision-making process. These data were coded descriptively and the codes were categorised into factors influencing decision-making. These factors were then mapped to the list of factors identified in the systematic review to check for congruence and any additional factors. The resulting factors were grouped into categories of similar meaning to achieve a workable list of potential choice experiment features. The qualitative data were checked to clarify meaning and to inform the decision to include or exclude an identified factor. For example, the data suggested that ‘communication with colleagues’ was mainly about process and did not influence the decision itself, so this factor was excluded. All of the factors in the final list included in the choice experiment were patient-related and covered the key factor categories from both the systematic review and the ethnographic data. Clinician- and organisation-related factors were reflected in the additional data collected in the choice questionnaire.

Family’s views about patient admission did not appear as an important factor in the systematic review. However, the ethnographic study showed that this was a relevant factor in understanding consultants’ admission decision-making. Moreover, one objective of this study was to determine whether or not patients’ preferences for ICU admission (as approximated by family’s views) significantly influence decision-making.

The next step was to identify levels for each attribute. We divided the attributes into those that related to established clinical guidance or scores such as the National Early Warning Score (NEWS), and those for which we needed to draw on the qualitative data for levels. For the latter, we reinterrogated the data to ensure that we covered the breadth of levels of each attribute. Attribute ‘levels’ were selected to reflect the range of observed clinical situations in our qualitative study. Levels of comorbidities were selected to reflect roughly comparable stages of disease: peridiagnosis, established disease and advanced disease with limited survival. The attributes and levels are presented in Table 8.

Step 2: designing patients’ profiles

The eight patients’ features were experimentally combined to form choice tasks following a multistage strategy. 162 First, we created a draft version of the choice experiment questionnaire based on a D-efficient design optimised for a multinomial logit (MNL) model with null priors (i.e. null a priori information about respondents’ preferences) and sent it to 30 ICU consultants. This quantitative piloting of the questionnaire fulfilled two objectives: to further improve the quality of the questionnaire by identifying wording issues and to obtain a set of preliminary results regarding consultants’ preferences for patients’ admission. We used this knowledge of consultants’ preferences to update the initial design of the experiment. This increased the statistical efficiency of the choice experiment, allowing for a more precise measurement of respondents’ preferences.

As we were interested in measuring the main effects of changes in the eight patients’ features (e.g. increase in patient’s age) on admission decisions, and also potential interaction effects between the type and severity of main comorbidity, it was necessary to include a minimum of 24 choice tasks in the questionnaire. 165 Each choice task includes two hypothetical patient profiles (Figure 8). Participants were asked to answer three choice questions: (1) would you admit patient A? (yes/no); (2) would you admit patient B? (yes/no); and (3) which patient should be admitted in priority? (A/B).

FIGURE 8.

Illustration of the choice task format. Reproduced from Bassford et al. 164 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/. The text below includes minor additions and formatting changes to the original text.

Reproduced from Bassford et al. 164 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/. The text below includes minor additions and formatting changes to the original text.

To reduce cognitive burden, the 24 choice tasks were divided into two blocks of 12 tasks, leading to two versions of the questionnaire. Participants were randomly allocated between these two versions. In addition to the 12 experimental tasks, we manually added a warm-up task (task #1) to familiarise the participants with the format of the choice tasks, and two data quality checks (a repeat of the warm-up task (#14) to test the stability of participants’ preferences for admission and a logical task (task #15) to check participants’ engagement). (More information about the data quality checks can be found in Appendix 6.) The order of the experimental tasks was randomised across participants to minimise the ordering effects that may result from learning/fatigue.

Step 3: creating the choice questionnaire

In addition to the choice tasks, the questionnaire included an information section that carefully explained the context of the study. We also collected information on respondents’ sociodemographic characteristics (e.g. age, sex) to ensure the generalisability of our results. We worked with a UK-based research company (Clinvivo Ltd, Tenterden, UK) to develop an electronic version of the questionnaire. The questionnaire was scripted such that participants could respond on either a standard or a tablet PC. The questionnaires can be found in Report Supplementary Material 7–10.

Sample of intensive care unit consultants

The personal characteristics of the ICU consultants are shown in Table 9. The sample mainly included male consultants (79.5%) who did not work for a university hospital (63.7%); 21.1% were aged < 40 years and 28.1% were aged > 50 years. The majority (76.9%) had been working in ICU for > 10 years.

Analysis of data quality

The results are reported in Table 10. The quality of responses was high, with 73.6% of participants meeting all four quality measures. No participants failed more than two tests; thus, none was removed from subsequent analyses. The criteria for assessing quality are reported in Appendix 6.

Preferences for intensive care unit admission

We modelled consultants’ preferences for ICU admission within the random utility maximisation framework. 167

From the responses, it is possible to derive a full rank ordering of the choice options (e.g. if patients A and B should be admitted but A is prioritised over B, then it is possible to infer that patient A > patient B > no admission). From this implicit ranking, it is possible to analyse consultants’ decisions as if they were made based on three options (i.e. which option do you prefer of admit patient A, admit patient B, and admit neither of them?). To take advantage of this data structure, we jointly analyse the probability of being first ranked (i.e. the MOST preferred choice option) and last ranked (i.e. the LEAST preferred choice option). This approach allows for a more accurate estimate of respondents’ preferences because (1) a choice from three options generated more information about preferences than a choice between two options, and (2) jointly modelling the MOST and LEAST preferred choices generated more observations about respondents’ choice behaviour than MOST preferred choices alone. This structure of the data is inspired by the best–worst scaling literature. 168

For each participant (n), at choice task (t), the probability of the option (j) to be first ranked is noted PntjMOST and its probability of being last ranked is noted PntjLEAST. These probabilities depend on the subjective values of the choice options (U_ntj), determined by a linear additive combination (V_ntj) of patients’ features (X_ntjk) with consultants’ preferences (β_nk) and an error term (ε_ntj) that is typically assumed to be identically and independently distributed as type I extreme value, leading to a MNL specification:

The estimated model parameters (β_nk) capture the effect of a marginal change in patients’ features (e.g. ‘being 66 years old’ vs. ‘being 39 years old’). We report the simulated maximum likelihood estimates of these (β_nk) preference parameters. To account for the panel nature of the data (i.e. multiple choices per participant), we include an individual error term (ω_n) that is assumed to be Normal (0;σ_ω). Because of this error term, the model log-likelihood needs to be simulated. We used 1000 Halton draws in addition to 200 draws as a ‘burn-in’ period and tried different sets of starting values to increase the chance of finding a global solution. To ease the interpretation of the results, we also computed ORs and scores of RI. 169

The results are reported in Table 11. All eight patient features have a significant effect on consultants’ decisions. All three age-related effects are significant and positive, meaning that younger patients are more likely to be admitted to ICU. Patients more likely to be admitted by consultants include those with prostate cancer, mild comorbidity severity, good functional status (mobility), a more severe acute condition (higher NEWS score), those who are not safe in their current (non-ICU) ward, those who receive a negative subjective assessment from the registrar and those for whom family insist on admission are more likely to be admitted by consultants.

Patients’ age has the largest influence on consultants’ decisions (RI 23.9%), with 39-year-old patients 12 times more likely and 66-year-old patients five times more likely to be admitted than 89-year-old patients (the reference group). This is followed by family views (RI 19.9%). When the family is against admission (‘no’), the patient is six times less likely to be admitted than when the family is unsure about admission. The third most important effect is severity of comorbidity (RI 17.9%). Patients with mild comorbidity are 6.4 times more likely to be admitted than those with severe comorbidity. Least important are type of main comorbidity (RI 3.8%), patient’s safety in non-ICU ward (RI 2.5%) and the severity of acute condition/NEWS (RI 7.5%). Patients with COPD, heart failure and dementia are 1.04, 1.34 and 1.48 times less likely to be admitted, respectively, than patients with prostate cancer.

Preferences heterogeneity among intensive care unit consultants

We estimated a latent class logit (LCL) model to investigate preference heterogeneity. 170,171 This model allows the identification of different segments (or classes) of consultants who differ in their preferences for ICU admission. These classes can be thought of as different preference patterns adopted by consultants to decide whether or not patients should be admitted. The LCL model is a popular approach in the choice experiment literature to investigate preferences heterogeneity. Following the literature, we estimate several LCL models with an increasing number of preferences patterns (or latent classes) and retain the solution that optimises a measure of statistical performance known as Bayesian information criteria.

A solution with four preferences patterns provided the best model fit (see Appendix 8). The results for these patterns are presented in Figure 9. In all four patterns, consultants are more likely to admit younger patients, patients with a good functional status, patients whom the registrar reports as ‘tiring’, patients with a less severe comorbidity and patients whose family is not opposed to their admission. All four patterns appear to give little consideration to the level of resource in the non-ICU ward (safety factor) and to the type of comorbidity. However, the four patterns differ in the importance given to the different patient-related factors. A perfectly balanced decision-making process, giving equal weight to all eight patient factors, would be associated with RI scores of 12.5% for each factor (i.e. 100%/8 factors = 12.5%).

FIGURE 9.

Preference heterogeneity among ICU consultants. Reproduced from Bassford et al. 164 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/. The text below includes minor additions and formatting changes to the original text.

Reproduced from Bassford et al. 164 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/. The text below includes minor additions and formatting changes to the original text.

We define the first preference pattern as ‘age-oriented’ decision-making; although consultants in this group consider all patient-related factors (except type of comorbidity and patient safety), they give slightly more weight to the patient’s age (RI 24%). The second preference pattern describes ‘age-dominated’ decision-making because decisions are largely driven by the patient’s age (RI 31%). The third preferences pattern is described as ‘balanced’ decision-making, with consultants giving approximately similar importance to the eight patient-related factors. The last pattern indicates a ‘family-dominated’ decision-making (RI of family views 35%). These four patterns represent 31% (‘age-oriented’), 33.2% (‘age-dominated’), 17.4% (‘balanced’) and 18.4% (‘family-dominated’) of the ICU consultants.

Effects of consultants’ characteristics on preferences heterogeneity

We examined the effects of consultants’ characteristics (see Table 9) on the probability of their belonging to the different preference patterns by re-estimating the LCL model with characteristics as predictors of class membership probabilities. To maintain the structure of the LCL model, we constrained all preference parameters to remain the same, and then only the effects of the characteristics are freely estimated. For mathematical reasons, one class preference pattern has to be omitted, serving as a reference. We specified ‘family-dominated’ as the reference.

From a statistical perspective, the characteristics of the consultants explain very little of the variability in the preference patterns. Regression results are presented in Table 12. Only six effects (out of a possible 30) reach significance at the 5% level. The admission decision-making of consultants aged > 40 years is more likely to be ‘age-oriented’ and ‘balanced’ than ‘family-dominated’ than that of younger consultants. This is especially true for consultants aged > 50 years (as indicated by relatively large ORs). Consultants working in a medium-sized ICU (i.e. 11–19 beds) and those working at a university hospital are less likely to adopt ‘age-oriented’ and ‘balanced’ decision-making, respectively.

Influence of type of comorbidity on the effect of comorbidity severity

We re-estimated the model used for estimating consultants’ preferences, allowing for interaction effects between preferences for type and severity of main comorbidity. The results are presented in Figure 10. Increasing the severity of all comorbidities is associated with a decreased likelihood of admission to the ICU; however, differences are noted across the comorbidities. For mild severity, patients in all four comorbidity groups are more likely to be admitted than ‘patients with severe prostate cancer’. When the severity changes from mild to moderate, the probability of ICU admission falls only in patients with COPD. At the most severe level, dementia is the comorbidity most likely to result in the patient not being admitted to ICU, followed by severe heart failure and severe COPD. This pattern of results indicates that the effects of severity of main comorbidity on consultants’ decisions depend on the type of comorbidity.

FIGURE 10.

Interaction between the effects of type and severity of comorbidity in consultants’ decisions. The dashed line indicates a null effect on consultants’ admission decisions (i.e. OR 1). No CI is reported for severe prostate cancer because it corresponds to the reference category and then this effect was constrained to be null (OR 1). All other effects were estimated relative to this reference category. Reproduced from Bassford et al. 164 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/. The text below includes minor additions and formatting changes to the original text.

Reproduced from Bassford et al. 164 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/. The text below includes minor additions and formatting changes to the original text.

Sample of outreach nurses

The characteristics of the nurses are shown in Table 9. The sample mainly includes female nurses (84.1%) who do not work for a university hospital (59.8%) and are aged 40–49 years (40.2%).

Data quality

Overall, the quality of the choice data was good, with 71.4% of the CCOR nurses fully satisfying all four quality criteria and none of them failing more than two tests (see Table 10). Therefore, all of the observations were kept for the estimation of admission preferences.

Sample preferences of nurses

The results are reported in Table 13. All eight patient-related factors have a significant effect on nurses’ decisions. The patients’ age is the most influential feature, with a score of RI of 21.6%. This is followed by both severity of main comorbidity (RI 17.1%) and severity of acute condition (RI 17.4%). In comparison, two factors appear to have less influence on nurses’ admission decisions: the type of main comorbidity (RI 5.4%) and patient’s safety in ordinary (non-ICU) ward (RI 3.4%). The largest and smallest effects on nurses’ decision are related to age and type of main comorbidity, respectively. On average, a 39-year-old patient is 5.36 times more likely to be admitted than an 89-year-old patient, everything else being equal. On the other hand, a patient with prostate cancer is only 1.06 times more likely to be admitted than a patient with dementia. The second most influential effect is being critically ill: a patient with a NEWS of 11 is, on average, 3.86 times more likely to be admitted than a patient in a less severe condition (NEWS of 5).

Preferences heterogeneity among clinical care outreach nurses

Five preference patterns were identified (see Report Supplementary Material 12). For all classes, nurses give similar importance to severity of comorbidity (RI ranges from 14% to 17%) and patient’s safety in a non-ICU ward (RI ranges from 1% to 5%) (Figure 11). The first preferences pattern gives relatively more importance to family views and functional status, and less importance to type of comorbidity and severity of acute condition (NEWS). Given the importance given to family views (RI 20%), we label this pattern ‘family-oriented’ decision-making. The second preferences pattern gives relatively more importance to patient’s age and registrar’s assessment, and less importance to acute condition (NEWS). Given the importance given to patient’s age (RI 27%), we label this pattern ‘age-oriented’ decision-making. The third pattern gives relatively more importance to type of comorbidity, functional status, severity of acute condition (NEWS) and family views, and relatively less importance to registrar’s assessment. Given that most factors show a similar level of importance, we label this pattern ‘balanced’ decision-making. The fourth pattern gives relatively more importance to severity of acute condition (NEWS) and less importance to functional status and family views. Given the importance given to NEWS (RI 27%), we label this pattern ‘NEWS-oriented’ decision-making. The last pattern gives relatively more importance to severity of acute condition (NEWS) and less importance to functional status, family views and type of comorbidity. This preferences pattern appears to be a more extreme version of the ‘NEWS-oriented’ decision-making, giving even more importance to NEWS (RI 39%), and is then labelled ‘NEWS-dominated’ decision-making.

FIGURE 11.

Preferences heterogeneity among CCOR nurses.

‘Family-oriented’ decision-making represents 66% (n = 125) of the sample; ‘age oriented’ represents 18% (n = 34); and ‘NEWS dominated’ represents 10% (n = 18). The two remaining preference patterns represent very few nurses: 5% (n = 10) and 1% (n = 2) for ‘balanced’ and ‘NEWS oriented’, respectively. Although nurses’ decisions reveal more types of decision-making than do those of ICU consultants, the relatively large market share for ‘family oriented’ suggests that CCOR nurses are less heterogeneous in their admission decisions than consultants.

Effect of nurses’ characteristics on preferences heterogeneity

The results are presented in Table 14. From a statistical perspective, allowing the characteristics of the nurses to predict decision-making styles leads to a minor improvement in performance. Although the log-likelihood of the model is improved (–3681.8 → –3662.9), this comes at the expense of 40 additional model parameters, leading to poorer Bayesian information criterion (8155 → 8454). Only four effects (out of a possible 40) reach significance at the 5% level, suggesting a limited contribution of characteristics to heterogeneity in preferences. Everything else being equal, CCOR nurses working in a larger ICU (i.e. number of beds ≥ 20) are more likely to adopt ‘NEWS-dominated’ decision-making than nurses working in a smaller ICU.

Influence of type of comorbidity on the effect of comorbidity severity

As with ICU consultants, there is evidence of an interaction between type and severity of comorbidity in CCOR nurses’ admission preferences (Figure 12). As anticipated, patients with mild comorbidity are more likely to be recommended for admission than those with severe prostate cancer. However, the magnitude of this effect depends on the type of comorbidity. Everything else being equal, patients with mild prostate cancer are more likely to be recommended for admission than patients with mild dementia, COPD or heart failure. This pattern tends to be stable across severity levels.

FIGURE 12.

Interaction between the effects of type and severity of comorbidity in nurses’ decisions. The dashed line indicates a null effect on nurses’ admission decisions (i.e. OR 1). No CI is reported for severe prostate cancer because it corresponds to the reference category and then this effect was constrained to be null (OR 1). All other effects were estimated relative to this reference category.

Strengths and weaknesses of the study

The study uses a validated and powerful preferences-elicitation method (namely choice experiment) to understand how consultants and CCOR nurses make admission decisions. We recruited a large representative sample of ICU consultants. We achieved a relatively high level of data quality, indicating that the preferences-elicitation instrument was well accepted by participants. We used a combination of ethnographic observations and systematic literature reviews to identify the most relevant features. A difficulty with all choice experiments, including this one, is that participants are asked to make hypothetical decisions, in this case regarding ICU admission. Although the choice tasks were created to mimic real-life situations, we cannot guarantee that ICU consultants and CCOR nurses would have made the same choices if the tasks had been real.

Initial development of the decision-support intervention

Developing a conceptual/cognitive framework

The first step was to develop a ‘best practice’ conceptual framework for decision-making, based on our previous work, that encapsulated what is important/essential to patients and clinicians in practice and what is ethically required.

Ethical reasoning is an integral part of clinical decision-making but is often implicit rather than explicit. Findings from our systematic review and ethnographic study show that clinicians’ values influence their decision-making, and that they do not explicitly balance different empirical and normative considerations when making a decision. There are some ethical frameworks and heuristics used in teaching medical ethics, most commonly a principlist approach. However, it seems unlikely from our observation work that providing a theoretical ethical framework for use in day-to-day practice would be successful. In seeking to develop a framework that could be embedded in daily practice, be congruent with clinical decision-making and prompt ethical reasoning, we identified the ‘accountability for reasonableness’ (AFR) framework157 as the most useful model on which to build our DSF. This is a commonly used ethical framework for considering the allocation of limited medical resources and focuses on the process of decision-making rather than relying on a specific moral theory. AFR has four requirements: decisions must be transparent, based on reasons that stakeholders can agree are relevant and revisable in the light of new evidence and arguments, and there should be an appeals process. Other authors have highlighted the requirement for priority setting decisions by clinicians (including ICU admissions) to be transparent (AFR requirement 1) and ethically justifiable (AFR requirements 2 and 3). 173,174 Therefore, any framework to support decision-making should improve the transparency and ethical justification of decisions about referral and admission to ICU. A further ethical requirement is one of equity, which can be interpreted as consistency of process (the factors taken into account and the decision-making process should be consistent across all patients and all decision-makers).

Process of developing the framework

A generic model of the steps in the decision-making process was created based on the findings from our ethnographic study (Figure 14).

FIGURE 14.

The generic model decision-making process. Reproduced from Rees et al. 175 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/. The text below includes minor additions and formatting changes to the original text.

Reproduced from Rees et al. 175 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/. The text below includes minor additions and formatting changes to the original text.

Clinical and non-clinical members of the research team were then asked to map the factors identified in the qualitative study and systematic reviews (tabulated in Appendix 10) to relevant stages of the process. From this, a draft framework was developed, specifying content at each stage in the decision-making process and ensuring that overall it was consistent with the AFR ethical requirements. The resulting framework was discussed and refined in a series of meetings involving members of the research team and two members of our PPIAG. The draft framework was then used as a basis for developing the supporting materials (Table 15).

TABLE 15 - Draft DSF

Stage in process	Factors to consider
Gathering relevant information	Clinical information about current acute illness and relevant chronic illness Functional reserve of patient Patient values, wishes and relevant outcomes Need to recognise clinician biases in interpretation of information
Assessing and evaluating the situation	Ward situation/resources available ICU resource availability (beds/nursing staff) Urgency of decision Experience of clinicians involved in decision
Balancing the ethical considerations in order to come to a decision	Explicit balancing of potential harms and benefits and reason for decision
Making and communicating the decision	Specifically informing medical and nursing staff of decision and the reasons for it Communicating the decision and reasons for it to the patient and/or their family
Planning of ongoing care	Specific documentation of plan for review, including who, when and in what circumstances
Arbitration in cases of disagreement	System and process in place to support this

Consensus conference

A 1-day conference was held in July 2016 at the University of Warwick to gain the views of a wider stakeholder population regarding the content of the DSI and the facilitators of and barriers to its implementation.

Delegates were invited from the following stakeholder groups (see Appendix 15 for the list of delegate representatives):

• Patients –

  • people who had survived a critical illness

  • patient groups who may be particularly likely to need intensive care treatment (e.g. those with heart disease or lung disease)

  • patient groups who may be disadvantaged in terms of access to health care (e.g. those with a learning disability)

• ICU doctors (including professional body representation)

• ICU nurses and CCOR nurses (including professional body representation)

• referring specialty clinicians (e.g. cardiology, renal, elderly medicine, respiratory, surgery)

• regulatory authority, legal and health-care management representation.

Post-consensus conference development of the decision-support intervention

A conference report, including the summary of key points and the changes made to the intervention in response, was sent to all delegates with an invitation to provide further comment (see Appendix 16). The DSI was then finalised for use in the implementation feasibility study. The changes made to the components of the DSI are outlined below.

Guidance for referral for intensive care support

We developed guidance to improve communication between clinical teams around referral. The guidance is summarised as follows:

1. Referral to intensive care should not be delegated to junior members of the clinical team.

2. Referring teams should clearly articulate what support they are seeking from the intensive care team.

3. The benefits and burdens of intensive care should be considered for each patient.

4. Ensure that the patient and/or their family has been consulted in the referral process.

5. The referral should follow recognised best practice using the SBAR format.

The referral form (see Appendix 11) provides a structured format for information gathering and communication with the intensive care clinicians to facilitate a collaborative decision-making process.

A decision-support framework to guide intensive care unit doctors when making decisions

The framework describes best practice for decision-making in three domains: (1) evidence – the collection of information relevant to the decision; (2) reasoning – using this evidence to identify relevant outcomes, balance the burdens and benefits of treatments and make recommendations for treatment; and (3) implementation – ensuring that decisions are acted on and communicated effectively. The framework aims to integrate ethical reasoning within the decision-making process in a practical and accessible manner. The decision-support form (see Appendix 12) enables the decision-making process to be recorded to facilitate transparency and consistency of decision-making.

A pocket-sized aide-mémoire (Figure 15), which has the referral guidance on one side and the framework for decision-making on the other, was provided to act as a reminder of the referral guidance and decision framework to referring teams and ICU doctors.

FIGURE 15.

A pocket-sized summary of the cognitive framework to act as an aide-mémoire.

Patient and family support information

Patient and family information leaflets (see Appendices 13 and 14) to be given to patients and/or their families/someone close to them when they are recognised as being critically unwell and a decision regarding ICU referral is being considered. The leaflets are intended to support discussion between clinicians and the patient/family and not to be given in isolation.

Educational resources

Educational resources to support the implementation of the DSI included (see Report Supplementary Material 14):

1. ‘train-the-champion’ session for implementation champions (including an introductory presentation to the legal and ethical background)

2. a presentation of the DSI and its rationale at hospital Grand Rounds or other formal meetings

3. a shorter presentation for use at more informal or smaller departmental meetings, for example quality improvement meetings

4. a brief ‘key points’ presentation for opportunistic teaching of individuals or small groups

5. a table-top case-based simulation exercise to familiarise clinicians with the intervention.

The final version of all components of the DSI formed the basis of our implementation feasibility study (see Chapter 7).

Settings and recruitment

A description of the study was circulated to intensive care leads through the National Institute for Health Research Clinical Research Network. ICU leads were asked to indicate their interest in participating in the study and to state the number of ICU beds in their trust. We purposively sampled by size of ICU: one small (< 20 ICU beds), one medium (20–30 ICU beds) and one large (> 30 ICU beds).

We worked with the principal investigator at each selected trust to identify implementation champions, who were tasked with planning and running the implementation. The identified champions were, in site A, an ICU consultant and a CCOR nurse; in site B, two ICU senior trainees (registrars); and, in site C, an ICU consultant and two CCOR nurses sharing the role.

Pre implementation

Before the DSI was implemented at each site, we ran a 1-day training session for the implementation champions. The day included:

1. Presentations on the background to the study and findings from the early workstreams, and an introduction to the ethical and legal considerations in decision-making for the critically ill patient.

2. A detailed description and explanation of the intervention, with particular focus on the DSF and its documentation, the referral process, and patient and family information leaflets.

3. An introduction to the range of educational support materials provided for the implementation champions to use in engaging with staff in their trust about implementing the intervention.

4. An interactive discussion on the barriers to and facilitators of implementation in the specific context of the individual site. The implementation champions used this to develop a detailed implementation plan appropriate for their trust, detailing which referral wards were included and a timetable.

Each site was provided with copies of DSI documentation. Coloured and clearly labelled boxes that included referral forms, decision-support forms, and patient and family information leaflets were positioned on ICU and wards identified as likely referrers. In sites with CCOR nurses, the outreach team also had copies of the forms to take with them to wards when seeing patients. Patient and family information leaflets were translated into the two non-English languages most commonly spoken by patients in the relevant trust. The implementation champions were given a resource pack (hard copy and electronic) that included education materials and presentations for training.

Data collection

An initial 8-week period preceded the main data collection period for the implementation champions to implement the intervention. During that time, we conducted audio-recorded interviews (telephone or face to face) with the implementation champions at approximately fortnightly intervals (see Report Supplementary Material 16). We also observed two training sessions at each hospital (see Report Supplementary Material 17) and made field notes.

The subsequent 6 weeks was the main data collection period. To assess form use, including consistency, we aimed to identify, from ICU referral logs, the clinical records of all patients referred to ICU during the data collection period. One site already kept a detailed log, but the other two sites had to create a new system. We did not seek clinical records of logged referrals from wards not included in the study or for planned ICU admissions. From the clinical records of referred patients, we extracted the following data: the time and date of the ICU referral/review; the patient’s year of birth and sex; and the name and role of the person referring/reviewing. If a form was used, the presence or absence of information in each section was recorded. Information on daily ICU bed occupancy was obtained from the ICU manager.

Acceptability of the intervention was assessed in interviews with ICU doctors recieving a referral and doctors who had referred a patient during that period (see Report Supplementary Material 18 for the topic guide). We used the ICU referral logs and clinical records collected during the data collection period to identify potential participants, purposively sampling to include a mix of grades, specialties of (referring) doctors and those who had/had not used a form when referring/reviewing a patient. In one hospital it was not possible to reach the desired sample based on the referral log owing to the low number of logged referrals, and therefore the local principal investigator identified additional doctors to participate. We conducted a group interview with the CCOR nurses at one site. Interviews were semistructured. Interviewees were asked their thoughts on the DSI overall. A referral or decision form (either blank or completed by them if available), and details of a specific referral were used in the interview to prompt recollection of using/not using a form in specific cases. Interviews were audio-recorded with written consent from participants. We held a debriefing meeting at each site with the implementation champions, reflecting on the intervention implementation process. This was audio-recorded and field notes were taken.

Data analysis

Implementation strategies

Each site took a different approach to implementation (see Appendix 17). Hospitals A and C opted to roll out the intervention to the whole site, whereas Hospital B identified three clinical departments: haematology/oncology, respiratory and ED (excluding out-of-hospital cardiac arrests). Each hospital placed boxes containing study forms in prominent positions on referring wards and on the ICU. Implementation champions at all three sites used the presentations provided by the research team at the Grand Round and departmental meetings, modifying them to make them trust-specific. The CCOR implementation champions did not use these resources, preferring to explain the study at team meetings, individually at shift changeovers and/or by utilising a group message service to remind colleagues about the study.

Challenges and enablers to implementation at the sites

Consistency of form usage

The extent of form use varied between sites, with uptake at hospital A higher than at the other two hospitals (see Appendix 18, Figures 21–35 and Tables 38–46, for further analysis). Forms were used in 44.4% (28/63) of examined referrals in hospital A. Among examined referrals, hospital B achieved a form use rate of 21.4% (3/14), and hospital C achieved a rate of 19.2% (20/104).

Overall, 28.2% of referrals included a referral and/or decision form. Documentation of the ICU referral and/or review was frequently absent from the patient’s record and, if present, it often did not include the name/role of the referrer or ICU doctor. When these could be found in the notes, the details of the doctor (e.g. name and role) were frequently absent (see Appendix 19).

In total, 45 completed referral forms and 36 decision forms were identified across all three sites, with both forms being used in 30 cases. The referral forms were used more often (n = 45, 25%) than the decision forms (n = 36, 20%). If a referral form was present, there was a higher chance that a decision form would be present (16.6% vs. 3.3%; p < 0.001).

Ease of use

Those doctors who used the form generally felt that it was easy to use.

It wasn’t difficult . . . If we do this for all the patients we send in I don’t have an issue with that.

Referring doctor 6, hospital C, consultant, used form

The writing on my part is actually relatively straightforward.

ICU doctor 1, hospital A, registrar, used form

Some doctors described how use or knowledge of the framework and forms informed their decision-making even in cases where the form was not readily available.

Did it prompt you to talk to the patient?

No I already had because I knew what was on the form. So it had prompted me . . . I already knew what was on the form, what I needed to fill in.

Consultant, used form

Difficulties encountered in using the form were largely in relation to the ‘capacity to recover’ and ‘benefits and burdens’ sections. Identifying and articulating the benefits and, particularly, the burdens of care, was challenging for ICU doctors:

There are risks of developing all sorts of complications in critical care. Trying to predict what they’re going to be is quite difficult.

ICU doctor 3, hospital B, registrar, never used form

They’re quite hard to write how you would, how would you write this down? . . . The burdens are quite hard to articulate, although we know they’re there and we know they’re profound.

ICU doctor 2, hospital C, registrar, used form

The quantitative data also supported this finding. The benefits were documented more frequently (86.1%, n = 31) than the burdens (69.4%, n = 25).

Concerns about workload

A common concern about using the forms was the creation of extra work and duplication of effort in documenting information twice (in the patient record and on the form), and the effect on the doctor’s time for their other obligations both to the patient in question and to other patients:

It needs to not to be a replication of what we need to document in the notes.

Referring doctor 6, hospital A, acute medical consultant, used form

Just another form really . . . Feeling overwhelmed with paperwork.

Referring doctor 3, hospital B, haematology/oncology consultant, never used form

The intention was that the referral and decision-support forms would form part of the patient record and hence information would not need to be duplicated; however, some doctors recorded it in both because the form was not trust-approved documentation. Our implementation champions in post-study debrief meetings suggested that successful ongoing implementation would require the forms to be an integral part of the trust’s electronic patient record system to demonstrate trust approval and facilitate ease of use. The ICU doctors interviewed felt that it would be easier to direct referrers to an online form than to a box on a ward, and the electronic system could prompt the reviewer to use the decision form:

If you come, sort of come in as a critical care person taking the referral, [and] the form comes up then it would be easier to fill in.

ICU doctor 8, hospital C, consultant, used form

Linked to concerns over duplication was the view of many doctors that they were already doing what the forms were prompting them to do:

[It’s] pretty similar to what we already do just on a piece of paper.

Referring doctor 3, hospital B, consultant, never used form

I think it probably err is duplication of the full assessment that we write in the notes anyway.

ICU doctor 5, hospital B, registrar, never used form

Our sample may have been biased in favour of doctors who already document decisions well, given that those in our sample were identified through patient records. The quantitative data showed that doctors, particularly at hospitals B and C, often did not document their referral or ICU reviews in the patient records before the patient’s admission to ICU, suggesting that doctors had an inaccurate perception of how reliably they recorded referrals and reviews in the patient records.

A further reason why completing the forms was seen as extra work was that the forms, particularly referral forms, were introduced or used at an inappropriate point in the decision-making process. The referring team were usually prompted to use the form by ICU doctors or CCOR nurses following an initial referral of a patient to ICU, and thus the forms were introduced after a decision had been made to refer rather than being used as part of the process of clarifying the reasons for a referral:

I’d already had to ring up and . . . hand over everything on the phone and it was all documented in the notes. I just felt like I was kind of copying what I’d already done.

Referring doctor 5, hospital A, registrar, used form

I think if it was to be used long-term then it would be better on the wards because then they would be thinking to do it before calling us.

CCOR nurse 2, hospital C

Perceived threat to clinical authority and expertise

Some doctors saw the introduction of referral or decision-support forms as a question about their clinical judgement by either implying that they needed assistance to make a decision or requiring them to provide written evidence of their thought processes:

One could argue it’s intensive care saying: ‘You guys don’t know what you’re talking about, we’ll make the decision for you’.

Referring doctor 7, hospital C, consultant, used form

I don’t need it to help me make a decision because like otherwise what have I been doing for the last 10 years?

ICU doctor 1, hospital B, consultant, never used form

It’s almost saying y’know, ‘Here’s this thing that you’ve been doing for 20 years, just write down how you do it please?’

ICU doctor 2, hospital B, consultant, never used form

This sense of having their expertise questioned was described only by consultants, and, with one exception, by those who had never used the form.

The particular context of the emergency department

At all sites, uptake of the forms in ED was very low. Concerns about the lack of time to complete paperwork, expressed by many participants, were particularly evident among ED physicians:

We are too busy to complete forms, it’s as simple as that.

Referring doctor 4, hospital A, ED consultant, never used form

It’s basically regurgitating my notes that I’ve already done . . . So I’m not saying it’s not good but it’s not practical.

Referring doctor 4, Hospital C, Emergency medicine registrar, never used form

Another feature of the ED context described by participants is that a request for ICU support is one element in a complex web of multiple decisions being made in an emergency situation, something that the ICU referral form could not capture fully. This is in contrast to referrals for patients on a ward where the decision is usually more discrete:

This is only pertinent to referring them to ICU. There is a whole load of other stuff that needs to be in those notes because . . . our notes involve multiple focuses . . . it’s not just ICU we interact with . . . we’re such a junction box.

Referring doctor 5, Hospital B, ED consultant, used form

The complexity and urgency of a clinical situation in which doctors are dealing with a severely ill patient in the ED means that the decision-making process is much more fluid, with doctors taking on new information and revising their treatment plans often within minutes rather than hours. Communication is rapid and verbal rather than written, and formal documentation is likely to be done retrospectively once the patient is stable:

If they have anaphylaxis or GCS [Glasgow Coma Scale] of 3 with head injury and things how am I meant to know all of this background and things? They just collapsed in front of A&E [accident and emergency] and I have no medical history and think ‘I want you to be here to tube for me so that you look after the patient until I collate information and look at [electronic records system]’.

Referring doctor 4, hospital C, ED registrar, never used form

I’m constantly scoping for new information if I need to update and revise my decision and that’s, y’know that’s the whole modus operandi we have to be in.

Referring doctor 5, hospital B, ED consultant, used form

Intensive care unit doctors were also reluctant to prompt ED doctors to complete a form when making a referral:

What would’ve happened if you’d gone in and asked for their form?

[Laughs] They would’ve said ‘Fine you, you know you be the one to come and do this guy’s breathing for him and I’ll fill out a form’. 10 minutes later . . . and this guy arrests and dies in front of me then I’m, y’know he’s going to be like ‘Well . . .?’ So . . . it’s not appropriate at all.

ICU registrar, never used form

However, not all clinical situations in the ED are so time-critical that a more considered assessment prior to ICU referral is not possible, and, once the patient is stabilised in the ED, the decision-support form could be helpful in structuring a decision about whether or not admission to ICU is the best next step. As one ICU registrar commented:

[ED is] like a busy bazaar so it might be, sort of particularly adds a little bit of resting quiet normality to an otherwise slightly potentially mad referral with a lot of noise and a lot of things going on in the background.

ICU doctor 6, hospital B, registrar, used form

Improving articulation and communication of referrals and decisions

Doctors who used the form often reported that it helped them to clearly set out the rationale for their decision and provided reassurance that their decision was sound:

It makes us as the referrers focus on exactly why we’re referring that patient.

Referring doctor 3, hospital A, acute medical consultant, used form

It just cemented and convinced me that actually I was doing the right thing . . . It gives you just a bit more err support in your decision-making so you can show . . . this is what I felt at the time.

ICU doctor 6, hospital B, registrar, used form

They also found that it helped with communication between doctors and teams, and promoted mutual understanding of decisions:

It helped me frame the conversation I was going to have with the ITU registrar in terms of this patient, OK she’s possibly borderline, she’s elderly, got some other comorbidities, not your kind of clear-cut ITU case but she’s functioning independently and her family say she’s got a good quality of life and I’ve got some evidence there to say that actually she should be, she’s probably got a reserve that could be managed in ITU.

Referring doctor 2, hospital A, registrar, used form

I guess it’s just to have better communication with the . . . patient’s primary team . . . to understand what our thought process is and why we sometimes would be a bit reluctant to admit a patient.

ICU doctor 4, hospital B, registrar, used form

Doctors thought that the forms were particularly useful for borderline cases where the arguments for and against admission to ICU were more nuanced and clear articulation of the reasoning was essential. For more clear-cut decisions, they saw the forms more as a checklist to record that everything had been considered:

I find this most useful for the kind of grey patients . . . the ones where the decision-making might be quite nuanced . . . The ones that it’s more obvious to me they should be going to ITU I’d see it as a . . . tick-box exercise.

Referring doctor 2, hospital A, registrar, used form

None of that’s relevant because they’re 21 and they’ve taken an overdose so y’know, but then for a 90-year-old from a nursing home that particular box [benefits and burdens] becomes much more crucial.

ICU doctor 6, hospital A, consultant, used form

Our quantitative data showed that older patient age was associated with higher use of both referral and decision forms and that decision forms were more likely to be used when patients had been in hospital longer than 2 days (p = 0.6994). Older patients are more likely to have complex comorbidities, and if patients deteriorate following admission then this may also suggest more complex situations that would support the interviewees’ views that they were more likely to use the forms here.

Using forms may facilitate the articulation and communication of relevant information and reasoning, but it does not guarantee that this will occur:

I’ve just looked at the e-records, this patient has metastatic cancer. He has got a colostomy, he’s got severe kyphoscoliosis, he had severe obstructive lung disease and [referrer] called him ‘fit and well’ . . . There are people who are putting the forms like that to me. And I’ve found myself seething in anger.

ICU doctor 2, hospital A, consultant, used form

The fact that this patient is err paranoid and refusing treatment isn’t even mentioned [on the completed referral form] . . . and that’s the main problem with this patient.

ICU doctor B, hospital C, consultant, used form

Failing to provide information or providing inaccurate information on a referral form might provoke more irritation than simply failing to mention it in a telephone call or on an entry in the medical record.

Considering patient wishes and values

Involving patients and/or their families in decisions about their care is both a legal and a moral requirement and a central tenet of professional guidance. 180 Data from our ethnographic study suggested that this was uncommon in the context of decisions regarding admission to ICU. Several referring doctor participants noted that the forms prompted them to specifically consider and document the views of the patient, something that they would not usually have done in this context:

I think the most important bit was actually speaking to the patient about their wishes. That I wouldn’t automatically think about.

Referring doctor 6, hospital B, registrar, used form

This form definitely made me write it in this case anyway . . . It’s a good prompt and something that you should think about documenting every time even if you’re not doing this form.

ICU doctor 7, hospital C, registrar, used form

Intensive care unit doctors often hoped or found that the form would encourage referring doctors to think about this earlier, and this was seen as a major benefit of the referral forms:

Trying to get the referring team in particular to, think more carefully and more sensibly about their patients and what they want for them and what’s in their best interests, and them to document that in advance of the patient falling in a heap would be very useful . . . I can think of a patient actually downstairs whereby the form prompted them to go and have that discussion. And in fact that patient didn’t come to ICU.

ICU doctor 3, hospital C, consultant, used form

However, a few participants expressed doubt about the usefulness of seeking the patient’s views about admission to ICU, given the time constraints in emergency situations and the difficulty for patients to have a sufficient understanding of both their condition and the nature of intensive care treatment:

Whether the patient can express coherently and with knowledge their own wishes given the complexity of their cases.

Referring doctor 2, hospital B, consultant, never used form

You [patient] don’t really understand . . . You don’t know what the pain and suffering of intensive care is.

ICU doctor 2, hospital C, registrar, used form

If someone is peri-arrest you can’t ask them.

Referring doctor 2, hospital C, registrar, never used form

The patient and family information leaflets were not implemented successfully at any of the sites, and, in general, participants were unaware of them. When asked to comment on them, participants thought that the concept was useful but noted practical difficulties around when to provide them to patients and their family:

Seeing it [ICU] can be quite scary and an information leaflet kind of warning them of what to expect is, is helpful.

Referring doctor 4, hospital A, registrar, used form

I don’t think he would have read it to be honest in that particular situation . . . I would probably prefer to use them in the less acute situation.

Referring doctor 6, hospital B, registrar, used form

I think it’s perfectly appropriate but I don’t think it’s going to work in the real world.

Referring doctor 6, hospital A, consultant, used form

The view that providing patient and family information leaflets would distress or overwhelm patients and family was reiterated in our debrief discussions with the implementation champions at all sites. A further practical problem, also noted with the referral forms, related to the timing of the use of the leaflets. We had envisaged these as supporting discussion with patients and/or their family about a decision to refer to ICU. However, as the prompt to use these usually occurred once a referral had been initiated, the usefulness of the leaflets was called into question.

These data indicate that our intervention prompted involvement of patients and family, and the documentation of these discussions, but revealed that doctors often struggle to make these conversations meaningful.

Improving documentation

Clear documentation of clinical decisions in a patient’s record is considered a key element of good practice,180 but evidence from our review of patient records shows that this does not always occur in the context of ICU referrals and reviews. Participants reflected that the structured documentation prompted by the forms improved transparency and accountability of decision-making:

Introducing an element of . . . accountability for that referral conversation is really, really useful.

Referring doctor 1, hospital B, registrar, never used form

It’s a good way of making sure that everyone documents the same . . . type of information.

ICU doctor 3, hospital A, registrar, used form

Although some doctors felt that they already documented their decision-making, most agreed that the form produced better documentation, particularly of the patient’s values and wishes, and of the benefits and burdens of ICU. They commented that documentation varied depending on whether or not a patient was to be admitted to ICU:

If I know I’m going to accept the patient I won’t justify the decision to accept in the notes because I’ve accepted them. It’s if I have to refuse the patient and then the documentation is different.

ICU doctor 2, hospital C, registrar, used form

If you go to a ward to see someone and decide to admit them sometimes there’s very little written in the notes on the basis that it’s all going to get repeated . . . [in] the clerking admission notes.

ICU doctor 6, hospital C, consultant, used form

However, our quantitative data do not support this suggestion. The majority of cases where the decision form was used were admitted (25/36, 69.4% of cases), which reflects the numbers of cases admitted overall (71.5%).

Even doctors who did not see a need for the form to aid their decision-making acknowledged that the improved documentation facilitated transparency, which could be beneficial for the doctor as well as the patient:

This is what I’ve done my whole life . . . it’s just a written-down version of what I do every day so . . . probably the most helpful thing for me is to get me to write it down rather than help me make the decision . . . [It] might save me from going to jail because I will have written things down in a more thorough format when anything goes wrong.

ICU doctor 1, hospital B, consultant, never used form

Strengths and limitations of the implementation feasibility study

Our implementation sites reflected the range of ICU settings in the NHS. The doctors interviewed were diverse in terms of specialty, grade, and use of the intervention documentation. Interviews focused on specific cases rather than hypothetical scenarios or generalisations. As the researcher was a non-clinician there was less risk of implicit assumptions about clinical practice being shared between researcher and interviewees. This would reduce the potential for lack of clarity of meaning in the data.

The 8-week implementation period was too short for full implementation. Retrieving clinical records during the time frame of the evaluation was challenging. Many referrers did not write their names clearly in the patient record so we were unable to identify them to request an interview. Although many challenges of its use were raised by participants, it is possible that those willing to take part in the interviews were more positive about the intervention. Poor documentation in patient records resulted in missing data so some of our results should be interpreted with caution. We were nevertheless able to identify differences in form use that were supported and complemented by the qualitative findings. The limited, and variable, uptake of the intervention across sites and the observed and reported challenges of implementing a system-wide change in practice make it difficult to come to a definite conclusion regarding implementation feasibility.

Although we observed formal training given by the implementation champions, we did not observe informal educational opportunities and nor did we observe referrals as they happened. We did not involve patients or their families in the feasibility study as participants, and so their voices are missing from the data. If the intervention is implemented more widely, future research will need to explore the process from the perspective of patients and families.

Method

We searched MEDLINE, EMBASE, PsycINFO and Web of Science (Science Citation Index and Social Sciences Citation Index) via Ovid using specific MeSH terms. We combined MeSH terms referring to clinical ethics, and included ethical decision-making; ethical decision; ethical value; moral deliberation; moral case deliberation; moral value; ethical deliberation; ethics consultation; ethics support; ethics education; ethics framework; ethical framework; evaluation; and outcome. We then combined the search results from all of the databases and removed duplicate articles. In addition, we searched grey literature using the same key terms. Our initial search was run on 21 March 2016. We repeated the search on 21 March 2018 to capture any studies published since that date (see Appendix 20, Tables 48–52, for search strategies).

We included empirical studies that evaluated interventions aimed at improving ethical decision-making in clinical care. Articles included also had to describe measures or tools that evaluated one or more components of the intervention(s). The combined searches yielded 4037 papers after deduplication (400 from the updated search in 2018) (Figure 16). Two primary reviewers (AI and AS) independently screened all included papers on title and abstract and identified 79 papers for full-text review. All full-text papers were read by both reviewers, and disagreement about final inclusion was resolved after discussion or by a third reviewer (KR). During the full-text review process, another three papers were identified from a bibliography search of included papers. Fourteen papers (12 studies)182–193–195 were included for data abstraction. All included studies were assessed for methodological quality of the evaluation tool development using an adapted version of the COSMIN (COnsensus-based Standards for the selection of health status Measurement INstruments) checklist (see Appendix 21, Table 53). 196

FIGURE 16.

The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flow diagram for evaluation tool systematic review.

Results

Overall, the quality of the identified studies was variable, with some studies reporting detailed validity and reliability testing and others not describing any testing of the instrument. All but one of the studies described either the development of an evaluation tool or the use of an existing tool to assess educational interventions to improve ethical decision-making in medical or nursing students. The educational interventions were diverse and included a general medical curriculum with some lectures and discussion relating to ethics;182 an ethics course as part of a medical or nursing curriculum;183–186 an integrated ethics thread in a medical curriculum;187,188 a specific educational tool for teaching ethics in a nursing curriculum (guided design189); and a general medical or nursing undergraduate curriculum. 190–193,197 Of these 11 studies, seven182–189,192 described evaluation tools based on written assessments, two190,191 described an objective structured clinical examination (OSCE) station, and one193 described a combination of performance-based assessment and a standardised patient and written assessment of the same case.

Of the 10 studies describing written evaluation tools, two used the Judging About Nursing Decisions Test,198 which assesses nurses’ ability to judge which course of action in a series of scenarios most closely accords with the American Nurses Association’s code of ethics, and how likely the participant is to follow it. 189,197 One study182 modified the Defining Issues Test (DIT) and combined it with another instrument (the Problem Identification Test). Both of these tests were questionnaire surveys based on vignettes in which participants were asked to list the ethical issues in the vignette (Problem Identification Test) and choose the most suitable action from a list (DIT). The remaining seven studies described the development of new instruments. Six used case vignettes, although the number of cases varied. Three instruments183–185 required students to state what they would do in each case vignette and justify their reasons, while one instrument asked students to select an action from a prespecified list for each vignette and then to justify their chosen action. 187,188,192 The final instrument193 combined written assessment with a performance based assessment of ethical decision-making. One further instrument,186 the nursing ethical decision-making ability scale, was not clearly enough described in the study to establish whether or not case vignettes were used.

Among the performance-based assessments, the two studies190,191 evaluating OSCE stations found that inter-rater reliability was acceptable [intraclass correlation coefficient (ICC) of 0.3–0.89 across six stations in one study and 0.44–0.8 across four in the other], but internal consistency was low. The authors concluded that the OSCE assessment was not a feasible standalone method for the evaluation of clinical ethics. In the third performance-based assessment study,193 no report was given of the reliability of the assessment tool. A summary of the studies included in the review can be found in Appendix 21.

Only one study,194 which was identified in our 2018 updated search, described an instrument for evaluating ethical decision-making in actual clinical practice rather than using hypothetical scenarios. This instrument was developed to evaluate clinical ethics case consultation, which provides advice to clinicians making decisions about patient care rather than ethical decision-making by the clinician responsible for the patient’s care. However, in evaluating ethical reasoning and decision-making in actual clinical cases, it is the most relevant instrument for our project of all of the studies/tools identified. 194 The instrument [Ethics Consultation Quality Assessment Tool (ECQAT)] is used to evaluate written records of case consultations, which then form part of the patient clinical record. It is based on a holistic assessment model covering four key elements: identifying the ethics question, eliciting consultation-specific information, ethical analysis, and making practical recommendations. Inter-rater reliability was 43% for the individual key element scores and 74% for the overall holistic assessment score.

We found no instruments that were designed to evaluate interventions to improve ethical decision-making by clinicians that assessed the quality of their ethical decision-making in actual clinical practice. The ECQAT is a potential model for developing such an instrument, but this study had not been published prior to our initial search.

Introduction

In the absence of an existing appropriate instrument to evaluate ethical decision-making in clinical practice, we developed an evaluation tool that was specific to the decision about admission to ICU and piloted it during the intervention implementation feasibility study. In a series of meetings, three members of the study team, Frances Griffiths (social scientist and general practitioner), Anne Slowther (ethicist and general practitioner) and Chris Bassford (intensive care consultant), agreed the broad framework of the evaluation tool. During these discussions we drew on a range of sources to inform our thinking:

1. an initial analysis of our ethnographic study data

2. anonymised extracts of 20 records of patients who had been admitted or referred to ICU from two of the hospitals participating in the ethnographic study

3. the findings of our systematic review of factors affecting decisions to admit to ICU

4. our concurrent development of a DSI based on our analysis of what would constitute a good decision-making process

5. existing ethical frameworks for decision-making.

It was agreed that the evaluation tool would be applied to the clinical record of a decision. This model of evaluation can be criticised because of the potential disparity between the actual process of decision-making and what is recorded in the patient notes. However, an evaluation tool that requires direct observation of the decision-making process would not be feasible to implement in a real-world situation. In addition, there is a more substantive ethical justification for this approach. If a requirement of ethical decision-making in this context is transparency, then it can be argued that having a clear record of the decision-making process – including the reasoning behind the decision – is itself an ethical criterion in any evaluation of the decision.

Key messages from our qualitative study included the importance of communication both between clinicians and with the patient and their family, clarity of process and responsibility for care of the patient, the absence of information about patients’ wishes and values, and a lack of explicit ethical reasoning in terms of weighing relevant factors in making a decision. Considering existing ethical frameworks for potential models, we selected AFR because of its focus on process, transparency, and review in the light of new evidence/information. 181 We considered that frameworks based only on ethical principles, for example ‘the four principles of biomedical ethics’, offered too narrow a model to capture the procedural elements. 155 However, the AFR framework itself does not specify the relevant reasons or factors that should be taken into account for specific decisions, and for this element of our evaluation tool we drew on our systematic review of factors, our ethnographic study data and our DSF.

Method

We first identified the dimensions of decision-making that we wanted to capture in an evaluation tool: ethics and values, transparency, consistency of process and opportunity for review. Consistency of process can be evaluated by looking at performance across decisions at any one time point and is not a substantive element of the evaluation tool itself. Within each dimension, we identified elements that should be included; for example, within the ethics and values dimension we would expect to see articulation of relevant factors (including clinical benefits and harms, and patient’s wishes), evidence to support these factors and a reasoned line of argument taking the factors into account.

We next interrogated anonymised patient records of decisions to admit (or not) a patient to ICU from two hospitals participating in the ethnographic study. Research nurses at each hospital selected 10 consecutive admissions to ICU, and one hospital, which had a referral logging system in place, also identified five consecutive cases in which a decision had been made not to admit the patient. For each record, the section relating to 24 hours before and 24 hours after a referral to ICU had been made was identified and photocopied. All patient and clinician identifiers were then redacted and the redacted copy was scanned and sent to the study team via encrypted nhs.net mail. The interrogation of the anonymised records included taking note of how our identified dimensions were recorded. This initial interrogation resulted in a draft evaluation tool with four sections:

• factors that are required to inform ethical reasoning

• evidence of ethical reasoning

• communication and review of decision

• factors that should not be present in decision-making without mitigation or explanation (e.g. age, DNACPR status).

Within these sections, further subdivisions were identified; for example:

• factors that are required to inform ethical reasoning –

  • evidence of need for intensive care

  • evidence of capacity to recover

  • evidence of what is important to the patient.

The iterative process of interrogating the notes using the current version of the tool, refining the criteria and developing/refining scoring levels continued for a further four cycles. Concurrently, a non-clinical research fellow (SR) trialled the tool with a sample of records and noted areas that required clarification for a non-clinical scorer and developed a glossary of terms and a user guide of frequently asked questions (see Report Supplementary Materials 19 and 20). The penultimate version of the tool was given to four members of the research team from different clinical specialties (GP, intensive care; JD, general practice; ZF, acute medicine; and SQ, CCOR), who were each asked to score two sets of notes and provide comments on the scoring system and the tool in general. Each set of notes was scored by two members and the resulting scores were compared and discussed, along with the scorer’s feedback, at a meeting of all scorers and the core study team. This version was also sent to the principal investigators at the six hospitals that took part in our ethnographic study to seek their comments as a further check of face validity. Further changes were made to the tool to clarify the scoring criteria, and a finalised version of the tool was drafted (see Appendix 22). The key elements of the tool are presented in Table 19.

Testing of the tool

The tool was used to score a sample of patient records in each of the hospitals in the intervention implementation feasibility study. Each hospital kept a log of all referrals to ICU during the study. At each hospital, research nurses identified 20 consecutive records from eligible referrals (see Chapter 7 for eligibility criteria) at the beginning of the implementation phase and 20 consecutive referrals in the last 2 weeks of the data collection phase. Records were redacted, scanned and e-mailed to the study team in accordance with the protocol described for the development stage. One hospital preferred not to send redacted copies out of the trust, so the researchers scored the sample on site. Four reviewers independently assessed a total of 120 records from across the three sites. Two reviewers, Anne Slowther and Chris Bassford, both clinicians, reviewed all records; and two reviewers (SR and Julia Walsh, a researcher external to the study team), neither of whom was clinically trained, assessed a subset. JW checked 40 notes each from hospitals B and C, and Sophie Rees reviewed 40 notes from hospital A only. Three reviewers (CB, AS and SR) were members of the study team. Prior to scoring the records, each scored three sets of records from the original 25 records used in the tool development using the final version of the tool and compared their scorings, discussing any disagreements to reduce any differences in approach to use of the tool. JW was an external scorer and, prior to scoring the notes from the feasibility study hospitals, had had two training sessions in the use of the tool with Chris Bassford and Sophie Rees.

Analysis

A detailed description of the scoring system is set out in Appendix 23. Each section of the tool is scored separately and then scores for sections A–C are combined, at which point section B (capacity to recover) is weighted to maintain parity with section A (evidence of clinical need). The section D score (red flags) is reported separately.

Descriptive statistics were generated for the total number of decisions and for each set of decisions provided by a reviewer. For continuous variables, the interclass correlation coefficient was calculated with its 95% CI. To allow for generalisation beyond the study reviewers, the single random raters (ICC2)199 variant was used. For categorical data, the unweighted kappa statistic and its 95% CI were calculated for each pairwise comparison between reviewers. A decision was considered eligible to be entered into the reliability analyses if all reviewers had given it a valid score. Scores from the non-clinical reviewers (JW and SR) were combined as these reviewers never scored the same decision.

All analyses were computed in R using the ‘psych’ package. 200 Typically, an outcome measure would require a reliability of at least 0.7 to be considered a reliable instrument for use in a clinical setting. 201 However, to assist interpretation of the kappa statistic, Landis and Koch202 have suggested arbitrary boundaries to classify the value. These categories have been used for both the kappa and the ICC statistics throughout the results section and are shown in Table 20 for reference.

Presence of decision and review

Reviewers were asked to note any decision about admission to ICU and then to assess that decision using the scoring sheet. However, it was not always clear from the record whether or not a decision had been made. Decisions were categorised as having been identified and assessed. It was possible to identify that a decision had been made (e.g. the patient had clearly been admitted to ICU), but as there was no clear record of that decision in the notes it was not possible to assess it. This would be categorised as identified but not assessed. From the 120 case notes scrutinised (40 from each of the three study hospitals), 143 unique decisions were identified by at least one reviewer, resulting in 429 potential scores. However, only 234 actual scores were recorded (54.5%) as some but not all decisions were either identified or assessed by all three reviewers. Table 21 shows the categorisation each reviewer gave to each of the decisions identified by each study site.

Appendix 24, Figures 38–40, gives a more detailed analysis of the combinations of reviewers who gave each decision a score at each hospital.

Agreement varied between reviewers and between hospitals. All kappa values were significantly better than agreement due to random chance (i.e. the 95% CIs did not include zero); however, many values are only ‘fair’ (see Appendix 24, Table 54). Agreement was not necessarily dependent on the clinical knowledge of the cases, as sometimes kappa values were higher between non-clinical and clinical reviewers than between clinical reviewers at a hospital. Hospital B generally had higher agreement levels than hospitals A and C. This may indicate that some hospitals had clearer record-keeping processes than others.

The location of the decisions are given in Appendix 24, Table 56. The majority of identified decisions (54.5%) resulted in the patient being admitted to ICU; however, this was driven by hospital A only. Both B and C were more balanced, with slightly more decisions resulting in the patient not being admitted to ICU (see Appendix 24, Table 57).

Scores for each section of the tool

Section A: factors necessary to inform ethical reasoning (see Appendix 24, Table 58)

The median score across each hospital for this section remained constant, but the interquartile range (IQR) varied considerably. Figure 17 shows boxplots breaking down each hospital by reviewer. Distinct differences by both hospital and reviewer can be seen. However, these patterns do not appear to be systematic.

FIGURE 17.

Section A total score by hospital and reviewer.

The overall reliability between the reviewers was ‘moderate’. However, at site level, there was a greater variability. The lowest agreement was between AS and JW at hospital B, which was not significantly different from random chance. All other ICC values observed were calculated to be better than random chance (Table 22).

TABLE 22 - Section A inter-rater reliability by hospital

Reviewed by SR.

Reviewed by JW.

Note

Values in bold are not statistically significantly different from random chance.

Hospital (N)	Comparison, ICC (95% CI); n decisions analysed)
	AS vs. CB (clinical vs. clinical)	AS vs. SR/JW (clinical vs. non-clinical)	CB vs. SR/JW (clinical vs. non-clinical)	All reviewers
All (103)	0.60 (0.42 to 0.74); 63	0.50 (0.28 to 0.66); 63	0.60 (0.37 to 0.75); 60	0.57 (0.41 to 0.70); 55
Aa (31)	0.52 (0.11 to 0.78); 19	0.63 (0.17 to 0.86); 15	0.65 (0.22 to 0.87); 14	0.61 (0.30 to 0.84); 13
Bb (34)	0.62 (0.30 to 0.82); 24	0.23 (–0.15 to 0.55); 26	0.41 (0.06 to 0.68); 27	0.42 (0.18 to 0.66); 24
Cb (38)	0.63 (0.28 to 0.83); 20	0.65 (0.15 to 0.86); 22	0.75 (0.40 to 0.90); 19	0.68 (0.41 to 0.85); 18

Looking at the inter-rater reliability of the individual items of section A, question 2 (‘interpretation of clinical situation’) had the lowest reliability. This item required interpretation of the clinical situation, so it is perhaps unsurprising that there was a marked difference between clinical and non-clinical reviewers for this question. For items 5 and 6 (patient wishes), reliability was generally higher than for the other items. This may be because specialist knowledge is not required to interpret the record with respect to patient wishes (see Appendix 24, Table 59).

Section B: ethical reasoning (see Appendix 24, Table 60)

The median scores across each hospital remained constant at zero, but the IQR varied considerably (Figure 18). At hospital A, the median score and IQR were zero, implying that the overwhelming majority of the 66 scored decisions were given a score of zero.

FIGURE 18.

Section B total score by hospital and reviewer. Owing to the large number of decisions given a score of zero, only the outlier scores are visible for reviewer CB.

As for section A total score, at least one estimate of inter-rater reliability was not significantly better than random chance (hospital B comparing AS and CB). However, the reliability of AS and CB’s scores was otherwise at least moderate (Table 23). Looking at the inter-rater reliability of each question in section B, item 9 (‘Was capacity to recover included in the balancing?’) had the highest reliability between the reviewers, and item 8 (‘Was acute physiology/system failure included in the balancing?’) had the lowest (see Appendix 24, Table 61).

TABLE 23 - Section B inter-rater reliability by hospital

Reviewed by SR.

Reviewed by JW.

Note

Values in bold are not statistically significantly different from random chance.

Hospital (N)	Comparison, ICC (95% CI); n decisions analysed
	AS vs. CB (clinical vs. clinical)	AS vs. SR/JW (clinical vs. non-clinical)	CB vs. SR/JW (clinical vs. non-clinical)	All reviewers
All hospitals (103)	0.55 (0.35 to 0.70); 63	0.60 (0.42 to 0.74); 62	0.78 (0.62 to 0.87); 60	0.66 (0.52 to 0.77); 55
Aa (31)	0.63 (0.25 to 0.84); 19	0.55 (0.05 to 0.82); 15	0.94 (0.83 to 0.98); 14	0.79 (0.56 to 0.92); 13
Bb (34)	0.29 (–0.07 to 0.60); 24	0.48 (0.13 to 0.72); 26	0.42 (0.07 to 0.68); 27	0.37 (0.13 to 0.62); 24
Cb (38)	0.73 (0.44 to 0.88); 20	0.74 (0.47 to 0.88); 22	0.89 (0.74 to 0.96); 19	0.79 (0.60 to 0.91); 18

For section C (see Appendix 24, Table 62), the distribution of scores across each hospital remained reasonably consistent, with the IQR constant at 0–3. No hospitals achieved the maximum theoretical score of 10, and the maximum score of 8 was awarded once (Figure 19).

FIGURE 19.

Section C: total score by hospital and reviewer. Owing to the large number of decisions given a score of zero, only the outlier scores are visible for reviewer JW at hospital C.

Table 24 shows that the inter-rater reliability of section C at hospital A is high between reviewers Anne Slowther and SR/JW, with reviewer Chris Bassford appearing to record very different scores. For individual items in section C, inter-rater reliability was generally moderate, except for decision communicated to family, which was fair (see Appendix 23, Table 63).

TABLE 24 - Section C inter-rater reliability by hospital

Reviewed by SR.

Reviewed by JW.

Note

Values in bold are not statistically significantly different from random chance.

Hospital (N)	Comparison, ICC (95% CI); n decisions analysed
	AS vs. CB (clinical vs. clinical)	AS vs. SR/JW (clinical vs. non-clinical)	CB vs. SR/JW (clinical vs. non-clinical)	All reviewers
All (103)	0.37 (0.13 to 0.57); 63	0.34 (0.06 to 0.56); 63	0.44 (0.21 to 0.62); 60	0.40 (0.23 to 0.56); 55
Aa (31)	0.41 (–0.06 to 0.72); 19	0.73 (0.35 to 0.90); 15	0.50 (–0.04 to 0.81); 14	0.56 (0.23 to 0.82); 13
Bb (34)	0.41 (0.0 to 0.70); 24	0.44 (0.07 to 0.70); 26	0.44 (0.01 to 0.70); 27	0.33 (0.09 to 0.59); 24
Cb (38)	0.34 (–0.05 to 0.66); 20	0.30 (–0.09 to 0.63); 22	0.32 (–0.11 to 0.66); 19	0.234 (0.07 to 0.63); 18

Section D: red flags

Only eight decisions were given any ‘red flags’ in section D. One decision was given two flags. Flags were given for the following: functional status (six decisions), previous knowledge of the patient (one decision) and presence of a DNACPR order (two decisions) (see Appendix 24, Table 64).

Total score (Figure 20 and see Appendix 24, Table 65). The maximum possible score is 36; however, the maximum score awarded to any decision was 30. Nine decisions (2.1% of all decisions) were marked as 0, and seven (1.6%) were awarded only 1 mark.

FIGURE 20.

Total score by hospital and reviewer.

The overall reliability across all decisions was moderate to substantial. However, this varied considerably between hospitals (see Appendix 24, Table 66). The relationship between total score and outcome of decision and location of decision is shown in Appendix 24 (see Tables 67 and 68, and Figures 42 and 43).

We did not analyse for differences in score between pre-intervention and post-intervention records to look for responsiveness of the instrument. Given the low uptake of the intervention in the time available for implementation as described in Chapter 7 and confirmed by the small number of records with a referral or decision-support form present in our samples we considered it unlikely that any change could be demonstrated. Further work on reliability of the tool is required before this could be attempted (see below).

Strengths and weaknesses

A major limitation of this study is the unanticipated problem of identification of referral decisions. As different reviewers extracted different numbers of decisions from a single set of notes, this made identification of each decision difficult. When possible, decisions were matched precisely, but it is likely that some decisions were incorrectly matched owing to missing data. This in turn reduced the effective sample size for the analysis, and is likely to have overestimated the errors when calculating the reliability. However, this finding is an important one for future evaluation studies using patient records.

The current experience of decision-making around intensive care unit referral and admission

Developing an intervention

At all stages of development, we sought to create a framework that was recognisable to clinicians, used non-academic language and could be embedded in established good clinical practice. However, it became clear during the stakeholder conference that education to support the implementation of a framework must include more specific engagement with the ethical concepts and principles underpinning it. This was confirmed in our implementation feasibility study when lack of time for champions to explain the underpinning ethical rationale behind the framework was identified as a challenge to successfully embedding the framework in practice.

The initial focus of the research was the decision whether or not to admit a patient to ICU. However, we observed that the ward teams had a wide range of reasons for seeking the help of the ICU doctor, and that intensive care input, when perceived to be working well, focused on the question of what was the best treatment for the critically ill patient rather than whether or not the patient should be admitted to the ICU. This prompted a paradigm shift in how the decision-making process (and, therefore, the support required) should be conceptualised, made concrete by a PPI participant in the stakeholder conference who suggested that the emphasis should be on not where the patient was treated, but how they were treated. This concept of intensive care as not constrained to the ICU has informed thinking on the development of CCOR teams,218 although the concept has been operationalised in a wide variety of ways, with inconclusive evidence of effectiveness. 219 If CCOR teams are to be used as an extension of intensive care beyond its boundaries, and CCOR nurses take on responsibility for decisions about admission to the ICU, it is worth noting the findings of our choice experiment, where differences were found between CCOR nurses and ICU consultants in the importance ascribed to factors such as NEWS when making the decision. Nevertheless, the broader message of patient-focused rather than location-focused decision-making from our empirical work led to a structuring of the DSF that first asked the question ‘What is the optimal treatment for this critically ill patient?’, and only then asked ‘Where can or should this treatment be delivered?’

A potential criticism of our intervention is that the process of making these decisions is a clinical skill that ICU doctors and referring teams exercise every day; there is no need for a framework to enable them to do this, a point made by some senior doctors in our implementation study. While it is clear that most health-care professionals make decisions and treat patients to a high professional standard, standards can be improved and patient care can benefit when checklists or prompts are used to support good practice in specific situations. There are numerous examples of checklists used in the ICU context, and their presence does not imply a lack of competence. 220,221 The recommendation for a checklist often arises from the observation that good practice can be compromised because of a combination of systemic, contextual and human factors. 222 This was identified in our ethnographic study. Reflecting the complex nature of decision-making for critically ill patients, the referral and DSFs developed in this study are less checklists but rather provide a structure for decision-making with prompts for decision-makers to consider. Our referral form was based on the SBAR tool, which is itself a checklist/framework that is widely used to support good communication between clinicians when escalating a problem or transferring care. 223

Is it feasible to implement the intervention in an acute NHS trust?

All feasibility sites were enthusiastic about implementing the intervention, which included support from the medical director. Despite the very short implementation period (8 weeks), we achieved an average of 28.2% usage of forms across sites. Following the study, two sites expressed an interest in taking the intervention or a variation of it forward as a service development. However, at all sites implementation was a challenge, and our feasibility study has provided useful data to facilitate future implementation initiatives. By using a mix of quantitative and qualitative methods, we were able to triangulate our data within the study and thus were able to understand how often, when and by whom the referral and decision-support forms were used, and also how and why clinicians used or did not use them. 224

The importance of context in successfully implementing interventions in health care is well documented. 225,226 The shared context of all of the sites was that of a busy NHS hospital, but the specific context of each site influenced implementation. The size of the organisation and the number of staff who had to be reached clearly had an impact, but so too did existing interprofessional relationships and structures (e.g. presence or absence of CCOR or existing systems for logging ICU referrals). Key difficulties common to all sites included having time for implementation, both longitudinally (for embedding the study into practice) and in individual champions’ time to devote to promoting the intervention. Linked to this is the challenge of institutional inertia. NHS trusts are large, complex organisations with well-established processes and protocols, and they cannot respond easily to rapid change (our implementation period was only 8 weeks). This represented a challenge for the champions, who had to break into existing training and information-giving opportunities within the trust to educate clinicians about the intervention. It was also a reason why it was impossible to integrate an electronic version of the forms into the trust electronic records system, something that all champions agreed would be essential for truly embedding the intervention into routine practice.

At all sites, to a greater or lesser extent, there was a mismatch between how we had envisaged the intervention working and how it actually worked in practice, particularly in the way the referral forms were used and the lack of use of patient and family information leaflets. Both of these documents were intended to be used when considering a decision to refer a patient to ICU; however, in most situations, use of the referral form was only prompted at the time a referral was being made, leading to frustration on the part of the referring team, who perceived the form as an additional burden rather than as the intended support. This emphasises the importance of understanding the context in which the intervention will be used and of being flexible in how it is implemented.

The nature of flexibility in implementation strategies has been highlighted by May et al.,225 who describe the importance of plasticity of intervention components and the elasticity of the normative and relational structure of the context in which the intervention is being implemented. Our intervention had some flexibility in the educational component, but the forms were less flexible. An electronic version would allow trusts to individualise the forms and process, which would increase flexibility and may improve uptake. However, the core structure and content of the referral and decision-support forms cannot become so flexible that key elements of the ethical decision-making process are absent. Negotiating the degree of flexibility necessary to maximise uptake while maintaining the integrity of the intervention will be key to successful implementation.

Interventions that involve normative changes in practice, such as introduction of shared decision-making or DSFs, can be experienced as a threat by the practitioners involved. 226 A few of the clinicians in the feasibility study expressed concerns that their clinical and ethical practice was being questioned by the requirement to complete a form. However, the main barriers to implementation were seen as more practical and related to concern about duplication of work, lack of time, and other organisational and situational pressures on decision-makers. These concrete barriers have been noted in other implementation studies of similar interventions, such as surgical checklists227 and advance care planning. 228

The challenge of involving patients and families in the decision-making process

Involving patients in decision-making about their medical care is both an ethical and a legal obligation for health professionals. Using a shared decision-making model is considered to be best practice, even in the ICU, where issues with a patient’s mental capacity pose challenges. 30 However, a striking finding throughout the project was the low level of involvement of patients and their families in the decision-making process. Clinician interviewees in the ethnographic study suggested that the patient’s views were highly valued when making a decision, and ICU consultants in the choice experiment considered the views of the family important. However, our observational data showed that, in practice, the views of patients or their families were seldom sought, and in the intervention implementation feasibility study the patient and family information leaflets, which were designed to support and promote their involvement in the decision-making process, were not used at any of the sites.

There are a number of reasons why active involvement of patients and their families may not occur when decisions are being made about a critically ill patient. The decision itself may need to be made quickly and often in an emergency situation when the patient is too sick to engage in discussion and their family is not available, and so it may not be possible to involve them. Clinicians may feel reluctant to engage in a discussion that involves the very real possibility that the patient will die, the uncertainty of outcomes, and the potential distress and harm that ICU interventions might cause. ICU doctors in the implementation study found it difficult to articulate the burdens of ICU on the decision-support form. Articulating these in a conversation with a critically ill patient or a distressed family member is likely to be even more difficult, and potentially distressing for the clinician. Feedback from site debrief meetings reinforced this concern during discussions of referring teams’ apparent reluctance to use the leaflets. This argument against the involvement of patients has previously been seen as a barrier to discussions about cardiopulmonary resuscitation229,230 or decisions to withdraw life-supporting treatment in ICU. 231,232

In other jurisdictions, consultation with family members or surrogates is legally mandated for important decisions about a patient who lacks capacity. Although surrogates often find this difficult, there is evidence that they prefer to take an active role in decision-making about end-of-life care in the ICU233,234 and experience little regret about the decisions they have made. 234,235 Patients wish to be involved in such discussions and do not experience undue distress from these. 236 In the context of DNACPR, UK law requires consultation with the patient or their family when this decision is being made. 237

Difficult conversations require good communication skills and there is a recognised need for training in this area among ICU clinicians. 238 Furthermore, negative attitudes and dysfunctional interdisciplinary relationships can be a barrier to patient and family involvement in decision-making. 239 Clarifying the responsibilities and processes involved in making these decisions may help. In end-of-life decisions, families can be confused about who is making the decision, which can have an impact on their ability to participate in the process. 240 When processes are clear, the family experience is better. 241

Challenges in evaluating ethical decision-making

A striking finding from our work in testing the evaluation tool was the difficulty reviewers had in identifying when a decision whether or not to admit a patient to ICU had been made from scrutinising the clinical record. This was true of the clinical as well as the non-clinical reviewers. There are several possible reasons for this finding. Lack of a detailed account of a decision does not in itself mean that a poor decision has been made. In urgent situations clinicians may feel that it is more important to make a decision and act on it than spend time recording in detail the reasons for the decision. Lack of clear documentation of this specific decision did not necessarily mean that general documentation in the patient record was poor. One of our sites had very clear, structured ICU records for patients who were admitted, which documented some of the information that would have informed the decision. However, this record was only for patients admitted and was not a retrospective documentation of the admission decision but the plan for ICU care.

The tool allowed only information and reasoning recorded by the person responsible for the decision to be scored, although explicit reference to previously documented evidence, such as physiological status, could be included. In some of our sample records, detailed notes by the critical care outreach team prior to ICU review of the patient included much of the documentation referred to in the score sheet. It is likely that an ICU doctor would have noted this when making their decision and therefore would not have recorded it again. However, it was not always clear how the doctor had used this information. The discrepancy between actual decision-making and the record of that decision is a weakness of any evaluation tool that is applied only to patient records. However, as we have stipulated a transparency requirement for ethical decision-making, and considering the potentially life-changing consequences of these decisions, it is reasonable to expect an adequate record of how the decision was made, and that this should be part of a standard of good decision-making. Our requirement for documentation of the decision-making process is in line with professional guidance for clinicians. 180

Although the overall total score reliability of the evaluation tool was moderate to substantial, there was considerable variability across sites, with reliability at some sites only fair. Variability was not linked to the clinical background of the reviewers, suggesting that clinical knowledge is not the main factor. A possible reason for the variability in scoring is the degree to which the scorers are required to interpret what is in the written record. It is not straightforward to assess what counts as evidence of capacity to recover or what demonstrates balancing of burdens and benefits and whether or not specific evidence has been used in that balancing. It may be that in our attempt to increase the granularity of our scoring system to increase its sensitivity, we reduced its reliability. Reliability could be potentially improved with better training of scorers, but this might limit its practical use.

The challenge of evaluating written records of complex decision-making processes using multiple criteria was also encountered by Pearlman et al. 194 when developing the ECQAT for assessing clinical ethics consultation records. Their approach was to adopt a holistic rather than an analytic scoring system. In this system, raters consider key elements and other factors that work together and give a score based on the total impression of the account rather than scoring individual items. Advantages include reduced likelihood of scorers struggling with interpretation of very specific criteria, and less time required for scoring. Possible disadvantages if assessors are clinicians is that they will overinterpret; that is, they will assume that the clinician has considered evidence or made a decision based on specific reasoning without clear evidence in the documentation. Critiques of holistic scoring models also recognise this concern, suggesting that they may mask deviances in how the descriptors for the holistic scores are applied. 242 However, it may be easier to give examples of what is considered a satisfactory or unsatisfactory decision-making process using this approach. Future development of the evaluation tool should consider both refining the individual criteria specifications in the current version and/or applying a holistic assessment approach to each section.

Although in its current version the tool cannot be recommended for summative evaluation of decision-making about admission to ICU, our experience in developing the tool suggests that it would be useful as a formative or an educational tool. Applying the tool to a sample of clinical records of patients who have recently been referred to a specific ICU during a quality improvement meeting would enable discussion of what counts as quality in decision-making and documentation, what counts as evidence, what reasons are relevant, and how reasoning should be articulated. Key points and lessons learned could feed back into future decision-making and recording, with the aim of improving overall quality of decision-making.

Strengths, weaknesses and methodological challenges

We have noted the strengths and weaknesses of the individual studies within this project in the relevant chapters. An overall strength of the project is its mixed-methods approach, which allowed the triangulation of findings between studies and enabled the development of an intervention firmly grounded in relevant empirical research evidence. We have been able to capture the process of decision-making around referral and admission to intensive care at nine NHS trusts in the UK (six in the ethnographic study and three in the implementation study) and also to capture decision preference patterns across a much wider cohort of intensive care consultants and CCOR nurses in our choice experiment. PPI and stakeholder involvement was strong throughout the project. The difficulty we had in involving patients and families in the project, specifically in the ethnographic study, was disappointing. We had anticipated that recruiting patients and families close to the decision-making event would be challenging and had worked with our PPI colleagues to develop processes to facilitate supportive approaches. Despite this, recruitment was difficult. We had not anticipated the difficulty in tracking patients following their discharge from the ICU to seek their consent for late-stage interviews, which further reduced potential recruitment. Thus, the patient and family perspective is not as strong as we had originally envisaged.

We also experienced challenges when conducting the two primary systematic reviews. The research questions required a broad search strategy, resulting in 34,000 abstracts and 83 included studies. The diverse methodologies of the studies and the heterogeneity of the results made it difficult to pool data and draw meaningful conclusions. In updating the reviews for this report, we adopted a narrative snowballing approach, using our existing knowledge of the literature and focusing on key journals that had the greatest number of relevant papers in our original review. The yield from this approach appears to be as productive as the formal systematic approach, although we cannot be certain that we have not missed a relevant paper. However, this approach was significantly more cost-effective, requiring only a maximum of three reviewers compared with a team of 16.

Summary of findings

This project has illuminated the complexity of decision-making around referral and admission to intensive care. Referring teams and ICU doctors making decisions about critically ill patients need to consider multiple sources of information with varying levels of uncertainty about prognosis or likely outcomes, often with little knowledge of the patient’s wishes or values. External constraints increase the pressure on the decision-making process. Our findings suggest that the focus should be, and in current practice often is, on the best treatment for the critically ill person, and how this can be delivered, rather than on whether or not they should be transferred to ICU. Good decision-making requires communication, both between colleagues and with patients and their families, strong interprofessional relationships built on mutual respect and trust, and careful consideration of the benefits and harms of intensive care for the individual patient. Although there are many examples of this process working well for patients, there is evidence that doctors find it difficult to articulate and explicitly balance the benefits and harms of treatment for patients, that implicit values can inform the decision, and that communication as part of the process of decision-making is often limited. Documentation of the decision to admit or not to intensive care is generally poor, making evaluation difficult and raising concerns about the transparency of the process. Support for junior doctors in making these difficult decisions is often patchy. The patient voice is often lacking in the decision-making process.

Implementing an intervention to support the decision-making process about treatment for a critically ill patient was challenging. Uptake and completion of the forms was low, at just under 30%. However, doctors who used the forms were generally positive about their use, observing that the forms helped them to articulate and communicate their thinking and prompted them to involve patients and their families. Concerns were raised about duplication of work and the time taken to complete the documentation distracting from patient care. Where the framework could be incorporated into existing systems, it appeared to work well. The implementation period in the feasibility study was, in retrospect, too short. Further work is needed, building on the lessons learned from the feasibility study.

Implications for practice

The concept of a structured intervention to support clinicians who are making decisions around referral or admission to ICU was generally welcomed by key stakeholders. NHS trusts wishing to implement such an intervention may need to consider how it will be incorporated into current working practices throughout the hospital; interventions aimed at improving decision-making are more likely to be effective if they engage with the whole process from the point at which the need for critical care support is first considered. Ensuring system-wide change needs senior institutional support and training for all clinical teams involved in the process. Incorporation into an electronic patient record and referral system will facilitate implementation. A quality improvement approach may be helpful when planning future intervention implementation work.

We identified a clear need for improved education of clinicians involved in decisions to refer or admit a patient to ICU, particularly around ethical reasoning and communication. Our project findings and resources may be helpful to professional organisations concerned with curriculum development and postgraduate training. National and international guidelines recommend consultant-to-consultant communication in these decisions, but our study suggests that this does not always occur, and junior doctors, particularly in referring teams, may feel unsupported. Given resource constraints and changes to team structures and working patterns within the NHS, trusts may wish to consider how best to support consultant-led decision-making and collaborative team-working, particularly out of hours.

Involving patients and their families in these difficult decisions appears to be extremely challenging for doctors, but the evidence from our study and others is that patients and families wish to be part of decision-making about their care, even in distressing and life-threatening situations. Further work needs to be undertaken to explore how information can be provided to patients and families to support their involvement. Examples from other areas of practice, such as the introduction of the national Recommended Summary Plan for Emergency Care and Treatment (ReSPECT), may be helpful in this regard.

Recommendations for future research

Our study has gone some way to suggesting how decision-making about referral and admission to intensive care can be improved. It has also identified areas in which further research is needed to inform future interventions and support evidence-based patient-centred decision-making. Priorities for research in this area suggested at our dissemination conference included:

1. Investigation into how to improve meaningful engagement of patients and families in the decision-making process. Difficulties in including patients and families in the decision-making process were evident in all parts of our research. Research to better understand the barriers to this involvement and to develop and test strategies to improve this aspect of the decision-making process is needed.

2. Development of patient-related outcome measures and investigation of the effect of interventions on patient-relevant outcomes. Short-term survival after a critical illness is a limited metric for judging the quality of care delivered by an ICU. Research to better understand the nature and range of specific outcomes that are important to critically ill patients is required to develop quantifiable metrics that can be used to improve patient care. This will allow more informed conversations with patients and families and inform evaluation and improvement of decision-making.

3. Development and evaluation of educational interventions to support ICU trainees with regard to decision-making around escalation of treatment decisions including admission to ICU.

4. Further development and implementation evaluation of DSI using a quality improvement approach.

5. Further work on validating the evaluation tool for ethical decision-making and to describe its role in service improvement and clinical research.

6. Evaluation of the importance of external influences on medical decision-making for the treatment of critically ill patients. We have shown that factors other than direct, patient-related, clinical information influences whether or not a patient is admitted to ICU. How these other, ‘external’, factors have an impact on the decision-making, and extent to which they have an impact, needs further investigation.

Achieving the best outcomes for an individual patient depends on everyone involved making the best possible decisions. Decisions about treatment for critically ill patients have life-changing consequences for the patients involved, and doctors making them will always be faced with difficult clinical and ethical challenges. This study has provided evidence and resources that will help clinicians and organisations aiming to improve decision-making for, and ultimately the care of, critically ill patients.

Search strategy

Search strategy

Search strategy

TOPIC: (“intensive care” or “critical care” or “intensive care unit*” or “critical illness*” or “critically ill*” or “critical care unit*” or icu or itu)

AND

TOPIC: (triag* or admission* or admit* or refus* or deny or delay or refer or referred or referral or limit or limited or limiting or transfer or transferred or “refusal to treat” or “referral and consultation” or “resource allocation”)

AND

TOPIC: (“professional practice” or “decision making” or judgement* or judgment* or “attitude of health personnel” or “medical futility” or choice* or decision* or “prognostic pessimism” or attitude* or experience* or “professional family relation*” or “professional-family relation*” or (futil* NEAR/5 (treatment* or care or medical)))

NOT

TOPIC: (neonat* or pediatric* or paediatric* or picu or nicu)

Search strategy

((SU.EXACT.EXPLODE(“Intensive Care”) OR (“intensive care” OR “intensive care unit*” OR “critical care” OR “critical ill*” OR “critically ill*” OR “critical care unit*” OR itu OR icu))

AND

(SU.EXACT.EXPLODE(“Hospital Admission”) OR SU.EXACT.EXPLODE(“Resource Allocation”) OR (triag* OR admission* OR admit* OR refus* OR deny OR delay* OR refer* OR limit* OR transfer*))

AND

(SU.EXACT.EXPLODE(“Decision Making”) OR SU.EXACT.EXPLODE(“Judgment”) OR SU.EXACT.EXPLODE(“Health Personnel Attitudes”) OR SU.EXACT.EXPLODE(“Choice Behavior”) OR (“professional practice” OR “medical futility” OR futil* OR judgement* OR judgment* OR decision* OR choice* OR “prognostic pessimism” OR attitude* OR experience* OR “professional family relation*” OR “professional-family relation*”))) NOT

(neonat* OR pediatric* OR paediatric* OR picu OR nicu)

Search strategy

(((SU.EXACT.EXPLODE(“Intensive care units”) OR SU.EXACT.EXPLODE(“Intensive care”)) OR (“intensive care” OR “intensive care unit*” OR “critical care” OR “critical care unit*” OR “critical ill*” OR “critically ill*” OR itu OR icu))

AND

(SU.EXACT.EXPLODE(“Triage”) OR SU.EXACT.EXPLODE(“Admissions” OR “Emergency admission” OR “Readmission”) OR SU.EXACT.EXPLODE(“Refusal”) OR (SU.EXACT.EXPLODE(“Direct referrals”) OR SU.EXACT.EXPLODE(“Arrest referral schemes” OR “Direct referrals” OR “Extra contractual referrals” OR “Referrals” OR “Selfreferrals” OR “Weekend referrals”)) OR SU.EXACT.EXPLODE(“Resource allocation”) OR (triag* OR admission* OR admit* OR refus* OR deny OR delay OR refer* OR limit* OR transfer* OR “refusal to treat”))

AND

(SU.EXACT.EXPLODE(“Professional practices”) OR (SU.EXACT.EXPLODE(“Anticipatory decision” OR “Clinical decision making” OR “Collaborative decision making” OR “Decision making” OR “Dynamic decision making” OR “Groupthink” OR “Moral decision making”) OR SU.EXACT.EXPLODE(“Decisions” OR “End of life decisions”) OR SU.EXACT.EXPLODE(“Clinical decision making”)) OR SU.EXACT.EXPLODE(“Adjudicators” OR “Affective judgments” OR “Causal judgments” OR “Censure” OR “Clinical judgments” OR “Compliments” OR “Confidence judgments” OR “Frequency judgments” OR “Judgments” OR “Moral judgments” OR “Praise” OR “Probability judgments” OR “Professional judgments” OR “Reflective judgments” OR “Social censure” OR “Social judgments” OR “Subjective judgments” OR “Value judgments” OR “Visual judgments”) OR SU.EXACT.EXPLODE(“Career choice” OR “Choice” OR “Informed choice” OR “Life choice” OR “Object choice” OR “Parental choice” OR “Social choice”) OR SU.EXACT.EXPLODE(“Family-Health professional relationships”) OR (“professional practice*” OR judgement* OR judgment* OR decision* OR choice* OR “prognostic pessimism” OR attitude* OR experience* OR futil* OR “medical futility”)))

NOT

(neonat* OR pediatric* OR paediatric* OR picu OR nicu)

Search strategy

Search strategy

all(“intensive care” OR “intensive care unit*” OR ICU OR ITU OR “critical care” OR “critical illness*” OR “critically ill*”)

AND

all(triage* OR admission* OR admit* OR refus* OR deny OR delay OR refer* OR limit* OR transfer*)

AND

all(judgement* OR JUDGMENT* OR decision* OR choice* OR “prognostic pessimism” OR attitude* OR experience* OR futil* OR “professional practice*” OR “professional family relation*”)

Search strategy

(“intensive care” OR “intensive care unit*” OR ICU OR ITU OR “critical care” OR “critical illness*” OR “critically ill*”) AND (triage* OR admission* OR admit* OR refus* OR deny OR delay OR refer* OR limit* OR transfer*) AND (judgement* OR JUDGMENT* OR decision* OR choice* OR “prognostic pessimism” OR attitude* OR experience* OR futil* OR “professional practice*” OR “professional family relation*”)