Factors That Influence Variation in Clinical Decision-Making about Thrombolysis in the Treatment of Acute Ischaemic Stroke: Results of a Discrete Choice Experiment

Patient-related factors that can influence clinician decision-making

An international Delphi study conducted in 2007 sought to establish consensus on the relative contraindications for offering intravenous thrombolysis in acute ischaemic stroke. 20 Although agreement was reached on 12 of the 18 patient factors, there was a failure to reach consensus on factors such as stroke onset time to treatment, recent medical procedures, spontaneous improvement rate and blood pressure treatment (Table 1). 20 This study highlighted the variation between expert clinicians in their attitudes towards factors that influence decision-making about thrombolysis and underlined the degree to which ‘grey’ areas remain in the evidence base.

Studies have suggested that sex may have a population-level effect on thrombolysis administration, with evidence of the underutilisation of intravenous thrombolysis in women,40 although this may reflect a different age and presentation profile in women. 41 This is despite a pooled analysis of randomised controlled trials demonstrating that women are more likely than men to benefit from thrombolysis. 42,43 Research to assess and clarify the impact of sex on thrombolysis utilisation and outcomes using individual patient data collated from randomised trials is ongoing and should clarify this effect, if any. 15

Uncertainty exists regarding a number of patient factors, where there are ‘grey’ areas in the guidelines and/or research evidence. Patient factors, such as pre-stroke disability, stroke severity and comorbidities, can operate as a major influence on clinicians’ consideration of the potential risks and benefits of thrombolytic treatment. Research has explored whether or not pre-stroke disability was associated with negative outcomes of thrombolysis. 44 It was concluded that pre-stroke disability does not independently increase the risk of sICH after thrombolysis. However, higher modified Rankin Scale (mRS) scores (high scores indicating greater dependency on others for assistance with daily living) were associated with an increased risk of death at 3 months post treatment. Despite this, the authors state that as one-third of previously disabled patients can return to their pre-stroke dependency, these patients should not be excluded from treatment with intravenous thrombolysis.

Uncertainty also exists regarding minor strokes and self-limiting transient ischaemic attacks. Relatively minor symptoms may lead clinicians to believe that patients are not sufficiently impaired to warrant treatment, based on the perceived risk of treatment. Furthermore, the licensing criteria state that clinicians should exclude patients as eligible for treatment when there is a ‘minor neurological deficit or symptoms rapidly improving before start of infusion’45 and, thus, clinicians may withhold or delay thrombolysis where symptoms are mild or improving. 46 Many clinical trials excluded patients with low NIHSS scores (usually NIHSS score of ≤ 3–5), limiting the evidence base for such patients. However, a recent Austrian study found that those with a mild deficit (NIHSS score of ≤ 5) had significantly improved outcome with thrombolytic therapy, compared with matched cases who did not receive thrombolytic treatment. 47 A pooled analysis of individual patient data from stroke trials demonstrated better outcomes in patients with mild ischaemic stroke (NIHSS score of ≤ 4) treated with thrombolysis, although < 10% of enrolled patients had mild stroke. 15

Clinicians may also delay treatment or decision-making with thrombolysis if blood pressure is high (> 185/110 mmHg). Active blood pressure lowering is associated with an increased proportion of patients being treated with intravenous thrombolysis,48,49 compared with a conservative strategy, without a difference in sICH rate.

Clinician-related factors that can influence clinical decision-making

There is strong evidence that clinician-related factors can have a significant influence on medical decision-making. 50 Shamy et al. recently conducted a study which addressed the decision of administering intravenous thrombolysis for acute stroke. 51 In their online survey of 70 neurologists in Ontario, Canada, they found that respondents were less likely to administer thrombolysis to patients over the age of 80 years, to patients from nursing homes and to patients diagnosed with dementia. 51 Post-hoc analyses revealed that neurologists who consulted on more than 10 acute stroke cases per month were significantly less likely to treat large strokes and those with more than 10 years of clinical experience were significantly more likely to administer thrombolysis to patients with dementia.

Further clinician-centred influences on the consideration of thrombolysis have been postulated, and include factors such as concern about adverse effects, especially sICH, and uncertainty about effectiveness. 37,52 Recent academic debates regarding the efficacy of thrombolysis are also likely to impact on clinicians’ decision-making and might have influenced clinicians’ perception of the evidence base and confidence in the efficacy of thrombolysis. Attitude towards thrombolysis has been independently associated with willingness to administer thrombolysis,37 and clinicians’ interaction and involvement with their professional community and information-seeking behaviour can also influence medical decision-making. 53

In the wider literature on clinical decision-making, several factors, including clinicians’ age, sex, medical specialty and level of experience, have been shown to influence decision-making. For instance, level of experience54 and characteristics of clinicians (e.g. their medical specialty, age and ethnicity) can interactively influence decision-making. 55 A study examining variation in diagnostic decision-making in heart failure (a similarly complex decision characterised by high levels of uncertainty) found that clinician characteristics, such as grade or level of seniority, influence how evidence is weighted and used to inform a diagnostic decision. 56

Personality and individual values also differ among clinicians, potentially influencing decision-making. Research indicates that personality characteristics are associated with working in certain medical specialties57 and can predict the choice of person- or technique-oriented medical specialty. 58 Individuals can also differ in their approach to risk,59 and risk assessment is a routine component of clinicians’ everyday practice. The perception of risk can be influenced in many ways, such as through interaction with the patient and other clinicians and through the clinicians’ own attitudes towards risk. Personal attitude to risk has been explored in previous decision-making research, and a US-based study found that emergency medicine physicians’ scale scores correlated significantly with their rates of admission for patients with acute chest pain, in that individuals in the high-risk-taking group were less likely to admit patients after treatment than colleagues scoring as low risk takers. 59

Another relevant feature of clinical decision-making in practice is making decisions in uncertain circumstances, for instance when a diagnosis is unclear or when details about the patient’s history may be incomplete or unknown. A systematic review of the barriers to delivery of thrombolysis found that physician uncertainty about administering thrombolysis was a key barrier. 52 Gerrity et al. developed a scale to assess physicians’ affective reaction to uncertainty. 60,61 Research employing this measure found that it has been associated with decision-making. For instance, primary care providers with higher levels of anxiety about uncertainty were more likely to refer a chronic kidney disease patient to a nephrologist. 62

Clinicians are also vulnerable to the cognitive biases that affect us all. These are cognitive ‘rules of thumb’ that are based on experience and serve to reduce cognitive load. 54 While they are a mechanism by which rapid clinical decisions can be made in typical circumstances, they can also result in error and decision-making bias if adhered to rigidly in situations which are atypical. The availability bias is one which may influence the decision to offer intravenous thrombolysis. Availability may be described as the tendency for things to be judged as occurring more frequently, or as more likely to occur, if they can be readily brought to mind: that is, they are available and easily accessible in memory. This can lead to distortion in estimates of base rates (i.e. risk of sICH) and may lead clinicians to under- or overestimate the probability of a negative outcome of treatment. Even clinicians who are cognisant of the potential bearing of cognitive biases on clinical judgement and decision-making may still be susceptible. 63

Clinicians’ may also vary in their willingness to control blood pressure before administering thrombolysis, as this was not standardised during the trials which produced evidence about the efficacy of thrombolysis for stroke. A recent study48 found that active blood pressure lowering was associated with an increased proportion of patients treated with intravenous thrombolysis, compared with a conservative strategy, without a difference in sICH rate. However, it is unclear if all clinicians would be comfortable with actively trying to lower blood pressure before administering thrombolysis.

Taken together, this literature highlights the variation that can exist between clinicians, even when consulting on identical cases, and thus underscores the importance of examining both patient factors and clinician factors in a decision-making model.

Environmental/systemic factors that can influence clinical decision-making

Research suggests that the culture within a hospital or treatment centre can impact on decision-making and practice style. 64–66 A study conducted in the Netherlands found that several features of organisational culture were positively associated with rates of thrombolysis administration, including the availability of feedback on practice, a learning culture and clinical leadership. 67 Structural factors such as access to CT scanning might previously have had an influence on variation on intravenous thrombolysis rates in the UK; however, with the widespread implementation of ‘24/7’ hyperacute stroke services and the extension of telemedicine facilities,68 continuing variation is likely to reflect differences in clinical decision-making.

Stage 1: exploratory work

In the initial stages, we considered all factors that might be important for a clinician in decision-making, as omitting potentially important factors in the DCE may bias findings. 7 We drew on qualitative data collected in a previous research programme examining decision-making in acute stroke14 and on work to inform the development and testing of a computerised decision support tool for thrombolysis in acute stroke care. 79 This included qualitative interview data on the views and experiences of stroke clinicians about thrombolysis decision-making, ethnographic data collected through non-participant observation of thrombolysis decision-making discussions and data from the usability and feasibility testing of the decision support tool.

Relevant literature was also reviewed to help to identify further factors that have been documented to affect decision-making about thrombolysis and factors influencing clinical decision-making more broadly. 20,51,52,80–82

In addition, nine telephone-based semistructured interviews were conducted with stroke clinicians and leading experts in stroke research. The sampling strategy was purposive in order to capture the views of clinicians who had not contributed to the development and testing of a stroke thrombolysis decision support tool (Computerised Decision Aid for Stroke Thrombolysis; COMPASS), and targeted clinicians from hospitals in both the upper and lower quartiles of thrombolysis rates, based on figures from national audit reports. 19 These interviews aimed to identify any additional patient-related factors or levels that represented the borderline and/or difficult cases, in particular the nature of trade-offs made between influential factors and levels when making decisions about whether or not to offer thrombolysis.

Interviews were audio-recorded and transcribed verbatim and anonymised transcripts were then imported into QSR International’s NVivo 9 (Warrington, UK) to facilitate the coding and analysis of data. The resulting data set was coded thematically and analysed using a framework approach. 83 This qualitative analysis of interview data and review of secondary data sources from the Development and Assessment of Services for Hyperacute stroke (DASH) Programme Grant for Applied Research,14 alongside the literature review, facilitated the generation of a comprehensive list of 22 potentially influential patient-related factors on clinical decision-making regarding thrombolysis (Box 1).

Stage 2: expert panel discussions – inclusion and exclusion criteria

Exploratory work commonly identifies too many factors/levels to include in DCEs and, owing to sample size constraints and the potential for respondent cognitive burden, it is often not practical or feasible to include all possible combinations that may influence decision-making. By including all potentially influential factors and levels of interest, the statistical power of a DCE to detect effects would be reduced, owing to the large number of possible combinations of levels and the inability of a small sample size to adequately assess all these various combinations. Methods are needed to select the most important factors and levels to include in a testable DCE. Although there are publications that provide generic advice,76,84 there is little agreement on the best approach to achieve this. A well-designed DCE has been described as ‘one that has sufficiently rich set of attributes and choice contexts, together with enough variation in the factor levels necessary to produce meaningful behavioural responses’. 27 Abiiro et al. 85 advise employing quantitative methods, such as ranking exercises, to support the process of selecting factors for inclusion to a manageable number.

In the first instance, the expert multidisciplinary panel in the current study (consisting of three stroke physicians who undertake clinical research, a trainee stroke physician, two patient representatives, two health psychologists, two health economists and an expert in shared decision-making) screened the list of factors in terms of whether they would be feasible or meaningful to include in a DCE, to be further scrutinised using a structured prioritisation exercise (SPE). Following discussions, a set of inclusion and exclusion criteria were agreed with reference to the study objectives (i.e. the study focus on decision-making in acute ischaemic stroke and not on diagnosis of acute stroke).

Stage 3: phase 1 – design of online structured prioritisation exercise

An online SPE was designed to ascertain the perceived relative importance of various patient-related factors for clinical decision-making about thrombolysis, as well as to help to identify the ‘grey’ areas and areas of uncertainty on specific factors. Questions were phrased to identify where uncertainty and ‘cut-offs’ existed on individual factors regarding the suitability of the patient for intravenous thrombolysis: for example,

Assuming all other indications point to thrombolysis, at what blood pressure levels [systolic and diastolic] would you no longer think it appropriate to offer intravenous thrombolysis?

Free-text boxes were provided after each question to capture any comments respondents had to provide more information about their response. The SPE was hosted on Qualtrics (www.qualtrics.com) and 30 stroke clinicians in the north-east of England were invited to participate. The rationale for limiting the sample to clinicians in this region was to avoid priming too many in the population who would be invited to participate in the main study. The SPE is included in this report as Appendix 2.

Stage 3: phase 2 – using the structured prioritisation exercise to inform selection of factors/levels

Based on the results from the exploratory work (Table 3), the SPE and expert panel discussions, the expert panel (n = 6) ranked each factor in order of priority for inclusion in the DCE (based on those which were perceived to have the greatest influence on thrombolysis decision-making), indicated how they would operationalise them and provided suggestions on possible levels for each patient-related factor (Table 4 provides the results). Results were then aggregated and fed back to the panel to inform the discussion on the selection of the final list of factors and levels for inclusion.

There was a need to compromise on the perceived importance of factors to ensure that concerns regarding sample size/design feasibility, ecological validity (i.e. ensuring that information presented is as consistent as possible with information that would be available in a real-life situation) and the potential impact of unconscious factors (e.g. patient ethnicity) were considered in the final factor selection process. Among the issues considered during these panel discussions were:

• Ensuring clinical face validity. It was vital that patient information presented in vignettes was realistic and that factors and levels could plausibly occur together. Certain factor levels could not reasonably appear together meaningfully; therefore, some initial constraints were imposed on the design. For instance, a mRS score of 0 or 1 (indicating that the patient is able to carry out all usual duties and activities) was considered implausible for a patient described as having severe dementia. Vignette-style presentation was considered optimal, as clinicians tend to be familiar (and probably more comfortable) with vignettes because these are regularly used in training and CPD. Consequently, participating clinicians were unlikely to treat the exercise as a test and, thus, to behave differently from how they would in the real world. Similarly, vignettes (as opposed to a traditional tabular list of attributes and levels that are commonly used in DCEs) more closely reflect the way in which clinicians receive information about their patients, thereby increasing the realism of the decision task.

• Ensuring sufficient information was present in vignettes. It was crucial to ensure that clinicians had sufficient information available to reach a decision about the offer of thrombolysis. It was decided that information on certain factors must be provided (as either variable or fixed factors) in the vignettes in order for clinicians to reach a decision and for the decision-making process to mimic real-life decisions as closely as possible. For instance, stroke severity score and the results of the CT scan were considered vital information. Factor order/placement in the vignette was decided based on the information that would be available to the clinician immediately and the typical order in which additional information may be obtained. The order of presentation of factors was also informed by work to develop the COMPASS decision support tool. 79

  At this point, we also considered the relative frequency of various presentations of acute stroke. It was decided that relatively uncommon complications or patient characteristics (e.g. patient on warfarin or had recent surgery) would be excluded to ensure that vignettes included those factors that were more common to most patients (e.g. NIHSS score indicating stroke severity, patient age and patient sex). Data from Safe Implementation of Thrombolysis in Stroke (SITS) helped to inform this discussion. 91

• Anticipated sample size and resulting design considerations. Although more levels can define the true relationship between factors and their influence on decision-making, there is an inherent trade-off between the number of factors and levels that can be included and the complexity of the design of the DCE. 27 Furthermore, there are important considerations regarding the maximum numbers of factors and levels a participant can process at one time, as well as issues regarding sample size. Owing to the limited size of the population of interest in the current study, restricting the number of variable factors and levels per factor was necessary to control the number of potential combinations in order to ensure that the DCE would have sufficient statistical power to detect important effects. The complexities and problems of performing sample size calculations for DCEs have been highlighted. 7 Optimal sample size requirements for the limited dependent variable models of the nature estimated in DCEs depend on knowledge of the true choice probabilities, which were not known prior to undertaking the research. 73

  However, previous DCE studies have shown that robust choice models can be estimated from sample sizes of between 50 and 100 respondents. 92,93 Based on previous studies and estimates of the total number of clinicians treating acute stroke patients in the UK, a sample size of 150–200 was anticipated as achievable in the current study. This estimate represents approximately 35–47% of the 422 clinicians registered as ‘full members’ of the British Association of Stroke Physicians (BASP), although the proportion involved in thrombolysis decision-making was unknown. 94 Moreover, owing to the potential for cognitive burden, we judged that 12–16 hypothetical patient vignettes would be the upper limit of the number of decision vignettes that should be administered to each participant (and this would be tested in the piloting of the DCE in stage 4).

  Factors that were deemed important to provide basic clinical information (but not included as variables in the DCE) were included as fixed factors in the design, meaning that they were described to respondents and held fixed in the choice context. The use of fixed factors helped to remove subjectivity around interpretation of additional issues, providing a common interpretation of such factors across respondents while retaining face validity. In the current study, blood glucose level was included as a fixed factor in each vignette, at a level of 6 mmol/l. An identical text description of a patient’s CT scan result was included in all vignettes to remove subjectivity around imaging interpretation and to minimise any uncertainty about diagnoses of acute ischaemic stroke.

  Definitions were included to standardise the interpretation of factors and levels. For instance, the standard definitions for mRS scores95 used to assess pre-stroke dependency were presented in each vignette. Definitions were also provided to describe pre-stroke cognitive functioning and were operationalised based on the Global Deterioration Scale for the assessment of primary degenerative dementia. 96 The calculation for each level of stroke severity was described using the NIHSS97 to ensure consistent interpretation. These definitions are available in Table 3.

• Consideration of explicit versus implicit influences on decision-making. The expert panel acknowledged the need to also consider implicit factors as having an impact on decision-making about thrombolysis. There is strong evidence that implicit factors can be influential in medical decision-making. 81 Therefore, the panel also considered factors that did not emerge from the exploratory work and SPE. For instance, there is a suggestion that sex may have a population-level effect on thrombolysis administration, with evidence of underutilisation of intravenous thrombolysis in women,40 although this may reflect a different age and presentation profile in women. 41 This suggestion comes despite a pooled analysis of randomised controlled trials demonstrating that women are more likely than men to benefit from thrombolysis. 42,43 The panel did not consider sex to be a conscious consideration at the individual, bedside level. Some factors may have an implicit effect on decision-making, such as ethnicity and social class. Panel discussions were focused on striking a balance between the inclusion of both implicit and explicit factors which may influence thrombolysis decision-making.

Consensus was reached during expert panel discussions on the factors and levels that were included in a pilot DCE.

Stage 4: pilot testing

A specialised software program, NGene (v1.1.1, Choice Metrics, Sydney, NSW, Australia; www.choice-metrics.com/), was used to generate a fractional factorial (d-efficient) design. This facilitated the generation of a design that was as balanced as possible, given the imposed constraints. A fractional factorial design was employed owing to the relatively large number of variable factors (nine) included in the final DCE, with number of levels varying from two to six on different factors. This allowed for the testing of a subset of possible combinations. The use of a blocked design allowed the number of vignettes presented to any one participant to be reduced to a number considered manageable in pilot testing to avoid overburdening participants. In each block, 13 hypothetical patient vignettes were presented to each participant.

The pilot testing of the initial DCE employed a ‘think-aloud’ approach that was guided by best practice guidelines. 98 This method asks participants to verbalise what they are thinking when responding to the survey. The aim of this pilot stage was to facilitate further testing of the credibility of the vignettes among the population of interest, examine participants’ understanding and interpretation of the task and questions, check the usability of the survey on different systems and browsers, and gauge how long the survey might take to complete.

Sixteen clinicians were invited to take part in pilot testing. Six agreed to take part and fully completed the pilot. Based on the results of pilot testing, some amendments were made. For instance, when two clinicians read that symptom onset began 4 hours and 15 minutes ago, they factored time to prepare the thrombolysis bolus into their decision-making (as the thrombolysis time window for treatment is up to 4.5 hours post symptom onset). Therefore, thrombolysis bolus preparation time was included as an additional fixed factor, stating that the treatment dose could be prepared within 5 minutes to address this.

The testing revealed that vignettes were credible, although a number of implausible combinations were suggested and added as constraints (i.e. excluded from the design). These included low NIHSS scores with aphasia combined with moderate/severe dementia. Three participants also commented that spacing should be larger within tables to improve readability. This was amended prior to the final testing of the survey by the expert panel. The pilot testing protocol is available in Appendix 3.

Stage 5: final expert panel discussions, key considerations and finalised discrete choice experiment design

Key considerations of the development of this online DCE were the size of the population of interest, the number of combinations of factor levels, respondent burden and likely response rate. It was crucial to consider these issues alongside the perceived importance of various factors and levels. Table 5 displays the fixed factors and levels included in the final survey and the rationale for their inclusion. Table 6 outlines the final variable factors and levels included in the final DCE and Table 7 provides the definitions used on various factors and levels. Based on our five-stage design process, we were confident that we had provided sufficient information to allow clinicians to reach a decision by using fixed and variable factors. The full survey is available in Appendix 4, and includes all demographic questions and scales used to collect information on clinician characteristics. A sample vignette is included in Figure 2. Optional free-text comment boxes were included after each vignette to allow participants to explain their decision-making and include comments as they felt necessary. These were included to assist with data interpretation.

FIGURE 2.

Sample patient vignette.

Institutional Culture Scale

For the purposes of the current research, in the absence of an available scale, a new scale was developed (from existing measures where possible) to gauge the culture of thrombolysis and stroke care, and the influence of colleagues at a respondent’s place of work. A six-item scale was collated to assess this dimension and Table 8 includes the items used for this purpose. Items were scored on a 6-point Likert-type scale from strongly disagree (1) to strongly agree (6). Factor analysis was conducted on this new scale and a one-factor solution was obtained, representing institutional culture of thrombolysis. The results of the complete results of the factor analysis are available in Appendix 5.

Attitude towards risk-taking

The risk-taking subscale of the Jackson Personality Inventory100 was employed. It is a six-item measure that has been previously used in clinical decision-making research59,101 and measures attitudes towards risk-taking. Scores were summed (reversed when appropriate) and, given that there were no missing data on scale responses, a final risk-taking raw score was computed for each individual.

Physician Reaction to Uncertainty scale

The Physician Reaction to Uncertainty scale was developed by Gerrity et al. 60,61 to assess the impact of affective uncertainty on clinicians. The scale has been widely used in medical decision-making research60–62,102 and was employed to understand how respondents react to and deal with uncertainty in their clinical practice regarding acute stroke care and decision-making about thrombolysis. Scores were summed (reversed when appropriate) and, given that there were no missing data on scale responses, a final raw score was computed for each respondent.

Questions relating to decision-making and experience of administering thrombolysis

Clinicians were also asked to state the recency of their last decision made about patient suitability for thrombolysis, how many patients they estimated they had thrombolysed in the past 12 months, how many were harmed as a result of treatment in the past 12 months and how recently a patient was harmed as a result of treatment with intravenous thrombolysis. These questions were aimed at assessing whether or not recency and regularity of thrombolysis decision-making, and recency and frequency of negative outcomes of thrombolysis, exerted an influence on decision-making about the offer of treatment. The recency of last decision about thrombolysis and recency of harmful outcome as a result of administration of thrombolytic treatment were transformed into total number of days for the purposes of the analyses.

Clinicians were also asked to state whether or not there was a formal acute stroke protocol for assessing patient eligibility for thrombolytic treatment at their site of practice. A 6-point Likert scale was also used to assess the impact of clinicians’ level of comfort treating a patient outside the licensing criteria on thrombolysis decisions. Two final questions (6-point Likert scales) in the survey asked clinicians to rate their level of comfort with communicating the risk/benefits of thrombolysis to patients (and/or family/carers) and their perception of the evidence base for thrombolysis (i.e. whether or not respondents believe that the evidence for treating acute stroke patients with intravenous thrombolysis is robust). Finally, participants were debriefed and thanked for their participation.

The responses to the questions on clinician characteristics were analysed using appropriate descriptive statistics.

Scripting error

On receipt of the data from the market research company, we identified that the programmers had made some errors in the final application of scripting the online survey. This was thoroughly investigated and we identified that, as a result, 10.3% of the vignettes included implausible combinations of patient characteristics (e.g. severe dementia with pre-stroke dependency of mRS1 and severe dementia with NIHSS 2 with aphasia). We therefore ran models both with and without the erroneous vignettes and found little difference in the results. We decided that our primary analysis would exclude the erroneous vignettes. All analyses reported are on this basis.

Model 1: patient-related factors

The results of the mixed-logit analysis for model 1 are presented in Table 12. The ASC is negative and statistically significant, indicating a general preference not to offer intravenous thrombolysis, all else equal.

Compared with their reference categories (in brackets), four patient factors were statistically significant predictors of decisions not to offer thrombolysis: (1) patient age of 95 years (68 years); (2) stroke symptom onset time of 4 hours 15 minutes, proximal to the upper limit of 4.5 hours stipulated in the licensing criteria for thrombolysis (50 minutes); (3) patients with severe dementia (no history of memory problems); and (4) systolic blood pressure (SBP) values of 200 mmHg (140 mmHg).

There were also eight statistically significant predictors of decisions to offer thrombolysis (compared with reference levels in brackets): (1) patient age of 85 years (68 years); (2) Afro-Caribbean ethnicity (white); (3) stroke symptom onset time of 2 hours 30 minutes (50 minutes); (4) a pre-stroke dependency score of mRS3 (mRS4); and (5) SBP values of 185 mmHg (140 mmHg). Compared with the stroke severity reference category of NIHSS 2 (without aphasia), respondents were significantly more likely to offer thrombolysis to patients with scores of NIHSS 5 (without aphasia), NIHSS 14 and NIHSS 23.

The standard deviation for the ASC is statistically significant suggesting there was considerable heterogeneity among respondents in their thrombolysis decision-making. There was also substantial heterogeneity on several of the random effects coefficients (i.e. the patient factor levels), with nine standard deviations significant, indicating substantial variation between participants in their thrombolysis decisions as a function of differing levels of patient-related factors; or put differently, the response to or preferences in relation to particular attributes differed over clinicians.

Model 2: patient-related factors and prioritised clinician factors

Table 13 presents the results of the mixed-logit regression analysis for model 2, which includes the nine patient-related factors and seven clinician characteristics added to the model as a way to explore observed heterogeneity. These variables were prioritised for inclusion based on an a priori ranking exercise conducted by the expert panel (see Chapter 3, Analyses conducted on the data set). The clinician variables included were respondents’ perception of the evidence base for thrombolysis; Physician Reaction to Uncertainty Scale score; clinicians’ attitude towards risk score; the number of days since a patient was harmed as a result of treatment with thrombolysis; total number of patients thrombolysed in the past 12 months; comfort treating patients outside criteria; and number of patients harmed by thrombolysis in the past 12 months.

Consistent with model 1, the ASC for model 2 is negative and statistically significant, indicating that there was a general preference not to offer intravenous thrombolysis. The standard deviation for the ASC for model 2 is also statistically significant, which indicates the presence of considerable heterogeneity among respondents regarding their thrombolysis decisions. There was also considerable heterogeneity for the random-effects coefficients (i.e. the levels of patient factors), with the standard deviations of 17 factor levels emerging as statistically significant.

The pattern of results obtained in model 2 is broadly consistent with that of model 1 (see Table 12). All of the statistically significant patient factor predictors of thrombolysis decisions from model 1 remained significant contributors to the model; however, an additional two patient factors were statistically significant predictors of decisions to offer thrombolysis: (1) moderate dementia and (2) NIHSS 5 (with aphasia).

Four of the seven clinician factors were statistically significant predictors of thrombolysis decisions. Respondents’ were significantly more likely to offer thrombolysis when they (1) perceived the evidence base for thrombolysis to robust; (2) had thrombolysed a high number of patients in the past 12 months; and (3) reported a higher level of discomfort with uncertainty (assessed by the Physicians’ Reaction to Uncertainty Scale). Finally, clinicians who reported being comfortable treating patients outside the licensing criteria were significantly less likely to offer thrombolysis.

Comparison of the Akaike and Bayesian information criteria established that model 2 was a better fit than both model 1 and model 3. The full results of model 3 are included in Appendix 7.

Predicted probabilities

The predicted probabilities of offering thrombolytic treatment to patients described in each vignette (based on model 1) are available in full in Appendix 8.

Predicted probabilities for decisions to offer thrombolysis ranged from 0% (vignette numbers 8, 10, 14, 18, 19, 27, 34, 39, 43, 47, 57, 59, 65, 68, 71, 87, 80, 96 and 97; total observed rate of offering thrombolysis = 0% to 23.1%) to 99.86% (vignette number 70; total observed rate of offering thrombolysis = 100%).

Based on the utility values (using the coefficients from model 1), the vignette in which respondents were most likely to offer thrombolysis was number 70, which included the following level specifications:

Age 95; female; white; 2 hours 30 minutes since symptom onset; no history of memory problems; pre-stroke mRS3; NIHSS 14; systolic blood pressure 140 mm/Hg; who you do not perceive as frail.

The vignette in which respondents were least likely to offer thrombolysis was number 39, which included the following specifications:

Age 85; male; white; 4 hours 15 minutes since symptom onset; moderate dementia; pre-stroke mRS4; NIHSS 2 (without aphasia); systolic blood pressure 200 mm/Hg; who you perceive as frail.

Patient demographics (age, sex and ethnicity)

There were relatively few comments on patient sociodemographics (age, sex and ethnicity) as factors influencing decision-making for patient vignettes. Several respondents stated that patient age (independently) would not influence their decision-making about thrombolysis:

Age itself is not a barrier and within good time window to benefit; it rather depends on the breakdown of the NIHSS categories.

Male, stroke clinician, vignette 83, thrombolysis offered

Age alone not a reason NOT to [thrombo-] lyse.

Male, stroke clinician, vignette 72, thrombolysis offered

However, a large proportion of comments related to the interaction of patient age and onset time when making thrombolysis decisions, in particular with reference to patients aged > 80 years and evidence from IST-3:

I have been taught locally that due IST3 data suggesting benefit of rTPA after 3h in over 80s is minimal.

Female, neurologist, vignette 12, thrombolysis not offered

The IST3 trial did not show benefit in treating patients over 80yrs beyond 3 hours.

Male, stroke clinician, vignette 80, thrombolysis not offered

No respondents made any comments about patient sex. Only two comments explicitly referred to patient ethnicity in combination with other factors; specifically, the cultural background of Asian patients was mentioned in both instances as drivers of decisions to offer thrombolysis, with references to beliefs about cultural (familial) values and acceptability of palliative care in hospital:

Mobility and level of communication with family as well as participation in family life would be important. Some Asian elderly people can do actually better to what ADLs [activities of daily living] assessments are.

Female, stroke clinician, vignette 17, thrombolysis offered

IST-3 shows she could be safely treated; she is likely to be at home prior to this given her cultural background and would wish to remain so; leaving her with this stroke may mean she needed a PEG and/or institutional care which would not be in keeping with her culture?

Female, geriatric medicine physician, vignette 83, thrombolysis offered

Frailty

Patient frailty was not independently noted as a contraindication for treatment with thrombolysis, although frailty in combination with other patient factors such as mild stroke and moderate/severe dementia (and concomitant perception of increased bleeding risk) was frequently cited as a reason not to offer treatment.

Low NIHSS and high risk of bleeding as delayed presentation. Also frail. I feel the risk would be too high.

Male, geriatric medicine physician, vignette 39, thrombolysis not offered

The patient is frail, dependent with a MRS of 4 and has a low NIHSS.

Male, geriatric medicine physician, vignette 16, thrombolysis not offered

Pre-stroke cognitive status

In vignettes where patients had moderate or severe dementia, many clinicians commented on the lack of evidence (and uncertainty) about the efficacy of thrombolysis and on increased bleeding risks, including concerns about the negative impact of severe dementia on the recovery process:

Evidence regarding safety of thrombolysis in dementia is lacking. Demented patient > 85 yrs may have microbleeds. Given mRS of 4, and reduced likelihood of benefit from late treatment, I would assume risk outweighs benefit.

Male, stroke clinician, vignette 7, thrombolysis not offered

Certainly not if moderate or severe small vessel changes on CT scan. If no such changes then perhaps: but severe dementia a very limiting factor to recovery.

Male, stroke clinician, vignette 71, thrombolysis not offered

I would perceive this frail patient at high risk of complications, although this may be balanced by the early time frame and significant motor deficit. I know that because of his dementia, he will find rehab difficult, so maybe thrombolysis is his best shot. I have easy rapid access to stroke physicians and this is the sort of case we would manage together.

Male, A&E physician, vignette 54, thrombolysis not offered

History of severe dementia would deter me from thrombolysis in this case.

Female, geriatric medicine physician, vignette 45, thrombolysis not offered

Systolic blood pressure

Comments about this patient factor related to both measured blood pressure values and blood pressure lowering. Regarding the latter, several clinicians would attempt to lower blood pressure (when it was above the licensing criteria cut-off of 185 mmHg) and then reassess the patients’ suitability for treatment, whereas others would offer treatment and simultaneously take steps to reduce SBP, although the importance of fully informing patients/family about the increased risk of bleeding in these circumstances was emphasised:

High BP but will treat and reassess.

Male, geriatrician, vignette 20, thrombolysis not offered

I would treat this BP first and if it came down easily I would lyse [thrombolyse] him with family assent.

Male, stroke clinician, vignette 53, thrombolysis not offered

After long discussion with patient and only when satisfied patient knows the risk. Would also simultaneously treat blood pressure.

Male, A&E physician, vignette 69, thrombolysis offered

There was also a subset of respondents who indicated they would actively monitor the patient’s blood pressure and delay treatment decisions until it was below the threshold stated in the licensing criteria. These considerations were influenced by onset time, and were mentioned in patient vignettes only when stroke symptom onset time was well within the 4.5 hours (i.e. either 50 minutes or 2 hours 30 minutes):

I would wait to see if the BP settled.

Male, stroke clinician, vignette 29, thrombolysis not offered

Might wait additional 15–30 minutes to check if SBP drops to criteria level before thrombolysis.

Male, stroke clinician, vignette 33, thrombolysis not offered

I would monitor SBP, ensure no pain, retention etc. but not pharmacologically lower immediately.

Male, geriatric medicine physician, vignette 22, thrombolysis not offered

Stroke severity score (National Institutes of Health Stroke Scale)

Respondents would be less willing to treat a patient with thrombolysis with mild stroke (NIHSS score of ≤ 5). NIHSS being ‘too low’ to warrant treatment was often cited by respondents as the primary reason underpinning their decisions not to offer thrombolysis to patients described in hypothetical vignettes. This may reflect the lack of a specific minimum NIHSS cut-off being stated in the licensing criteria and a lack of robust evidence for absolute benefit of treatment for very mild stroke:

NIHSS too low.

Female, geriatric medicine physician, vignette 30, thrombolysis not offered

Very mild stroke. No evidence substantial gains to be had from TPA.

Male, neurologist, vignette 13, thrombolysis not offered

However, some clinicians commented that low NIHSS with aphasia may influence their decision to offer treatment, as they considered aphasia a ‘significant deficit’ that would impact negatively on quality of life (which trumped low NIHSS scores).

Although NIH low, aphasia very disabling, potentially will mean significant change in social and functional status. Not bothered by age.

Female, geriatric medicine physician, vignette 30, thrombolysis offered

I don’t use age as a criteria - she has some dependency but not the severest level. This is not a mild stroke – aphasia if long lasting could make a big difference to her care requirements.

Male, stroke clinician, vignette 23, thrombolysis offered

At the higher NIHSS scores, even though respondents were cognisant that patients with high NIHSS scores were significantly more likely to have a poor outcome from thrombolysis, clinicians were more likely to offer treatment based on considerations that they had more to gain from treatment:

At higher risk of poorer outcome due to high NIHSS but time of onset within original NINDS 3 hour window so would thrombolyse.

Female, neurologist, vignette 21, thrombolysis offered

I would explain that the chances of a good outcome are lower than usual but she has had a very severe stroke and has little to lose.

Male, stroke clinician, vignette 35, thrombolysis offered

Severe Stroke. Little to lose from Rx[treatment].

Male, stroke clinician, vignette 60, thrombolysis offered

Pre-stroke dependency (modified Rankin Scale)

The influence of pre-stroke dependency on thrombolysis decision-making was variable. Patients’ pre-stroke dependency was described by some clinicians as a contraindication for treatment when the mRS score was 3 or 4. This may reflect modest inter-rater reliability of mRS assessments (κ < 0.5)105 and considerations about quality of life that may trump mRS assessments. Indeed, several clinicians commented that they would not rely on mRS alone in their assessment of quality-of-life gains from treatment and risk–benefit ratios for individual patients.

mrs = 3

Male, neurologist, vignette 59, thrombolysis not offered

MRS 4 risk > benefit

Male, neurologist, vignette 85, thrombolysis not offered

Too high level of dependency is primary reason for not thrombolysing

Male, A&E physician, vignette 27, thrombolysis not offered

mRS doesn’t tell the whole story so I would want to get a quick chat with family or carer to get a clearer picture of QOL. If appropriate based on this discussion I would lyse [thrombolyse]

Male, stroke clinician, vignette 35, thrombolysis offered

Symptom onset time

Early presentation/symptom onset time was a key driver of decisions to offer thrombolysis (which reflects robust evidence of increased benefit from earlier treatment) and other potential contraindications were traded off (such as severe stroke and mRS score of > 2) against the potential benefit of early treatment. In these cases clinicians were willing to give patients the ‘benefit of the doubt’, as such patients were considered to have ‘little to lose’.

This is a severe stroke with a very poor outcome. She has nothing to lose from treatment particularly as she can be treated early in time window.

Male, stroke clinician, vignette 15, thrombolysis offered

Very early presentation and therefore maximal likely benefit (& normal BP - probably less likely to bleed) in patient with significant speech defect probably will justify the treatment even with less than favourable premorbid state.

Male, stroke clinician, vignette 23, thrombolysis offered

With early presentation benefit probably outweigh concerns about premorbid state.

Male, stroke clinician, vignette 74, thrombolysis offered

However, when symptom onset was 4 hours 15 minutes (approaching the upper end of the treatment time window for thrombolysis) clinicians were reluctant to offer treatment in the context of other factors that would indicate a higher risk of bleeding/poor treatment outcome.

The time factor is important and it is 4 hours 15 minutes + the NIHSS of 2 would not persuade me to thrombolyse.

Male, stroke clinician, vignette 39, thrombolysis not offered

Time since symptom of onset is quite long and if true would decrease the likelihood of benefit from thrombolysis. Would discuss with consultant/seniors but my inclination would be not to thrombolyse.

Female, neurologist, vignette 35, thrombolysis not offered

Consult with colleagues or patient/family

There were many instances in which respondents would consult with colleagues in stroke medicine and/or engage the patient or family/carer in shared decision-making before making a final decision about thrombolysis, especially for cases at the extremes (grey zone) of the licensing criteria. Several respondents also expressed a wish to enquire further about the patient’s living status (nursing home or otherwise) in order to inform their thrombolysis decisions:

This is one I would discuss with a stroke physician. I would consider the patient to have high vascular fragility and at risk from complications. However, this patient has a significant disability. This is a case for shared decision making.

Male, A&E physician, vignette 23, thrombolysis not offered

Approaching the end of the longer 4.5 hour time window therefore I would discuss with consultant covering stroke whether their practice would be to thrombolyse or not.

Female, neurologist, vignette 50, thrombolysis not offered

I would discuss with family. She could be a good candidate but needs a best interest assessment given low NIHSS and high premorbid disability.

Female, neurologist, vignette 95, thrombolysis not offered

Patient age

Patients aged 95 years were significantly less likely to be offered treatment than patients aged 68 years. Respondents were also significantly more likely to treat 85-year-olds than patients aged 68 years. The benefits of thrombolysis for older patients in terms of reduction in disability is at least as great as those for younger patients, despite an increased risk of sICH. Patients aged 85 years have more to gain from treatment than younger patients (irrespective of risk) and this would also reflect acceptance of IST-3 data that report benefit for patients aged > 80 years, in particular for those with onset to treatment time of < 3 hours (the cut-off stated in the licensing criteria and the upper age limit in trials) and increased recognition that chronological age does not necessarily equate to physiological age. 106 That respondents were more likely to treat 85-year-olds than 68-year-olds could indicate that, all things being equal, clinicians may be more willing to risk a potential negative outcome of thrombolysis in 85-year-old patients than in younger patients. Based on the results and qualitative data, it appears that clinicians are not disinclined to treat older patients until those patients reach higher ages (≈95 years). It may also indicate that clinicians are more likely to give the ‘benefit of the doubt’ to older patients and avoid denying treatment based on age. It is also feasible that a social desirability effect has been observed, whereby respondents may be overcompensating to avoid denying treatment based on age.

However, disinclination to treat patients aged 95 years may represent the upper limit of data from IST-3 or reflect clinical practice or experience (there are few patients who have been treated in this age range in routine practice). For instance, SITS data indicate that only approximately 10% of treated patients were older than 80 years106,107 and Stroke Improvement National Audit Programme data indicate that only 2.5% of the total number of patients who received thrombolysis between April 2010 and November 2011 were older than 90 years. 108

Time since symptom onset

Regarding the ‘time since symptom onset’ factor, consistent with expectations and strong evidence of increased benefit with earlier treatment,16 we identified a statistically significant decreased likelihood of offering treatment to patients who could be treated at 4 hours 15 minutes from symptom onset compared with 50 minutes from symptom onset. The former is at the extreme end of the licensing criteria, and, given the time dependency of the outcome, and that onset to treatment is estimated, would account for this time period being an inhibitor of decisions to offer treatment. Clinicians were significantly more likely to treat patients at 2 hours 30 minutes than at 50 minutes. This could indicate that clinicians have not been in this situation very often because acute stroke presentations typically occur later than this, and it may be that clinicians wait a short time in some borderline cases to see if the symptoms spontaneously improve.

This may indicate that some clinicians are willing to delay the final decision about thrombolysis if there are some factors which create uncertainty; for instance, in mild stroke when a patient’s symptoms are rapidly improving, the clinician may delay to see if symptoms resolve (per licensing guidelines45) or to potentially give blood pressure time to stabilise or to take more readings before making a final decision about the patient’s eligibility for thrombolytic treatment. 109–111 This is in spite of research which confirms that earlier treatment is associated with substantially more favourable outcomes,16 particularly when treatment occurs within the ‘golden hour’ (< 60 minutes from symptom onset),110 and that delaying treatment owing to perceived patient improvement or in the case of minor stroke could still risk significant disability or death. 111

Stroke severity score (National Institutes of Health Stroke Scale)

Respondents were significantly more likely to offer thrombolysis to patients with severe strokes (NIHSS 14 and NIHSS 23) and to not offer thrombolysis to patients with mild stroke. This finding reflects the lack of data on and uncertainty about the benefit of treatment of minor stroke and about poor outcomes for patients with severe strokes. There was significant heterogeneity among respondents on the influence of NIHSS 2 with aphasia and NIHSS 5 without aphasia. This implies that clinicians differ in their thresholds for treatment of minor stroke and may also differ when considering the gains in quality of life for individual patients with isolated language difficulties.

Patient ethnicity

Compared with white patients, Afro-Caribbean patients were significantly more likely to be offered treatment, although there was no effect for Asian ethnicity. This is contrary to US-based research which found that African American and minority patients were significantly less likely to receive thrombolytic treatment for acute ischaemic stroke. 112,113 Although efforts were made to exclude any diagnostic uncertainty, these findings may be related to a perception of increased diagnostic certainty of stroke, given the evidence of increased rates among this ethnic group. However, social desirability bias may help to account for the findings, with clinicians overcompensating to avoid being seen as denying treatment based on race.

Pre-stroke dependency (modified Rankin Scale)

Patients with pre-stroke dependency of mRS3 (moderate disability: requiring some help, but able to walk without assistance) were more likely to be offered treatment than those with mRS4 (moderately severe disability: unable to walk without assistance and unable to attend to own bodily needs without assistance), but mRS1 (no significant disability despite symptoms) was not significantly associated with thrombolysis decision-making. We would have expected that respondents would be more willing to treat patients with mRS1 than those with mRS4; however, the failure of this to reach significance may be due to an imbalance between the levels of pre-stroke dependency in the study. Vignettes including mRS1 were under-represented and therefore may have reduced the power to adequately compare these levels.

Pre-stroke cognitive status

Patients presenting with severe dementia were significantly less likely to be offered thrombolysis in models 1 and 2, with moderate dementia emerging as significant in model 2, indicating that clinicians were more likely to offer thrombolysis in these cases (compared with patients with no memory problems). Inspection of the standard deviations reveals significant variation between respondents on the influence of both moderate and severe dementia on thrombolysis decision-making. This again emphasises that clinicians are weighing up the pros and cons of treatment with reference to individual patient characteristics that are not part of the licensing criteria, although it may also reflect perceptions that dementia increases risk of sICH, which is contrary to research demonstrating no increased risk. 114 In the comment boxes after the vignettes, many respondents expressed that they would seek to discuss the risks and benefits of treatment with the patient’s family/carer or seek further information about living status/dependency level prior to making a final decision about the offer of treatment. It was also noted that patients with dementia may find recovery/rehabilitation more difficult and, therefore, in some instances, thrombolysis may be worth the risk in order to give the patient the best chance of regaining his or her pre-stroke quality of life.

Systolic blood pressure

Patients presenting with SBP of 200 mmHg were significantly less likely to be offered treatment than those presenting with SBP of 140 mmHg. Given that this level is outside the licensing criteria for treatment with thrombolysis, this is not surprising. However, results revealed that clinicians were more likely to offer thrombolytic treatment when SBP was 185 mmHg (compared with the reference category of 140 mmHg). This finding was surprising and no clear explanation was suggested for this in the comments. It may be that once clinicians make the decision to attempt to reduce blood pressure, the decision to treat with thrombolysis is reinforced. Thus, the decision to lower blood pressure may become interlinked with the decision to offer thrombolysis, increasing the likelihood of thrombolysis being offered if attempts are made to reduce blood pressure.

Clinician factors

Four of the seven clinician factors ranked a priori as potential significant influences on decision-making emerged as significant. There was a significant association found between the respondents’ perception of the evidence base and the offer of thrombolysis. Although the mean score for perception of the evidence was high, suggesting that the majority of respondents considered the evidence robust, those who perceived the evidence as more robust were more likely to offer treatment.

The Physician Reaction to Uncertainty scale60,61,102 has been demonstrated to influence diagnostic decision-making and, in the current study, higher discomfort with uncertainty was associated with increased likelihood of offering thrombolysis. This may indicate a preference for action over inaction in instances of high uncertainty, or what may be termed commission bias. 54 Commission bias is defined as the tendency towards action/intervention rather than inaction. 54 Given the high scores observed on the Institutional Culture Scale in the study, indicating a strong culture of administering thrombolysis in respondents’ institutions, clinicians may worry more about decisions not to treat and therefore those with higher levels of discomfort with uncertainty may be more willing to thrombolyse patients. Clinicians who experience high discomfort with uncertainty may be more likely to offer thrombolytic treatment to patients if it is perceived as the dominant or favoured position among colleagues or peers.

The results also indicated that clinicians who reported being more comfortable treating patients outside the licensing criteria were less likely to offer treatment in the current study; however, we do not have a clear explanation for this finding.

Finally, there was a significant association between clinicians’ experience of thrombolysis and the likelihood of offering thrombolysis in the current study. This may indicate that familiarity with administration of the treatment (and positive outcomes) increases likelihood of future use, and has important practical implications for how clinicians are supported when they begin involvement in decision-making about thrombolysis. Alternatively, it may simply represent the fact that those more likely to offer treatment in the DCE are more likely to offer treatment in practice, and therefore have higher reported treatment rates.

Papers

De Brún A, Flynn D, Joyce K, Ternent L, Price C, Rodgers H, et al. Understanding clinicians’ decisions to offer intravenous thrombolytic treatment to patients with acute ischaemic stroke: a protocol for a discrete choice experiment. BMJ Open 2014;4:e005612.

De Brún A, Flynn D, Ternent L, Price CI, Rodgers H, Ford GA, et al. Factors that influence clinicians’ decisions to offer intravenous alteplase in acute ischemic stroke patients with uncertain treatment indication: Results of a discrete choice experiment. Int J Stroke 2017: in press. DOI: 10.1177/1747493017690755.

Conference presentations

De Brún A, Flynn D, Ternent L, Lancsar E, Price C, Rodgers H, et al. Understanding clinicians’ decisions to offer intravenous thrombolytic treatment to patients with acute ischaemic stroke: a discrete choice experiment. 7th National Institute for Health Research Stroke Research Network Annual Meeting, Newcastle, UK, 19–20 September 2013.

De Brún A, Flynn D, Ternent L, Price CI, Rodgers H, Rudd M, et al. Development process for a discrete choice experiment to explore variation in clinical decision-making about thrombolysis in acute ischaemic stroke. Division of Health Psychology Annual Conference 2014, York, UK, 10–12 September 2014.

De Brún A, Flynn D, Ternent L, Lancsar E, Price C, Rodgers H, et al. A discrete choice experiment to explore factors influencing decision-making about treatment of acute ischaemic stroke. Oral presentation at the ISDM-ISEHC Conference, Sydney, NSW, Australia, 19–22 July 2015.

De Brún A, Flynn D, Ternent L, Lancsar E, Price C, Rodgers H, et al. Factors that influence clinicians’ decision to offer intravenous thrombolysis for patients with acute ischaemic stroke. Poster presentation to the ISDM-ISEHC Conference, Sydney, NSW, Australia, July 2015.

De Brún A, Flynn D, Ternent L, Lancsar E, Price CI, Rudd M, et al. Patient factors that influence variation in clinical decision-making about thrombolysis in the treatment of acute ischaemic stroke: results of a discrete choice experiment. Poster presentation, UK Stroke Forum, Liverpool, UK, 1–3 December 2015.

Training/continuing professional development events

Presentation and interactive session at the Northern Deanery Neurology Specialist Registrar Training Day, The Royal Victoria Infirmary, Newcastle upon Tyne, UK, 15 October 2015.

Presentation and interactive session at the Reperfusion Masterclass, Royal York Hotel, York, UK, 16 October 2015.

Papers

De Brún A, Flynn D, Ternent L, Lancsar E, Price CI, Rudd M, et al. A novel development and design process for selection of factors for inclusion in discrete choice experiments and optimal presentation of choice sets: case study exploring variation in clinical decision-making about thrombolysis in the treatment of acute ischaemic stroke. In preparation.