Preparatory study for the revaluation of the EQ-5D tariff: methodology report

Measuring cost-effectiveness

Health-care resources are limited and need to be allocated efficiently. The National Institute for Health and Care Excellence (NICE) was set up to help make better health-care resource allocation decisions. NICE bases its recommendations on cost-effectiveness analyses, with the quality-adjusted life-years (QALYs) gained as the outcome measure. A QALY combines values for quality and quantity of life in a single figure, and allows the assessment of the effectiveness across interventions and treatments for different conditions using a common metric. This requires a value for the health-related quality of life (HRQL) or utility for a particular health state, which is then multiplied by the duration of the health state to calculate the number of QALYs. Utility values can be generated using generic preference-based measures of health, such as the EQ-5D.

EQ-5D

The EQ-5D1 is the preferred instrument to use to derive utility values to assess the HRQL impact of medical interventions. 2 The EQ-5D assesses HRQL across five dimensions (mobility, self care, usual activities, pain/discomfort and anxiety/depression) each with three response levels (no, some or extreme problems). Therefore, the entire descriptive system generates 243 health-state descriptions, each of which produces a utility value (known as the value set).

The current UK EQ-5D value set is based on the Measurement and Valuation of Health (MVH) study. This study used face-to-face interviews of a representative sample of the UK general population to value 45 hypothetical EQ-5D states using the preference elicitation method time trade-off (TTO). The results of the valuation study were modelled using regression to provide a utility score for all 243 health states (range of −0.594 to 1). 3 Utility scores are anchored on a 0–1 scale, where 1 is equivalent to full health, 0 to dead, and negative values to states worse than dead.

The UK EQ-5D value set is used for economic evaluations by a range of decision-makers and researchers. They are also used in a range of further applications, including population health surveys (e.g. the Health Survey for England); burden of disease studies; hospital inpatient surveys and the NHS Patient Reported Outcome Measures (PROMs) initiative. 4 However, there is the need for a new EQ-5D value set to be developed for the following reasons:

• A five-level version of the EQ-5D, the EQ-5D-5L,5 has been developed (response categories: none, slight, moderate, severe, extreme/unable). This version generates a possible 3125 health states, and there is the need for a value set to be developed for the larger descriptive system so that the instrument can be used in the economic evaluation of new interventions and treatments.

• It is possible that the preferences of the general population may have changed since the original MVH study was carried out in the 1990s.

• Change in demography may mean that although individual preferences may not have changed, the composition of people across the country has changed, so that average preferences may have changed.

• There has been recognition of the shortcomings of the MVH protocol used to generate the UK EQ-5D value set, in particular in regards to the valuation method used for states perceived by respondents as worse than dead.

• There have been advances in health-state valuation methods, including the potential application of discrete choice experiments (DCEs) to derive utility values, by including duration as an attribute [discrete choice experiment incorporating duration (DCE_TTO)].

• There have been advances in the administration modes available for valuation studies [e.g. using computer-assisted personal interviews (CAPIs) or online methods].

The ‘Preparatory study for the Re-valuation of the EQ-5D Tariff project’ and ‘Preparatory study for the Re-valuation of the EQ-5D Tariff project – Additional Sample’

The methods used for the generation of EQ-5D-5L population value sets needs to be up to date and informed by the latest understanding of the techniques used to value health states. The ‘Preparatory study for the Re-valuation of the EQ-5D Tariff project’ (PRET) is a methodological study that aims to contribute to the generation of EQ-5D-5L population value sets by exploring a range of methodological issues associated with a range of health-state valuation techniques, including TTO and DCE_TTO. The ‘PRET – Additional Sample’ (PRET-AS) study is an extension to PRET and allows further investigation into health-state valuation-related methods. This report will cover both projects.

The PRET study is a methodological study that has four stages. In stage 1, a large scale online survey is carried out to explore a series of methodological issues related to health-state valuation using binary choice questions. PRET-AS is an extension of stage 1, and involves a further online survey investigating two binary choice techniques that can be used to generate utility values. In stage 2, a segment of the PRET stage 1 online survey is carried out in a face-to-face environment using CAPI to test the equivalence of responses to the valuation questions across different modes of administration. Stage 3 uses CAPI to investigate the strategies and processes used to answer health-state valuation questions based around DCE_TTO and TTO. Stage 4 uses in-depth cognitive interviews to investigate the completion of TTO and DCE_TTO in more detail.

The remainder of this chapter is organised as follows. In the Methods of health-state valuation considered in PRET section below, the health-state valuation techniques TTO and DCE_TTO are introduced. In the Issues with the design of health-state valuation studies considered in PRET and PRET-AS section, the methodological issues that need to be considered in the design of any health-state valuation study are outlined, with reference to the issues investigated in this study.

Sections of this chapter are reported in an online discussion paper6 (accessible at www.shef.ac.uk/polopoly_fs/1.165490!/file/1116.pdf).

TTO

Time trade-off7 is a widely used method for valuing health-states valuation, and the MVH TTO protocol8 that was used to derive the EQ-5D value set1,3 has also been used to generate utility scores for condition-specific preference-based measures of health. 9–13

Time trade-off is an iterative cardinal technique that elicits a health-state value by asking respondents to trade off time in full health to avoid living in a hypothetical health state described by the classification system. The value is derived at the point where respondents are indifferent between the scenarios. The MVH TTO protocol used face-to-face interviews, and the task follows a set procedure to derive each utility value. First, respondents are asked whether they would prefer to live in health state (H) for 10 years, or to die immediately, or whether they were indifferent between the options. This established whether the health state was perceived as better than, worse than, or equal to being dead.

For states perceived as better than dead, respondents choose between (1) living in H for T years (usually 10) or (2) living in full health for X years (where X ≤ 10). The duration of full health (X) is varied iteratively until respondents are indifferent between the options. The value for H is then calculated as X/10. Typically, zero time preference is assumed.

For health states that are ‘worse than dead’, respondents are asked to consider a choice between (1) H for W years followed by full health for X years (where W + X = 10) after which they will die or (2) immediate death. Both years in full health, X, and years in the health state, W = 10 − X, are varied until respondents are indifferent between the two options. However, this may result in extreme values. For example, if a respondent is indifferent between (1) H for 3 months followed by full health for 9 years 9 months after which they die and (2) immediate death, this would suggest that the value of health is −0.25/9.75 = −39 on a scale on which ‘1’ = full health and ‘0’ = being dead. Traditionally, this has been regarded as unacceptable and arbitrary transformations have been applied. 3,14–16 Under the established convention, H is calculated as −X/10.

Regardless of whether the state is better or worse than being dead, the iterative process is susceptible to bias. This is because when people are asked two (or more) consecutive questions, the later question is not independent of the earlier question. For example, the response given to a TTO question where X is 3 years will be affected by the value of X used in the preceding question (e.g. whether it was 1 year or 5 years). The issue of biases caused by iterative questioning is a key topic in the literature on the monetary valuation of health but is under-researched in the literature on health-state valuations.

To estimate utility values for each health state defined by a classification system, the results of the TTO study are modelled using multivariate regression. The disutility coefficient for each severity level of each dimension is calculated using level 1 (no problem) as the baseline. Therefore, full health is anchored at 1, and the utility value for each overall health state is calculated by subtracting the disutility value for each dimension from 1.

Lead time – time trade-off

A concern with the MVH TTO protocol above is the process used to value states worse than dead. 16–18 The following problems were identified with this procedure: (1) The method is not the same as the method for states better than dead; (2) time spent in health state H has become the decision variable, with the result that respondents are not valuing a specified time in the state (as is the case for states better than dead); and (3) the raw result is non-linear for the time spent in state H, such that as H approaches zero the index approaches negative infinity ( Figure 1 ) and, as is pointed out above, transformations used are arbitrary, and render the values incommensurable with those for states better than dead.

FIGURE 1.

Increase in health-state value as time in poor health decreases.

The lead time – time trade-off (LT-TTO) was devised to overcome these problems. In order to allow X to take negative values, LT-TTO appends a stretch of ‘lead time’ in full health before the usual TTO scenarios begin. For example, if lead time is 15 years, a LT-TTO question compares living in full health for 15 years followed by H for 10 years against living in full health for some duration between zero and (15 + X) years. If indifference is achieved at, say, 20 years in full health, then by removing the lead time this is the same as X = 5 in a TTO without lead time (20 − 15 = 5). If indifference is at 12 years in full health then this corresponds to a ‘negative duration’ (12 − 15 = −3). Note first that the value of living in H for 1 year is given by X/10, regardless of whether H is better or worse than dead, and, second, the calculation assumes that the size of T is unaffected by the addition of the lead time. 16

As can be seen, a lead time of 15 years against a duration of 10 years will allow LT-TTO to elicit values in the range [−1.5 to 1]. If the value of H is < −1.5 then a respondent will strongly prefer immediate death over the prospect of 15 years of lead time plus 10 years in state H. This is called ‘exhausting’ the lead time, and if the objective is to identify a point of indifference for all states for all respondents, it calls for a longer lead time (or a shorter duration). There have been a number of attempts to explore the optimal ratio of the lead time to the duration but a clear answer is yet to emerge. 19

DCEs

There is growing interest in using ordinal techniques such as DCE to generate health-state values. 20 DCEs generate ordinal preference data by asking respondents to indicate their preferred option from a set of health-state profiles (usually two profiles are presented), in which each profile is described in terms of attributes and levels. The results of the choice exercise are then modelled using regressions to generate a coefficient value for each level of each attribute, or dimension. DCE does not require the application of an iterative procedure, and therefore may be less cognitively demanding and avoid the bias associated with iterative procedures.

Discrete choice experiment assumes that preferences are measured on a ‘latent’ scale, and are directly unobservable, but can be modelled in terms of observed characteristic of each choice. As a result, the raw regression coefficients are also on a latent scale, with no direct meaning. Thus, to use DCE to generate utility values that can be used as the HRQL adjustment weights for the QALY, coefficients must be anchored on the full health–dead utility scale. This has typically been done using external values generated, for example, from a TTO exercise. 21 Recently, a method has been developed that avoids the need to use external values by incorporating duration as an attribute of the health-state profile, therefore interpreting DCE data as a TTO exercise (DCE_TTO). 22 To estimate utility values for health states, a regression model incorporating interaction terms between each level of the health-state dimensions and the duration attribute are estimated (see Chapter 8 , DCE _TTO analysis, for more details). The approach has been tested using EQ-5D health states, and has been shown to be a feasible approach to deriving logical and consistent health-state values.

Whose values?

The value sets for the three-level EQ-5D,3 and SF-6D,23,24 generic preference-based measures of health and a range of condition-specific measures9–13 are based on general population values, and this is recommended by NICE. 2 However, it is possible that the values given to hypothetical health states by the general population may differ from values given by patients, and there has been debate about whose values should be used. 25,26 Health state satisfaction and adaptation to the health state can be used as a proxy to test this issue as a recent study has demonstrated that if the general public can be informed about the extent to which patients are satisfied with their condition, the discrepancy in values may diminish. 27 PRET tests this further by incorporating a level of life satisfaction, health satisfaction, or adaptation into the health-state description. Alongside this, PRET also tests whether health-state values, which contain satisfaction levels, are influenced by the respondent’s level of satisfaction with their own health or life.

There is also a normative element to this debate, concerning whether general public values ought to be used over patient values. The use of general public values is typically justified with reference to the non-welfarist argument. This states that as the values are used for decision-making in a publicly funded health-care system, they should come from people as informed citizens, not from people as consumers. 28 The traditional approach to health-state valuation, and that used for the current EQ-5D MVH value set, has been to obtain valuations by asking respondents to imagine themselves in the health state. If an informed citizen perspective is taken then a different framing of the TTO question may be required to reflect that the respondent is valuing health states on behalf of other members of society. However, it is unclear what impact an alternative perspective will have on values. PRET investigates this by comparing responses using the standard individual perspective with two alternatives reflecting the citizen approach.

What mode of administration?

Face-to-face interviews with pen and paper questionnaires have been the most widely used method for collecting health-state valuation data using iterative techniques, and was the mode used for EQ-5D using TTO,3 and SF-6D using standard gamble (SG). 23,24 Advances in technology means that it is now also possible to carry out TTO studies using face-to-face CAPI, and this mode was used to derive EQ-5D population value sets for Australia29 and Denmark. 30 Health-state preferences have also been elicited using DCE in a face-to-face setting,21 and the feasibility of carrying out both TTO and DCE_TTO in an online setting has been investigated. 22

A comparison of the online and face-to-face delivery of TTO found that the responses differed by administration mode, with the online sample displaying more variation in response. 31 Tests of the person trade-off (PTO) valuation technique across online and CAPI administrations also found potential differences across modes. 32,33 Therefore, iterative health-state valuation tasks administered online may generate different results from face-to-face studies, but it is not clear whether the difference comes from the mode of administration or an interaction between the iterative task and the mode of administration. Furthermore, there are also concerns about potential differences in the characteristics of samples collected using face-to-face and online modes, and therefore the overall level of comparability.

The equivalence of responses to binary choice health-state valuation questions (which are amenable to online delivery) across administration modes has not been investigated. Therefore, one of the purposes of PRET was to carry out a head-to-head comparison of an online administration (in stage 1) and a CAPI administration (in stage 2) of an otherwise identical survey containing binary choice health-state valuation questions. A secondary aim is to investigate the similarities and differences of the samples recruited to each mode using the standard recruitment procedure utilised in studies of this kind.

What method of valuation?

As has been described above, there are a range of available techniques for the valuation of health states, and there is also the potential for new and innovative techniques to be developed in the future. The issues with each method need to be considered by those designing health-state valuation studies, as this may impact on the final value sets produced. Further work needs to be done to investigate a range of issues related to each technique described above (TTO, LT-TTO and DCE_TTO), and one of the aims of PRET is to further the knowledge base in this area, and therefore inform the choice of valuation technique for any health-state valuation exercise. This is described below.

As was described above, MVH TTO protocol used to value the EQ-5D has problems regarding the procedure used to value states worse than dead, and this lead to the development of LT-TTO. Further investigation of the LT-TTO technique is required to identify the optimal length of the lead time used, particularly in very poor health states for which a number of respondents may use up or ‘exhaust’ all of the lead time. One of the objectives of PRET was to provide evidence on this issue. Moreover, another concern is that if the value of a health state depends on its timing and on a preceding health state, then the addition of lead time may distort the TTO value. PRET compares the values produced using binary choice versions of the original and LT-TTO methodologies described in Chapter 2 .

The DCE_TTO has been shown to be a feasible method for producing values for EQ-5D. However, further testing using a larger descriptive system, such as that found in EQ-5D-5L, is required to investigate the validity of the technique further. PRET and PRET-AS investigate these issues further. Following on from the development of DCE_TTO, it is possible that other binary choice techniques could be used to produce utility values anchored on the full health–dead scale, and this includes versions of both TTO and LT-TTO, in which one of the choices includes full health. Little is known about the acceptability of both the traditional iterative and new binary choice methods for deriving utility values, and also the ways in which respondents perceive, process and complete the tasks. One of the objectives of PRET is to investigate these issues using both CAPI techniques and detailed qualitative interviews, and this may inform the choice of valuation technique used in future studies.

How many, and which hypothetical health states to value?

The original three-level EQ-5D has 243 possible states. The current MVH TTO value set is based on direct valuations of 45 of these. However, the introduction of EQ-5D-5L means that there are now 3125 possible health states to model. Findings from the DCE_TTO questions included in PRET, and the modelling approach used to select questions for the study, may be used as prior information to assist in the selection of states and design of the revaluation study.

One aspect that needs to be considered in the design of DCE_TTO studies is the number of choices each participant can be asked to make. In a recent review, De Bekker-Grob and colleagues20 found that the mean number of choice sets per respondent in health-related DCEs is 14 and it has been suggested that including 8–16 choice tasks is good practice. 34 Furthermore, limited formal work has been done to establish the sample size requirements for DCEs, and PRET and PRET-AS investigate these issues further.

How long should each hypothetical state last?

The current MVH TTO value set is based on participants being asked to imagine each health state lasting for a duration of 10 years. However, the MVH also estimated TTO tariffs for different durations because there was a concern that the tariff values may be a function of the duration of the health state. There are four related issues, all of which are also relevant to DCE_TTO. 35 One is whether or not constant proportional time trade-off (CP-TTO) holds so that the utility associated with a marginal survival in a given health state remains constant regardless of the health state or the duration. It has been argued that for very severe states there may be a ‘maximal endurable time’ limit, beyond which the marginal benefit of survival diminishes. 36 The second issue is whether or not respondents use a positive temporal discount rate when valuing hypothetical health scenarios. 37–39 The third is the impact of life stage concerns in health-state valuations. If the duration of the state is too long then the scenarios will not be credible for older respondents and vice versa. Furthermore, depending on the duration, people may be thinking about life stage events rather than about the trade-off between longevity and quality of life.

The final issue is whether or not 10 years is the most relevant duration for NICE decision-making. If the issues highlighted above mean that the value of a state is a function of its duration then the revaluation of the EQ-5D may not be based on scenarios with a 10-year duration. The PRET stage 1 online survey examines the impact of varying duration on health-state preferences.

PRET stage 1

The aim of the PRET stage 1 online survey was to test a range of methodological assumptions and questions related to health-state valuation, using health states based on EQ-5D-5L. This was done using binary choice questions based on TTO and DCE. The questions tested were:

1. whether health-state preferences are independent of duration (see Chapter 3 )

2. whether health-state preferences are independent of person perspective used (see Chapter 4 )

3. investigation of LT-TTO (see Chapter 5 )

   1. whether health-state preferences are independent of lead time

   2. to what extent respondents exhaust lead time under very poor health

4. whether the preferences of others’ health is independent of when health events take place (see Chapter 6 )

5. whether health-state preferences are independent of satisfaction in the state (see Chapter 7 )

6. whether DCE_TTO is feasible for EQ-5D-5L, and if so which states should be valued (see Chapter 8 ).

Issues 1–5 are methodological, and therefore the questions used to investigate this are not designed to produce utility values anchored on the full health–dead scale for EQ-5D-5L. Issue 6 uses a method that is designed to elicit utility values.

PRET-AS

The aim of PRET-AS was to investigate the feasibility of two binary choice question types that can be used to derive utility values anchored on the full health–dead utility scale. This was done by conducting an additional online survey of similar size to that in stage 1.

1. The first question type in PRET-AS was also used in the PRET stage 1 survey to investigate issue (6) and presents a DCE with an associated duration level using EQ-5D-5L health states. The results are presented alongside the findings from PRET stage 1 (see Chapter 8 ).

2. The second question type presented a binary choice version of LT-TTO using whole three-level EQ-5D health states. The results are presented alongside the findings from the PRET stage 1 LT-TTO investigation (see Chapter 5 ).

The methods used for the surveys are briefly described in the next six sections, and the study design to test each methodological question is described in detail in the relevant chapter.

PRET

The seven different question types were presented across three experimental modules:

• Module 1 Five type I questions.

• Module 2 Five questions specific to the questionnaire version (using question types II–VI).

• Module 3 Two type VII questions.

Each respondent completed 12 binary choice questions and there were 15 versions of the online survey overall ( Table 2 describes the question types included in each survey). The ordering of the questions within each module was randomised. For 14 of the versions, module 2 consisted of five binary choice questions from one of types II, III, IV, V or VI. Therefore, respondents who completed these versions faced three question types each. However, for version 15, module 2 included one question each of types II, III, IV, V or VI. Therefore, respondents allocated to version 15 completed all seven question types. This was done so that we could compare the results for all question types across different modes of administration at stage 2 of the project (see Chapter 4 ).

Furthermore, there were 60 subversions of the survey (each of the 15 versions has four subversions) for module 3. This enabled 120 DCE_TTO pairs to be allocated across the 60 subversions (i.e. two per subversion).

PRET-AS

The PRET-AS respondents completed either 15 type VII questions across three experimental modules of five questions or 10 type VIII questions across two experimental modules of five questions.

PRET

Respondents were recruited from an existing commercial internet panel. Overall, approximately 3000 respondents were recruited into stage 1 of PRET across the 60 subversions of the online survey, with each version completed by a minimum sample size of 50. Respondents were sourced from an existing internet panel following set quotas for age across five age groups (18–24, 25–34, 35–44, 45–54, and 55–65 years, although a handful of respondents reported that they were older than 65 years) and gender, in an attempt to recruit a sample that was representative of the UK general population in this age range. To recruit participants, invitations were sent out by e-mail. Respondents were screened out prior to starting the experimental questions if the relevant quota for age and gender was complete, or after completing the survey if they answered all of the survey questions in less than the minimum imposed time limit of 5 minutes. Those who successfully completed the survey received online points worth approximately £1.

The same recruitment procedures that were used for the PRET online survey was followed for PRET-AS, with approximately 1800 respondents across the 36 type VII question surveys and 1200 across the 27 type VIII surveys. We reduced the minimum completion times so that respondents were classified as non-completers if they completed the survey in < 3 minutes (and the time to complete the overall survey and each experimental question module was recorded).

Respondents entering the survey firstly completed the same demographic and self-reported health questions. They were then presented with information about the tasks. This included details about the EQ-5D-5L health dimensions, and instructions to imagine that they would experience each health state for the period shown without relief or treatment, that death would be very swift and completely painless, and that they would have no other health problems besides what was indicated. A practice task was then completed, followed by the valuation questions.

PRET

Overall, 34,892 panel members were invited to take part in the PRET stage 1 online survey, and 7750 (22.2%) clicked the link to access the survey. Of those who entered the survey, 668 (8.6%) did not start the questions, 2158 (27.8%) dropped out during the survey, 1765 (22.8%) either completed the survey in less than the minimum time of 5 minutes or did not click the final page so were classified as non-completers, and 3159 (40.8%) fully completed the survey. The available demographics for responder and non-responder samples overall and across the VII question types are reported in Table 3 .

PRET-AS type VII

Overall, 5552 respondents were invited to take part, and 4513 (81%) respondents accessed the survey. Of these, 1183 (26% of those accessing the survey) were turned away because their quota was full, leaving 3330 (74%) to enter the survey. Of these, 1020 (31%) dropped out before reaching the DCE_TTO questions. Of the remaining 2310 who entered the DCE_TTO questions, 23, 50 and 33 dropped out during the first, second, and third modules, respectively. A further nine completed all of the DCE_TTO questions but failed to formally sign out from the survey and to be counted. Finally, 396 respondents (17% of those who started the DCE_TTO questions) were excluded because they completed the survey in less than the minimum time limit of 3 minutes. Therefore, 1799 respondents (40% of those accessing the survey) fully completed the whole survey in > 3 minutes. This amounts to 40% of those accessed the survey, 54% of those who entered and 78% of those who started the DCE_TTO questions. The available demographics for responder and non-responder samples are reported in Table 4 .

PRET-AS type VIII

Overall, 4696 respondents were invited, and 3570 (76.0%) accessed the survey. Of those who accessed the survey, 1035 (29.0%) did not start the questions, 658 (18.4%) dropped out during the LT-TTO survey, 677 (19.0%) either fully completed the survey in < 3 minutes or completed but did not click the final link so were classified as non-completers, and 1200 (33.6%) fully completed the survey in > 3 minutes. The available demographics for responder and non-responder samples are reported in Table 5 .

Question format and study design

The type I binary choice questions used the following format (and an example of how the question was presented in the survey is shown in Appendix 2 ).

1. [Scenario A]: You will live in health state H for T years and then die.

2. [Scenario B]: You will live in full health for (VT) years and then die.

3. Which scenario do you think is better?

The health state H was a ‘CS’ and used the following five health dimensions adapted from EQ-5D-5L health dimensions. CSs were used so that variation could be linked to a single dimension, and also to make the health scenarios easy to imagine. Respondents were instructed to assume that they have no other health problems other than those indicated in the scenario.

• ‘Slight problems walking about’ (level 2 of the mobility dimension from EQ-5D-5L state 21111).

• ‘Slight pain’ (a segment of level 2 of the pain dimension using pain only from EQ-5D-5L state 11121).

• ‘Unable to walk about’ (level 5 of the mobility dimension from EQ-5D-5L state 51111).

• ‘Extreme pain’ (a segment of level 5 of the pain/discomfort dimension using pain only from EQ-5D-5L state 11151).

• ‘Extreme depression’ (a segment of level 5 of the anxiety/depression using depression only from EQ-5D-5L state 11115).

Duration T took one of four values: 10 weeks, 1 year, 5 years and 10 years. The values were chosen as follows: 10 years for comparability with the ‘standard’ MVH TTO protocol, 5 and 1 years as intermediate whole-year values, and 10 weeks to test the maximum endurable time of the more severe CSs.

Two sets of V values were used: 0.8 and 0.9 for the two states using level 2 (i.e. slight), and 0.4 and 0.6 for the three states using level 5 (unable/extreme). V values of 0.4 and 0.8 are described as ‘V(low)’, and 0.6 and 0.9 as ‘V(high)’.

All 15 versions of the online survey included five type I questions as the first module presented to respondents, meaning that there were 75 ‘slots’ for this question type overall. Combining the five health states H, four dimension levels T, and two values for V used for type I questions resulted in a total of 40 possible combinations. Therefore, 35 of the combinations appeared twice in different versions of the online survey, with five (one for each health state) appearing once. The allocation of the question combinations across the different survey versions are displayed in Appendix 3 .

In addition, each respondent was given a question similar to a type I question, but tests for logical consistency. In this question, scenario A was dominated by scenario B: scenario A was to live for a shorter duration in worse health and scenario B was to live for a longer duration in full health. Thus, the logical answer is to choose B. If respondents were choosing randomly between A and B then around half of them would choose A. In other words, double the proportion of those choosing A for the logical consistency test question may be interpreted to represent the proportion of respondents who were not fully engaged.

Analysis

For question type I, the outcome of interest is the proportion of respondents selecting scenario B, which means preferring less time in full health over more time in worse health, and thus represents the proportion of respondents for whom the value of V* of state H is lower than the value of V used in the scenario pair. The proportions of those choosing scenario B were analysed across the different scenario attribute combinations and background characteristics. For type I questions, the proportion of respondents who violated logical dominance was also assessed.

The findings were tested for the overall sample, and also by splitting the sample into two groups based on the median time taken to complete question module 1 (which included five type I questions). Group 1 included those who completed the question module in less than the median time taken to complete the module, and group 2 included those who completed the module in more than or equal to the time taken to complete the module.

Probit regression was used to explore the significant impacts on choosing scenario B across each set of scenario attributes and background characteristics. The equation used is as follows:

where Pr represents probability, the β _is are the estimated parameters, D represents the background characteristics of respondents, SWB represents self-reported satisfaction levels (SWB _H and SWB _L), X represents the properties of the health state using health state (H), duration (T), lead time in full health (L), person perspective (P), and satisfaction level (S), and the function Φ(.) is the distribution function of the standard normal distribution. Marginal effects are reported where, for example, a marginal effect of −0.1 for female indicates that being female reduces the probability of choosing scenario B by 10%. Statistical significance levels of both < 0.05 and < 0.1 were used.

Demographics

As all respondents were given type I questions, the sample here consists of 3159 full survey completers who each completed five type I questions (of which one was a test of logical consistency). Therefore, the responses to the four logical questions generated 12,636 type I observations. Each combination of H, V and T was completed by either (approximately) 200 or 400 respondents. The characteristics of the sample are displayed in Table 3 .

Objective 1: descriptive analysis

Of the 3159 respondents, 200 (6%) failed the test of logical consistency (i.e. responded that they would rather live for less time in one of the five health states than a longer time in full health). Using bivariate analysis, there is a disproportionate number of males (chi-squared test; p = 0.001) – those whose education continued after minimum school leaving age (p = 0.012) and those in poorer self-reported general health (p = 0.015) – who failed the logical consistency test. Age, having a degree, and time taken were not associated with failing the logical consistency test. When assessing the proportions of respondents failing the logical consistency test across the two groups defined by the time taken to complete question module 1 described above (see Analysis), it was found that the proportion of respondents failing the test did not differ significantly between group 1 (n = 85, 5.5%) and group 2 (n = 115, 7.7%) (p = 0.06).

Based on the remaining four type I questions in module 1, Figure 3 illustrates the proportion of respondents choosing scenario B, broken down by the health-state dimension, duration of the health state, and the value used to generate the associated time in full health in scenario B. In other words, this was the proportion of respondents for whom the value used in the scenario (V) was larger than the value they perceive (V*) for the state. It should be noted that the majority of bars are > 50%, some of them as high as 90%. This suggests that the values of V used in the design of the question types may have been set lower (this is discussed in relation to all of the question types in Chapter 12 , Weaknesses of the project).

FIGURE 3.

Proportion of sample choosing scenario B (type I questions).

Within each health-state dimension, the proportion of respondents choosing B was higher when the value of V was larger. For example, the bars for ‘slight problems walking about’ with a high V value [M2(0.9)] are taller than the corresponding bars for ‘slight problems walking about’ with a lower V value [M2(0.8)] across the same duration of time spent in the health state. The exception (by a small margin) is for ‘extreme depression’ [D5(0.4)] and [D5(0.6)] with a duration of 10 weeks. Within each specific health problem (where a comparison is possible), the proportion choosing scenario B was always higher for the more severe state so that the bars for ‘unable to walk about’ (M5) are taller than corresponding bars for ‘slight problems walking about’ (M2), and the bars for ‘extreme pain’ (P5) are taller than the corresponding bars for ‘slight pain’ (P2). This demonstrates that respondents are more likely to choose the full health option when the state is severe. There does not seem to be a pattern to the proportions of respondents choosing scenario B across the four duration levels. For example, the bars for the 10-week duration scenarios tend to be taller than the corresponding bars for longer durations, but there are exceptions.

Table 6 summarises the results of a series of probit regressions explaining the propensity to choose scenario B (living in full health for a shorter period of time), without (models 1–5) and with (models 6–10) controlling for a series of covariates. As the distribution of data for own health in EQ-5D-5L is skewed, dummy variables indicating any problem in mobility, pain/discomfort or anxiety/depression were used. The models by state indicate that generally the higher the value of V, the higher the probability of choosing to live in full health for a shorter duration [although this is not significant for ‘extreme depression’ (D5)]. There were no covariates that affect the choice consistently across all states. Regarding the effect of respondents’ self-reported health in EQ-5D (models 6–10), the exercise finds that, controlling for duration and V value, having a mobility problem was associated with being less likely to choose scenario B (living for less time in full health) for the two mobility-based states (M2 and M5) and extreme pain (P5); having pain/discomfort was associated with being less likely to choose scenario B when the state is ‘slight problems in walking about’ (M2) and ‘slight pain’ (P2) but not ‘extreme pain’ (P5); and having anxiety/depression was associated with being less likely to choose scenario B when the state was ‘extreme depression’ (D5). This indicates that, to some extent, respondents who have experience of the health state they are valuing may be more likely to hypothetically associate it with a higher utility value.

All of the state dummies were significant when pooling across states ( Table 7 ; model 11 without covariates, and model 12 with covariates). It suggests that ‘slight problems walking about’ (M2) was perceived as being worse than ‘slight pain’ (P2), and ‘unable to walk about’ (M5) was perceived as worse than ‘extreme depression’ (D5), which, in turn, was worse than ‘extreme pain’ (P5). As the values are clustered by the severity groups, the state coefficients cannot be compared across the mild states and the severe states. All duration and value dummies were significant. The dummy for the V value 0.9 was omitted as there was collinearity in the design (this is because all scenarios with M2 or P2 that do not use the value of 0.8 use 0.9). Having problems on the EQ-5D-5L dimensions of Mobility, Pain/discomfort, and Anxiety/depression were all significant in the pooled model, indicating that having existing health concerns impacts on the propensity to choose full health (p < 0.001).

Objective 2: assessing assumption 1 – CP-TTO

The coefficients of interest for assessing CP-TTO are those for duration spent in the health state. The 10-year duration value is used as the reference as this is the value used in the ‘standard’ MVH TTO protocol. For the two milder states of ‘slight problems walking about’ (M2) and ‘slight pain’ (P2), the 10-week duration had a significantly positive effect relative to 10 years. However, durations of 1 year and 5 years were not significant (see Table 6 , model 1 for M2 and model 2 for P2). For the states ‘unable to walk about’ (M5, model 3) and ‘extreme pain’ (P5, model 4), none of the duration coefficients was significant. For ‘extreme depression’ (D5, model 5), the 5-year duration is significant. The same pattern of significance was found when covariates were controlled for (see Table 7 , models 6–10). In the model pooling across states, only the 10-week coefficient was significant (see Table 8 , model 11 without and model 12 with controlling for covariates). The positive coefficients indicate that the shorter duration value was associated with having higher preferences for the health state presented.

The above analysis demonstrates that whether or not CP-TTO holds depends on the dimension and severity of the state. In the case of D5, the pattern was not monotonic. It is somewhat surprising that the extreme (level 5) states, where one may have expected maximal endurable time to apply, have resulted in no significant duration coefficients. There also seems to be no pattern of interactions between the state, associated duration, V value and respondent characteristics.

Question format and study design

This analysis used type I and type II binary choice questions, which take the following format (see also Appendix 2 ).

Type I:

[Scenario A]: You will live in health state H for T years and then die.

[Scenario B]: You will live in full health for (VT) years and then die.

Which scenario do you think is better?

Type II:

[Scenario A]: [Person P] will live in health state H for T years and then die.

[Scenario B]: [Person P] will live in full health for (VT) years and then die.

Which scenario do you think is better?

The ‘corner’ health states and duration and V values used across both question types were the same as described in Chapter 3 (see Methods). For type II questions, the two perspectives used were ‘Somebody else’ (SE) and ‘Somebody else like you’ (SY). To assess the impact of perspective P, the health state H, duration T and V value combinations were matched across question types I and II, so the impact of varying only perspective could be assessed.

Question type II appeared on four survey versions, with 16 available ‘slots’ for questions across 80 possible combinations of health state H, duration T, perspective P, and V value. Eight slots were allocated to the perspective SE, with the same eight combinations allocated to SY. Four of the states across each perspective were allocated to the low V category, and four to the high V category. Two of the eight states across each perspective were allocated to each duration level. As described in Chapter 3 , type I questions appeared in all 15 versions of the survey, and the relevant state and duration combinations matched across the question types were extracted for the comparison.

Analysis

The effect of perspective was analysed using descriptive analysis, chi-squared tests and probit regression, which was used to explore the impact of different perspectives on choosing scenario B while controlling for background characteristics:

where Pr represents probability, the β _is are the estimated parameters, D represents the background characteristics of respondents, SWB represents self-reported satisfaction levels (SWB _H and SWB _L), X represents the properties of the health state using health state (H), duration (T), lead time in full health (L), person perspective (P) and satisfaction level (S), and the function Φ(.) is the distribution function of the standard normal distribution. Equation 2 is the same as Equation 1 (see Equation 1 on p. 22) and, again, marginal effects are reported. A backwards stepwise approach was taken to the selection of explanatory sociodemographic variables included in the final models using a level of statistical significance of p < 0.1. Joint tests of statistical significance were used for categorical variables expressed as sets of dummy variables.

Demographic characteristics

Overall, 829 respondents completed type II questions across four survey versions, and this sample was matched with those who completed the comparable type I questions. The characteristics of the samples are shown in Table 3 .

Descriptive analysis

Across all eight combinations between type I and II questions, 75% of respondents chose scenario B when perspective was SE, 73% chose scenario B when the perspective was phrased as SY, and 72% chose scenario B when the perspective was phrased as ‘you’. The proportions choosing scenario B for each question combination included are shown in Table 8 .

Pearson’s chi-squared tests across the three perspectives established statistically significant differences in the proportion of respondents reporting a preference for scenario B for two of the states describing extreme problems (M5 and D5). More respondents favoured less time in full health (scenario B) than the extreme mobility state (M5) when the perspective for the scenario was phrased as ‘you’ or SY compared with when it was SE. Fewer respondents presented with the perspective SY preferred less time in full health when compared with one of the D5 states (T = 5 years, V = 0.6) when compared with the SE or ‘you’ perspective.

Regression analysis

Probit models were estimated for each question combination separately ( Tables 9 and 10 ). Four of the eight models found no impact of the different perspectives on the probability of choosing scenario B. Two models established that respondents were less likely to trade off full health for P2 and extreme mobility problems (M5) when the scenario perspective was SE compared with ‘you’ (p < 0.05). For D5, respondents were more likely to choose scenario B if the perspective was SY in comparison with ‘you’ (p < 0.05, where T = 5; p < 0.1, where T = 1). There is no clear pattern to the impact of demographic variables.

Analysing all of the type I and II responses together (see Table 10 ) showed that the reduction in the likelihood of choosing scenario B when referring to SY was modest but statistically significant. The coefficient for the dummy variable for SE was not statistically significant; however, a joint test for significance approached the 5% level (p = 0.059). No statistically significant interactions were found between perspective P and duration T, state H or value V.

Question format and study design

Question types I, III and VI

The impact of lead time (types I and III), and the propensity to exhaust lead time across different L : T ratios (type VI) was analysed using descriptive analysis, chi-squared tests and probit regression, which was used to explore the impact of perspective on choosing scenario B whilst controlling for background characteristics:

where Pr represents probability, the β _is are the estimated parameters, D represents the background characteristics of respondents, SWB represents self-reported satisfaction levels (SWB _H and SWB _L), X represents the properties of the health state using health state H, duration T, lead time in full health L, and the function Φ(.) is the distribution function of the standard normal distribution. Equation 3 is the same as Equation 1 (see Equation 1 on p. 22).

Question type VIII

For question type VIII, the data were analysed through a series of logit regressions. The first model regressed the propensity to choose scenario B on the V value used, pooling across all states. Model 2 controls for state by adding state dummies, and model 3 also controls for duration. Models 4 and 5 examine whether constant proportional time trade-off and zero temporal discounting (and additive separability) hold by controlling for duration T (and lead time L). Model 6 then controls for covariates. The choice of logit here (as opposed to the probit for types I, III and VI) was an entirely practical one, arising from the need to calculate the cumulative function.

Note that as the logit density function is symmetric, the mean and the median of the distribution coincide. The median value of a state is given as the value V* that has a predicted probability 50% of selecting scenario B. Assuming:

then, as

holds, for given duration T and lead time L the mean value for state H can be obtained by identifying the value V* that has a predicted probability of 50%. Predicted values of V* are reported for each state using regression coefficients from the logit models, under key assumptions regarding levels of duration T and lead time L, where relevant.

Demographics

The number of respondents completing types I, III, VI and VIII varied from 847 (type III) to 3159 (type I). Approximately half were male and the average age was around 40 years. Tables 3 and 5 display the characteristics of each sample.

Objective 1: are health-state preferences independent of lead time?

Table 11 summarises the frequencies of respondents choosing in live in scenario B across question combinations with (type III) and without (type I) the addition of lead time. The correlation between the proportions of those choosing scenario B across the two question types was 0.93. Of the 16 matched scenarios, six resulted in statistically significant differences across question types I and III. The rows of Table 11 are ordered from the most significant to the least significant according to the chi-squared test for the null hypothesis that the proportion choosing scenario B in matched types I and III questions are the same. One may observe that severe problems tend to appear nearer the top of the table than the mild problems, or that the shortest duration or the higher ratios do not appear near the top. However, overall, there is no clear pattern across the dimension of health, the severity of health, duration or L : T ratio.

Table 12 reports the results of probit regressions where the propensity to choose scenario B is explained in terms of the key scenario parameters and select covariates. The coefficients used for duration T and value V are significant, alongside state H, when the data are pooled across states. On the other hand, lead time L is not always significant, indicating that the inclusion of lead time L cannot be said to affect the propensity to choose scenario B in a systematic way. However, long lead times and the use of depression in the health scenario seem to make the introduction of lead time significant. A range of background variables are also significant.

Objective 2: to what extent do respondents exhaust lead time under very poor health?

Figure 4 illustrates the propensity to exhaust lead time. Figure 4a shows the data by L : T ratio, where the bars are ordered from high to low propensity. As can be seen, the bars are also ordered from the lowest ratio to the highest, except for the last two bars, where the ratio 10 : 1 achieves a lower incidence of exhaustion of lead time than 25 : 1. Figure 4b breaks this down by the actual length of lead time and duration. The bars continue to be bunched by the L : T ratio, suggesting that the within-ratio variation is relatively small. Each bar in Figure 4b represents roughly 200 or 400 respondents.

FIGURE 4.

Propensity to exhaust lead time (type VI questions). Bars are ordered from high to low propensity to exhaust lead time. Propensity to choose scenario B by (a) L : T ratio and (b) lead time and duration.

Regarding the actual incidence rate of exhaustion of lead time, it is only when the L : T ratio is as high as 5 : 1 that less than half of the respondents exhaust lead time. This is in contrast with previous work carried out using iterative LT-TTO in computer-assisted face-to-face interviews, where 20–25% of respondents exhausted lead time ( Table 13 , taken from earlier LT-TTO work19). Two of the L : T combinations were common across the two studies (10 years–5 years and 5 years–1 year) and it can be seen that, in either case, the incidence of exhaustion of lead time is much higher in the online administration of binary choice LT-TTO (see Figure 4 ) than in the face-to-face administration of iterative LT-TTO ( Table 14 ).

Table 14 shows the results of regressing the propensity to choose scenario B (immediate death), and thus to exhaust lead time, on lead time L and duration T. The negative lead time coefficients indicate that, although controlling for duration, the longer the lead time the lower the incidence of exhaustion of lead time, which is as expected. The positive duration coefficients mean that, although controlling for lead time, the longer the duration the higher is the propensity to exhaust lead time, which is also expected. When background characteristics are controlled for, a number of them are significant but do not affect the sign or the magnitude of the main effects coefficients.

Is it feasible to elicit LT-TTO values using binary choice questions?

Figure 5 plots the proportion of those choosing scenario B along different values of V used, by state H: note that the horizontal axis is not continuous and the bars pool across the different combinations of duration T and lead time L. As expected, across all of the states, the proportion of those who choose scenario B increases as value of V used increases (the higher the value of V used then the longer the survival in full health in scenario B, thus the higher the proportion of those selecting scenario B over scenario A, other things the same). Except for state 33333, all of the bars start at < 50% and go beyond 50%. For state 33333, > 60% of respondents already choose scenario B when the value of V used is as low as −3.0.

FIGURE 5.

Raw plots of proportion choosing B against value V (type VIII questions). (a) Raw plots state 11211; (b) raw plots state 22121; (c) raw plots state 23232; (d) raw plots state 32211; and (e) raw plots state 33333.

Table 15 summarises the regression results. All of the coefficients are highly significant. The propensity to choose scenario B is, as expected, a function of the value of V used (treated as continuous). The coefficients of the state dummies in model 2 onwards represent the severity of each state relative to 11211. Their magnitude is very stable across different specifications. Their positive sign indicates that the reference state 11211 is the best state. They also consistently imply that state 23232 is worse than state 32211. The regression results above from the type III data would suggest controlling the regressions for duration T and lead time L. In models 3–5, the duration coefficients are significant and negative, which is consistent with both positive time preference and maximal endurable time. When state-by-duration interactions are added to model 3, none of these is significant (results not shown), which may suggest that the negative coefficient for duration is not caused by maximal endurable time. However, when the same interactions are added to model 4, the coefficients for the worst two states are significant (for state 33333 at 1% and for state 23232 at 5%). The lead time coefficients in models 4 and 5 are significantly positive, indicating that the addition of lead time results in lower (non-discounted) values. Although a number of covariates are found to be significant in model 6, their inclusion does not affect the magnitude of the coefficients for the main effects.

Table 16 reports the results of running models 2 and 4 by state H, in order to obtain predicted medians. Models by states are used because the pooled models in Table 16 do not give an intercept for the reference state. In Table 16 , each intercept represents the severity of the state (the worse the state, the smaller the intercept), and has the same relative ordering across states observed in Table 13 . The coefficients in Table 13 are used to calculate the value of V* that will result in a predicted propensity to choose scenario B of 50%, i.e. the median (and the mean) health-state value. Table 17 reports the predicted median values for each state. Model 2 does not have any parameter assumptions, but model 4 requires parameters on duration T and lead time L to be set exogenously. No scaling has been used. As can be seen, the values for the two mild states are very stable, whereas the same cannot be said of the remaining three states: they are affected widely by the model and by the parameter assumptions.

Figure 6 plots the predicted probabilities of choosing scenario B at different values of V used in the range [−3 to 1]. The relative ordering of two of the states will depend on the value of V used.

FIGURE 6.

Predicted probabilities of choosing scenario B at different values of V used in the range [−3 to 1] by state (type VIII questions).

Question format and study design

The three binary choice questions have the following format (see also Appendix 2 ):

Type I:

1. [Scenario A]: You will live in health state H for T years and then die.

2. [Scenario B]: You will live in full health for (VT) years and then die.

3. Which scenario do you think is better?

Type II:

1. [Scenario A]: [Person P] will live in health state H for T years and then die.

2. [Scenario B]: [Person P] will live in full health for (VT) years and then die.

3. Which scenario do you think is better?

Type IV:

1. [Scenario A]: [Person P] will live in full health for L followed by health state H for T years and then die.

2. [Scenario B]: [Person P] will live in full health for (L + VT) years and then die.

3. Which scenario do you think is better?

The same CSs H, person perspectives P, duration T and lead time L values described in Chapters 3 – 5 were used. The combinations of person perspective, health state and duration were matched across question types II and IV so that the impact of the addition of lead time could be assessed in terms of time preference.

All 15 versions of the online survey included five type I questions. Question type II appeared on four survey versions, with 16 available ‘slots’ for questions across 80 possible combinations of health state H, duration T, perspective P and V value. Eight slots were allocated to the perspective SE, with the same eight combinations allocated to SY. Four of the states across each perspective were allocated to the low V category, and four to the high V category. Two of the eight states across each perspective were allocated to each duration level. There were also 16 available ‘slots’ for type IV questions across the 320 possible scenario combinations. Each lead time L, health state H, V value and duration T combination used was matched across each perspective P. L : T ratios of 1 : 1 were included for each duration level. Eight of the states used the low values for V, and eight used the high V value. Appendix 3 details the full allocation of attributes across the possible combinations.

Analysis

By making a number of assumptions the responses to question types I, II and IV can be used to derive information regarding the respondents’ time preference rates. One important assumption made in what follows is that the respondents’ time preferences are exponential rather than, say, hyperbolic. Notation is as introduced previously, with the addition that two values of V are distinguished in order to allow the value of the health state H (V ₁) to differ from the approximate health-state value (V*) used to determine the time in full health offered in the choice. The conceptual representation of question types I, II and IV are displayed in Figure 7 . For all question types, the indifference on the part of the respondent between option A and option B implies that:

FIGURE 7.

Representation of question types I, II and IV for time preference analysis.

Given that V* and T are specified in the question, the value of r which solves Equation 6 is readily identified if V ₁ is known or can be assumed. In the current case, V ₁ for each individual is not known and thus it is necessary to assume a value. In the analysis that follows, V ₁ is set equal to the DCE_TTO values generated in PRET-AS (see Chapter 8 ) that best describe the health state in the particular question.

Respondents choose option A or option B, and thus each choice they make implies that their time preference rate is above or below the value of r that solves the equation for that particular choice. More specifically, the proportion of respondents choosing B in response to a particular question (given the assumptions made above) indicates the proportion of respondents with a time preference rate above the equation-solving value of r.

Demographics

The number of respondents completing types I, II and IV are 3159, 829 and 849, respectively. The demographic characteristics of the sample are reported in Table 3 .

Time preference analysis

The results for the type I questions (framed with a ‘you’ perspective) are reported in Table 18 . The value of r which would be implied by indifference between options A and B is generally positive (with few exceptions). This is simply a consequence of the choices offered and the health-state values assumed. The numbers presenting information regarding time preferences are the per cent of respondents choosing option B when confronted by a particular choice. For example, consider the choice between 10 years with the D5 state and four years in full health, given the assumed health-state value (0.522) the 64.4% of respondents choosing full health have an implied discount rate > 0.10.

The data in Table 18 are slightly difficult to interpret as they comprise the percentage choosing B at different threshold values for r. If the time preference rates implied by the respondents’ choices belonged to the same distribution then we would expect that the lower the threshold value of r, the higher would be the percentage choosing B. But this is not observed. The main reason for this is that the implied discount rates are not independent of duration.

When the proportion choosing option B are considered for different durations of time in particular health states, there appears to be a tendency for the underlying distribution of time preferences to shift leftwards as duration increases, at least in the case of P5 and M5. In other words, the percentage choosing option B tends to hold up as the equilibrating rate falls with increasing duration. The overall picture is mixed, for example, in the case of P2, there is a tendency for the percentage choosing option B to fall slightly as duration increases.

The results for the lead-time choices reported in Table 19 also highlight that the implied time preference rates are influenced by time period over which discounting takes place. Consider the health state P5 (assumed value of 0.504), for which 90.2% of respondents have an implied discount rate of > 4.41 over 10 weeks and 93.4% have an implied discount rate of > 0.09 over 10 years. Similarly, for slight pain over 10 weeks 69.9% had an r > 5.28, whereas over 10 years 59.5% had an r > 0.23. Again these observations are consistent with leftward shifts of the distribution of time preferences as duration increases.

Little can be said about the extent of negative time preferences because these are group data rather than individual data. There are, however, eight occasions when the equilibrating value of r is negative. The relatively low percentage choosing scenario A are indicating they have negative time preferences. As with the choices when the equilibrating rate was positive, there is clear evidence that time preferences are not independent of duration. For example, for choices concerning extreme pain, as duration increases and the equilibrating rate rises (−4.07, −0.64, −0.16, −0.08) the percentage choosing scenario B increases slightly. If time preferences were independent of duration one would expect the proportion choosing B to fall. The responses for the choices with a negative equilibrating r are consistent with a rightward shift in the distribution of implied discount rates as duration increases.

The impact of making an alternative assumption regarding the health-state value to apply to time spent in health state H is illustrated in Table 20 . Using the example of the health-state M5, the value of 0.698 (based on the values generated in PRET-AS) is replaced by 0.648 and 0.748. Higher (lower) health-state values increase (decrease) the value of r consistent with indifference between options A and B. The effect is greatest when considering shorter associated durations.

Table 21 compares the implied time preference rates by perspective. A higher proportion of respondents tended to choose option B when the perspective was SE rather than SY (but note the exception of the M5 state). The proportion choosing option B also tends to be higher for the ‘you’ rather SY perspective (but note there are two exceptions). However, the differences in proportions are not great and it would not be appropriate to suggest on these data that implied time preference rates are higher for SE compared to SY, or for ‘you’ compared to SY. A comparison of the SE and SY perspectives is also possible with the lead time choices and these data show no tendency for the proportion choosing B to be higher given a particular perspective.

Question format and study design

The binary choice questions took the following format (see also Appendix 2 ):

Type I:

1. [Scenario A]: You will live in health state H for T years and then die.

2. [Scenario B]: You will live in full health for (VT) years and then die.

3. Which scenario do you think is better?

Type V:

1. [Scenario A]: You will live in health state H for T years with satisfaction S and then die.

2. [Scenario B]: You will live in full health for (VT) years with satisfaction S and then die.

3. Which scenario do you think is better?

The health state H, duration T and value V were matched across both question types so that the impact of the addition of satisfaction could be assessed. The durations were fixed at 5 years for scenario A and 3 years for scenario B. Three of the CSs were used: M5, P5 and D5. These are combined with one of four satisfaction states: ‘high life satisfaction’ (High LS), ‘high health satisfaction’ (High HS), ‘low health satisfaction’ (Low HS), and a situation described as ‘learnt to live’ with the health condition (LL) – ‘high/low’ were chosen as there is no ambiguity in which of the two is better, and ‘learnt to live’ illustrates an adaptation to the state.

Type V questions appeared in three versions of the online survey, meaning 11 question ‘slots’ in total. Five slots were allocated to scenarios investigating HS, two slots to LS, and four slots to LL. Type I questions appeared on all 15 versions of the online survey, and the matched questions (in terms of health state and duration) from across the surveys were used in this analysis.

Analysis

The probability of respondent i choosing to live for 3 years in full health (scenario B) was estimated using probit regression:

where H × S denotes the combination of a health state H with a level of satisfaction S under scenario A. The subscript j represents the number of multiattribute health states. DEMO represents a set of demographic variables available for the respondent: these are gender, age, age squared, marital status, employment status and education level. HEALTH is a set of 15 dummy variables capturing respondents’ own state of health obtained from their completion of the EQ-5D-5L. 5 SWB represents a set of self-reported health or life satisfaction variables, each on a 0–10 scale, with ‘0’ denoting ‘not at all satisfied’ and ‘10’ denoting ‘completely satisfied’. The correlation between health and life satisfaction was 0.67, which may lead in multicollinearity. Therefore, the results control only for life satisfaction but the overall pattern of the results does not alter if health satisfaction is controlled for.

Equation 7 is estimated by pooling together all responses of the scenarios. As respondents in versions 1 and 2 of the survey face multiple scenarios of health state–satisfaction combinations, we estimate robust standard errors clustered at the individual level. Type I questions act as the reference category for each dimension to assess any change in preferences attributed to the introduction of satisfaction levels.

Demographics

Overall, 645 respondents completed type V questions, 830 completed the matched type I questions, and the demographics of the sample are reported in Table 3 . Approximately 73%, 87% and 71% of the respondents did not have any problems with mobility, self-care, and performing usual activities, respectively, as measured by EQ-5D-5L. The corresponding proportions for pain/discomfort and anxiety/depression are about 45% and 56%, respectively. Average SWB_H and SWB_L of the sample were 6.4 and 6.3, respectively. The proportion of those reporting the best state in EQ-5D-5L (i.e. 11111) is 31.6%. Average own health satisfaction (SWB_H) and life satisfaction (SWB_L) are 6.4 and 6.3, respectively.

Impact of satisfaction

Pooling all 2128 responses, the proportion of respondents choosing the full-health scenario B is 70.1%. There is a significant difference between the proportion of respondents choosing scenario B between the low and high satisfaction groups for both M5 and P5 ( Table 22 ). The difference between High HS and LL for D5 is not significant.

Table 23 reports marginal effects coefficients regressing experimental variables and demographic characteristics on to the propensity to choose scenario B (3 years in full health). The probability of choosing scenario B in reference to the base category (D5 and LL) with the health state increased significantly when the state was ‘M5 and Low HS’ (about 17%) and when it was ‘P5 and Low HS’ (about 20%). The remaining statistically significant states were associated with the health state M5; the High HS, High LS or LL descriptors significantly reduced the probability of choosing full health by approximately 17%, 19%, and 14%, respectively.

Demographic variables were not statistically significant, except for respondents who were homemakers. Moderate pain/discomfort (level 3) and moderate to severe anxiety/depression (levels 3 and 4) on EQ-5D-5L also reduced the probability of choosing full health, and respondent’s SWB_L level was not significant.

We also compared different combinations of states. Consider the scenario of M5 and Low HS, with a marginal effect of 0.168, and P5 and Low HS, with a marginal effect of 0.203. As the satisfaction level in both of these states was the same, any difference in marginal effects was arguably due to the health state. Therefore, P5 in comparison with M5 increased the likelihood of choosing 3 years in full health by 3.5% (= 0.203 − 0.168). A Wald test rejects the null hypothesis that these two coefficients are equal (χ² ₍₁₎ = 38.24, Prob = 0.000).

We also assessed matched health states that differ by satisfaction level. The scenario ‘M5 and Low HS’ had a marginal effect of 0.168, and ‘M5 and High HS’ had a marginal effect of −0.162. As the health state was the same, any difference may be due to the difference in the levels of satisfaction. Thus, facing ‘Low HS’ compared with ‘High HS’ increased the likelihood of choosing 3 years in full health by 33% (= 0.168 + 0.162).

Table 24 presents the marginal coefficients of estimations grouping scenarios based on EQ-5D dimensions. This compares type I and type V PRET questions [i.e. comparing health-state scenarios without any information on LS or HS included (type I) to comparable health-state scenarios but which contain information on satisfaction in the state (type V)]. For the health state M5, the probability of choosing full health significantly increased by 23% when associated with Low HS. However, when this state was combined with either High HS or LS, the probability of choosing the full health scenario decreased by about 12% and 14%, respectively. This indicates that respondents would prefer to cope with the given health dimension for a longer time period when they were more satisfied with their health or life. Learning to live with the condition as a proxy for adaptation had no statistically significant effect on preferences. For the health state P5, the addition of Low HS did not have a significant effect on preferences. In line with the mobility dimension (M5), the presence of High HS or LS reduced the probability of choosing the full health scenario by 29% and 32%, respectively. In contrast with M5, the addition of LL also reduced the probability of choosing the full health scenario by approximately 32%. For the health state D5, the effect of High HS and LL were similar to the corresponding effects for the M5 and P5 dimensions.

Table 25 demonstrates that the relative ordering between High LS and High HS was consistent, but the relative ordering of LL was not. This suggests that the meaning and importance of learning to live with a health condition depends heavily on the state. Across the dimensions, respondents’ SWB_L did not have a statistically significant impact on preferences and there was no pattern to the impact of demographic variables.

The impact of the dimensions of the self-reported EQ-5D-5L on TTO preferences was also assessed. Respondents faced with the P5 and D5 tended to opt for scenario B if they self report extreme problems with mobility (level 5) and slight problems performing usual activities (level 2), respectively. Respondents who reported problems with pain and depression preferred living longer when facing the P5 and D5 scenarios.

Respondents facing ‘High HS’, ‘High LS’ or LL were significantly more likely to prefer the ‘poor’ health state (i.e. choosing scenario A) than those faced with Low HS. Therefore, we could also investigate preferences between health states for a given level of satisfaction in that state. Marginal effects probit coefficients based on the grouping of health states according to the associated satisfaction level are presented in Table 25 (with mobility as the reference category). For Low HS the health dimension included had no impact on the likelihood of choosing scenario A or B. This differs to the pooled model (see Table 23 ), in which a significant difference was found between ‘M5 and Low HS’ and ‘P5 and Low HS’.

With High HS, respondents facing either P5 or D5 in reference to M5 were more likely to prefer scenario B (full health). The difference in impact between P5 and D5 was not statistically significant (χ² ₍₁₎ = 0.54, Prob = 0.464). For scenarios including High LS, respondents were more likely to choose scenario B for P5 than M5. There was also some evidence that respondents with a medium level of SWB_L had a stronger preference for choosing scenario A (living in the health state). The results for LL can be interpreted in a similar way, as extreme depression tended to have a stronger impact than extreme pain but the difference was not significant (χ² ₍₁₎ = 0.03, Prob = 0.852). When M5 and P5 are accompanied by Low HS (column 1), there was no significant difference between them. However, when these states were accompanied by High HS, High LS, or LL (columns 2–4) there was a significant difference, suggesting an interaction between the health state and the satisfaction level.

No demographic variables were significant across all four models. The age effect observed in Table 24 appears for only the Low HS model. Different employment status categories were significant across the different satisfaction levels. The demographic and own health variables suggest that high levels of health and life satisfaction were perceived differently by respondents.

Question format

Type VII questions are based on the DCE_TTO design developed by Bansback and colleagues. 22 DCE_TTO questions present a whole EQ-5D-5L health state H with an associated attribute for duration T followed by death and take the following form (see also Appendix 2 ):

1. [Scenario A]: You live in EQ-5D-5L state H _A for duration T _A then die.

2. [Scenario B]: You live in EQ-5D-5L state H _B for duration T _B then die.

The DCE_TTO scenarios used in each pair consist of ‘you’ living in a particular EQ-5D-5L state for one of three-levels of duration T (where T = 1, 5 or 10 years) followed by death (see Figure 5 ). In contrast with question types I–V, whole EQ-5D-5L health states were used. We used a duration of 10 years to be commensurate with the standard time frame used for the MVH TTO protocol, 1 year was selected as it was also included in the development study (and it is the lowest possible whole year value), and 5 years was selected as an intermediate value. Respondents were asked which health scenario they think is better.

Type VII questions were used to first investigate the selection of health scenario pairs that could be used in a full DCE_TTO study, and, second, to investigate the feasibility of the DCE_TTO method with a larger health-state classification system. This was done by modelling the results to produce coefficients for each level of each EQ-5D-5L dimension, and comparing this with the results of the DCE_TTO development study. 22

The EQ-5D-5L has 3125 possible health states. Combining this with a duration attribute with three levels amounts to 9375 possible DCE_TTO scenarios. This means that 87.9 million DCE_TTO scenario pairs could be produced. The number of parameters for DCE_TTO of EQ-5D-5L with three duration levels is 62 [EQ-5D-5L main effects 5 × (5 − 1) = 20; duration main effects 3 − 1 = 2; and interactions 20 × 2 = 40]. This is the minimum number of pairs required to estimate coefficients for all of the parameters in the model. Confidence in the parameter estimates is improved with more pairs, and we therefore selected 120 based on the D-efficiency criterion using the modified Fedorov algorithm. 49,50 We produced 10 different designs based on different random starting points from the full factorial design, and assessed the resultant designs for efficiency level, and the number of pairs for which duration differed. We did not restrict the design to exclude potentially implausible states, as it is not clear what the criteria for implausible states in EQ-5D-5L should be. All 10 designs had similar efficiency levels, and, owing to the nature of the design algorithm included, potentially implausible dimension combinations. However, as it is difficult to establish sufficient evidence for a state to be determined implausible, it was decided not to restrict any design at this stage, and investigate the issue further during the qualitative phases of the study (see Chapters 10 and 11 ). For one design, duration differed between the scenarios on 18 (15%) of the pairs, which was higher than the other nine designs. Therefore, this design was used in PRET.

Study design

DCE_TTO – analysis

Objectives 1 (PRET) and 2 (PRET-AS) To determine the coefficients for the DCE_TTO a conditional logit model was used as described by Bansback and colleagues. 22 Briefly, the utility function μ of each respondent i is defined to be a multiplicative between a vector of levels for each EQ-5D attribute x and life-years t in each scenario j so that:

Of these, the constant α can be included to examine level balance, but is expected to be equal to zero; β represents the value of living in full health for the specified duration and is expected to be positive; λ represents the disutility of living with the specified set of EQ-5D-5L health problems for the same duration and thus is expected to be negative; and ε _ij is a random term which is assumed to be the independent and identically distributed extreme value. Duration is treated as continuous and conditional logit regression is used to estimate the coefficients, controlling for clustering of responses among respondents.

Bansback and colleagues22 show that the value for each health state anchored on the health utility scale (V) can be calculated from the estimated coefficients using the following formula:

Thus, the value of a health state is expressed in two arguments: the value of full health and the disutility determined by EQ-5D-5L. For the state of full health, λ ^ = 0 and so V = 1. If the absolute value of λ ^ is equivalent to β ^ then λ ^ / β ^ = 1 and V = 0. If the state is severe then the absolute value of λ ^ may exceed β ^ or in other words, the magnitude of the disutility associated with the state may be larger than the difference between full health and being dead. If so, this would result in a negative V, implying a state worse than being dead.

Note that the anchoring of the utility function for dead at 0 is achieved through the relative size of the two regression coefficients β and λ in Equation 8 above, and does not rely on the inclusion of the state of being dead in the DCE_TTO, or as a supplementary question. The anchoring of the utility function for full health at the value 1 is achieved through equation (4): as λ = 0 for full health, Equation 9 anchors full health at whatever value given in the first argument.

Three additional analyses were carried out to assess whether respondents trade time when presented with pairs where duration differs. First, time trading behaviour was explored by examining the frequencies of respondents who were willing to trade time when presented with a pair where duration differed between the scenarios. The proportions of respondents choosing the shorter duration was investigated irrespective of the EQ-5D-5L state presented. Second, the proportions of respondents who sometimes chose the longer and sometimes chose the shorter duration was examined, where the pairs presented allowed us to investigate this. Third, trading behaviour was assessed in relation to the utility value associated with each health scenario included in the pairs where duration differs.

Objective 3 To compare the results of PRET-AS data with the DCE_TTO development, and PRET studies, the size of the coefficients within each dimension used to generate utility values were compared. This is because a direct comparison across all three studies was not possible, as PRET and PRET-AS use EQ-5D-5L health states, and the DCE_TTO development and MVH studies3,22 used the three-level EQ-5D. The three DCE_TTO studies were compared graphically.

The same comparison was not possible for the MVH study coefficients owing to the use of the constant and the N3 term (a coefficient that is used when any EQ-5D dimension is at level 3) in the generation of utility values (i.e. the worst level). Therefore, to provide a broad overview of the potential feasibility of the values produced we compared the predicted values with the three-level EQ-5D tariff produced by the MVH study3 for the states where there is some level of comparability. The MVH TTO tariff is not used as a ‘gold standard’, but rather as a comparison tariff for the PRET-AS model. The comparison should not be interpreted to mean that DCE_TTO should reproduce, for EQ-5D-5L, the same range and distribution of values as the three-level EQ-5D. We matched level three of the EQ-5D-5L dimensions (i.e. moderate problems) with level 2 of the three-level EQ-5D (equivalent to some/moderate problems). Level 5 of EQ-5D-5L (extreme/unable) was matched with level 3 of the three-level instrument (with the caveat that the wording for the worst level of the mobility domain has changed significantly from ‘confined to bed’ to ‘unable to walk about’). We assessed the mean absolute difference between the predicted values, the relationship between the predictions across the utility scale, and also the comparability of states valued as worse than dead across the two models.

Objective 4 To explore the extent of agreement between individual ordinal preferences and aggregate cardinal values, first, the difference in the value of the health scenario in QALYs was calculated across the 120 health scenario pairs to represent the aggregate cardinal values. The value for each scenario was based on the predicted value of the EQ-5D-5L state multiplied by the specified duration. The differences in QALYs across scenario pairs were then compared with the proportion of respondents choosing each scenario. If the majority chooses the health-state scenario with the lower predicted QALYs then this would indicate a ‘disagreement’ between individual ordinal preferences and aggregate cardinal values.

Objective 5 We used two approaches to examine the existence of learning and fatigue effects. First, the predicted values obtained from models estimated on the module 1(All), module 2(All) and module 3(All) subsamples were compared against each other, and against the predicted values for the whole sample. As these batches represent the first, second and final modules that respondents answered, a divergence in the predictions can be interpreted as evidence of learning and fatigue effects. Second, along the lines of Swait and Louviere,51 we estimated a model on the full sample in which the scale of the error term is allowed to vary by batch. The scale of the first batch is normalised to one for identification purposes. As the scale is inversely proportional to the error variance, an increase in scale towards the end of the choice sequence can be interpreted as a learning effect, and vice versa. Furthermore, the LR statistic (see Equation 10) can be used to test the null hypothesis that the respondents’ preferences are stable throughout the choice sequence.

Here LL _R is the log-likelihood of the model estimated on the full sample, which allows for scale differences but assumes that α, β and λ do not vary by batch. This restricted model is estimated using the Stata module clogithet (StataCorp LP, College Station, TX, USA). 52,53 LL _U is the sum of the log-likelihoods of the three models estimated on the batch-specific subsamples. Together, these form the unrestricted model, which allows for variations in both scale and preferences by batch. Under the null hypothesis, the test statistic is chi-squared, distributed with 40 degrees of freedom. The number of degrees of freedom is given by the number of parameters in the unrestricted model minus the number of parameters in the restricted model.

Objective 6 To compare allocating the same number of DCE_TTO questions across different numbers of respondents holding the design constant, module 1(All), the two module 12 batches and the three module 123 batches are used. And, finally, to examine the effect of sample size holding the design constant, batches module 1(All), module 12(All) and All are used.

Demographics

All 3159 PRET online survey respondents completed type VII questions. The demographic characteristics are displayed in Table 3 .

Demographics

Overall, 1799 respondents completed the survey (see Table 4 ). The background characteristics do not differ across batches or versions (not shown). The DCE_TTO pairs were presented in blocks of five across 36 survey versions. The number of respondents completing each of the 36 survey versions ranged from 43 to 52. The number of observations for each of the 120 pairs ranged from 145 to 309 (as a number of pairs were repeated in more than one block).

Objective 2: feasibility of DCE_TTO with EQ-5D-5L using PRET-AS

DCE_TTO coefficients

Table 29 reports the unanchored DCE_TTO regression coefficients. The coefficients reported are based on a model with no intercept. The model with an intercept results in a small but significantly positive intercept. The other coefficients change slightly but they only have a negligible effect on the anchored coefficients. The positive intercept suggests that there is a bias towards selecting the scenario presented on the left-hand side. The results of the model with the intercept are available on request. The coefficient for Mobility level 2 interacted with duration (M2 × T) did not have the expected sign but was not significant. All other coefficients were ordered as expected. The difference between three sets of dimension levels (out of 20) was not significant: Mobility levels 1/2; Self-care levels 4/5; Usual activities levels 4/5; and Anxiety/depression levels 4/5. Figure 9 displays the distribution of the predicted utility scores for all 3125 EQ-5D-5L health states produced from the anchored coefficients. The value predicted for the worst EQ-5D-5L state (55555) was −0.845, and 31.5% of the 3125 EQ-5D-5L health states had a negative value (i.e. are worse than dead).

TABLE 29 - PRET-AS DCETTO coefficients overall

AD, Anxiety depression; MO, Mobility; PD, Pain/discomfort; SC, Self-care; UA, Usual activities.

*p < 0.1; **p < 0.05; ***p < 0.001.

Dimension × duration parameters	Whole sample		Unrestricted model						Restricted model
	ALL		Module 1		Module 2		Module 3		ALL
No. of DCE	15		5		5		5		15
M2 × T	0.006		0.014		0.001		0.002		0.006
M3 × T	−0.021	***	−0.008		−0.028	***	−0.027	***	−0.022	***
M4 × T	−0.096	***	−0.086	***	−0.095	***	−0.107	***	−0.102	***
M5 × T	−0.116	***	−0.125	***	−0.117	***	−0.110	***	−0.125	***
SC2 × T	−0.004		0.002		0.002		−0.016	*	−0.004
SC3 × T	−0.022	***	−0.024	**	−0.016	**	−0.026	***	−0.024	***
SC4 × T	−0.103	***	−0.114	***	−0.104	***	−0.095	***	−0.111	***
SC5 × T	−0.133	***	−0.152	***	−0.123	***	−0.129	***	−0.144	***
UA2 × T	−0.025	***	−0.037	***	−0.019	**	−0.021	**	−0.027	***
UA3 × T	−0.048	***	−0.057	***	−0.049	***	−0.040	***	−0.052	***
UA4 × T	−0.082	***	−0.088	***	−0.087	***	−0.074	***	−0.088	***
UA5 × T	−0.092	***	−0.107	***	−0.095	***	−0.077	***	−0.099	***
PD2 × T	−0.027	***	−0.040	***	−0.027	***	−0.016	*	−0.029	***
PD3 × T	−0.064	***	−0.081	***	−0.065	***	−0.049	***	−0.070	***
PD4 × T	−0.155	***	−0.167	***	−0.153	***	−0.149	***	−0.167	***
PD5 × T	−0.197	***	−0.216	***	−0.191	***	−0.188	***	−0.212	***
AD2 × T	−0.033	***	−0.047	***	−0.025	**	−0.029	***	−0.036	***
AD3 × T	−0.065	***	−0.081	***	−0.058	***	−0.059	***	−0.071	***
AD4 × T	−0.169	***	−0.208	***	−0.156	***	−0.149	***	−0.184	***
AD5 × T	−0.189	***	−0.207	***	−0.190	***	−0.174	***	−0.204	***
T	0.393	***	0.431	***	0.372	***	0.384	***	0.424	***
Log of scale parameter: module 2									−0.083	***
Log of scale parameter: module 3									−0.144	***
LL statistic	−15,813.054		−5162.8781		−5281.012		−5334.5721		−15,806.85
Observations	26,985		8995		8995		8995		26,985

FIGURE 9.

Histogram of all 3125 EQ-5D-5L health-state values (PRET-AS data).

Objective 3: comparability of the PRET-AS model