Developing and testing methods for deriving preference-based measures of health from condition-specific measures (and other patient-based measures of outcome)

Introduction

The methods described here cover the six stages (Figure 1) that can be used for deriving a CSPBM from an existing non-preference-based measure of HRQoL using traditional psychometric analysis and Rasch analysis. Item response theory can be used as an alternative to Rasch analysis. These stages are a guide to the key components in the development of a preference-based measure rather than a prescriptive methodology, as it is not always practical or possible to follow each stage separately or sequentially and the precise technique used may differ depending on the size and structure of the original instrument.

FIGURE 1.

The six stages for deriving a preference-based HRQoL measure.

Stage I: establishing dimensions

Conventional preference-based measures of health use a health-state classification system in which the dimensions are structurally independent in order to avoid nonsensical health states generated by the statistical design or multi-attribute utility theory as described in stage VI. In other words, there must be a low correlation between the dimensions. One technique for identifying structurally independent dimensions is factor analysis (confirmatory or exploratory, and this can use orthogonal or oblique solutions), although other techniques can be used. Factor analysis can be helpful in confirming the original dimensional structure of a measure or show where dimensions are not sufficiently independent and suggest ways to reduce the number of dimensions. It can also be used to suggest a possible dimensional structure when none was proposed by the original instrument developer. 27,31 Alternatively, it can suggest modifications to the dimensional structure proposed by the instrument developer; for example, in a study deriving the AQL-5D from the AQLQ, factor analysis suggested that there were five dimensions, whereas the original instrument had four. 28 Factor analysis needs to be used with care, however, as the factors it suggests may not make conceptual sense. Another technique that has been used is cluster analysis. 35 The extent to which items belong to a single dimension can also be examined using Rasch analysis (see Stage II: selecting items, below).

Stage II: selecting items

A dimension of a health-state classification system of preference-based measures is usually represented by one item, or occasionally two items, from the original instrument. The selection of items must be undertaken with great care. This process has been assisted in past research by a combination of conventional psychometric analysis, Rasch analysis and preference data,29,31 although other techniques such as item response theory can be used. A technique that has proven helpful in the process of item selection is Rasch analysis. This is a mathematical technique that converts qualitative (categorical) responses to a continuous (unmeasured) latent scale using a logit model. 36,37 The intuition underlying this approach is that the probability of an affirmative response to each item (or each response to each item) depends on the degree of difficulty of the item (or severity in the case of health) and the ability of the respondent. In the development of a health-state classification system, Rasch analysis can be used to eliminate items that poorly represent the underlying latent scale (for a brief overview of the concept of Rasch analysis, see Appendix 1).

The process of selecting items in a number of studies has been broken down into two stages. The first has been the elimination of poorly performing items that do not meet key criteria. 31 However, for larger measures of HRQoL this will leave a number of items in some dimensions and so the second stage involves selecting the best item for each dimension.

Stage III: exploring item-level reduction

In practice, patients may not be able to distinguish between item response choices. 5 Therefore, we recommend including this stage as an exploratory stage in examining the possibility of reducing the number of item levels in stage II. Item threshold probability curves from Rasch and response frequencies can be examined when selecting potential item levels for collapsing. Items for the AQL-5D, for example, were collapsed from 7 to 5 using evidence from the Rasch analysis. Cognitive debriefing of respondents by researchers may also help to inform this process.

Stage IV: validation

The application of stages I–III derives a health-state classification system that is small enough for valuation. However, before proceeding with the valuation process, we recommend validating the selected items by repeating the analysis on an alternative sample from the same data set that was used in stages I–III, an alternative time point from the data set used in stages I–III or an alternative data set.

Stage V: valuation survey

It is infeasible for members of the population to value all health states generated by most health-state classification systems, which typically define several thousands of health states. Therefore, a sample of health states is selected from the classification system for valuation. Two alternative approaches are used to sample health states and subsequently estimate utility values for all states (described below, see Stage VI: model health-state values): the decomposed approach and the composite approach. The decomposed approach uses multi-attribute utility theory to select states to determine the functional form (usually multiplicative or additive). This involves three stages: valuing each dimension separately to estimate single dimension utility functions; valuing ‘corner states’, where, for example, one dimension has extreme problems and all other dimensions have no problems; and valuing a selection of non-corner states. The composite approach involves the valuation of a sample of states chosen using a statistical design such as an orthogonal array and uses regression techniques to estimate values for all states defined by the classification. Both approaches generate states that contain combinations of levels across the dimensions, many of which will involve high levels on some dimensions and low levels for others. The problem this creates is discussed later and is one of the issues addressed in this research.

For use in economic evaluation health-state values must be valued on a common scale with an upper anchor of one at full (or perfect) health and a lower anchor at zero that is assumed equivalent to being ‘dead’. Once the health states have been selected they can then be valued by a sample of the population of interest, usually the general population, using valuation techniques such as time trade-off (TTO), standard gamble, visual analogue scales (VASs)5 or combinations of these. These valuation techniques are individual valuations of health states that require the individual to imagine themselves in the health state of interest. Social valuations of health states can also be used, such as person trade-off where individuals are asked to make a societal judgement of how many patients in one health state they would need to cure to be equivalent to curing a specified number of patients in another group. Individual valuation techniques have been typically used to elicit values for health state, although some researchers argue that societal valuation techniques may be more appropriate.

Stage VI: model health-state values

The decomposed approach produces utility values for all health states by solving a system of equations to generate preference weights for each dimension and any interactions specified in the model. 38 This approach was used to estimate utility values for the HUI-2 and HUI-34,39 and CSPBMs in rhinitis and asthma. 20,21

The composite approach uses regression techniques to estimate utility values for all health states valued. The standard model uses individual-level data, for which the value of a health state is defined as:

where i = 1, 2 … n represents individual health-state values and j = 1, 2 … m represents respondents. The dependent variable is the value for health state i valued by respondent j and X is a vector of dummy explanatory variables for each level λ of dimension ∂ of the health-state classification, where level λ = 1 acts as a baseline for each dimension. Beta represents the coefficients on X. ‘ε_ij’ is the error term that is subdivided, ε_ij = u_j + e_ij, where u_j is respondent-specific variation and e_ij is an error term for the ith health-state valuation of the jth individual, and this is assumed to be random across observations.

A number of alternative models are usually fitted to the data and the preferred model is selected based on goodness-of-fit statistics including mean absolute error (MAE), adjusted R², and the number of health states with errors of > 0.05 and 0.1. The algorithm from the preferred model can then be used by others to obtain utility values for the preference-based measures.

Lack of structural independence between dimensions

The approach of developing health-state classification systems has been successfully applied to a number of instruments, but one problem that has emerged is that some condition-specific measures do not have a clear multidimensional structure. This arises where items are found to be highly correlated. The items could be seen as unidimensional, although they may nonetheless tap important nuances in the impact of the condition. A condition may impact on a number of related areas of life, and a richer picture is provided by using more than one item. A lack of independence between items in a health-state classification system creates problems in the valuation and modelling stages. Many of the states that need to be valued for the modelling, using for example an orthogonal design, would involve combinations of dimension levels that would not be credible (e.g. feeling downhearted and low and happy most of the time). This problem is more likely to arise with condition-specific measures, as they tend to define a narrower range of domains.

One approach to this problem is to construct a sample of representative health states without using a health-state classification system. This involves defining health states that represent patients with particular severity levels of a health problem. This approach was pioneered by Sugar, Lenert and others using κ-means cluster analysis to break up the data into states. In one study they identified patterns of the physical and mental health summary scores of the SF-12 into models with varying numbers of discrete states. 40 They selected six states from the SF-12 data in a sample of depressed patients (i.e. near normal, mild mental and physical health impairment, severe physical health impairment, severe mental health impairment, severe mental and moderate physical impairment, and severe mental and physical impairment). These were defined in terms of scores, so a process of turning the score distributions of each state into words taken from the original 12 items to define the states had to be developed based on expert judgement. This is an interesting approach but it suffers from three limitations. First, the derivation of the states uses essentially arbitrary cut-offs in the cluster analysis. Second, it uses dimension scores that then need to be related to the item descriptions to generate the states and this uses expert judgement. Although some expert judgement is always needed in this type of work, it should be minimised where possible. Finally, this method has only been used to value a small sample of states and it is likely that it is not possible to allocate all patients to these states (in contrast to the inclusive approach of the health-state classification).

An alternative approach examined in this report avoiding these problems is to use the results of Rasch modelling to generate states to describe typical respondents at different points along the latent variable. The last decade has seen the increasing use of Rasch modelling in the development and testing of PROMs. As well as providing a method for assisting in the selection of items, we have pioneered its use in selecting health states for valuation that represent different levels of severity. This ‘Rasch vignette approach’ generates logical states based on the natural occurrence of states in the data set and avoids the infeasible combinations generated by statistical designs (e.g. an orthogonal array) from a health-state classification system.

A potential disadvantage with the Rasch vignette approach, as with clustering, is that it generates a small subset of potential states for valuation and so leaves a large number unvalued. A solution examined in this report is to estimate the relationship between the health-state utility values and the latent variable produced by the Rasch model using regression techniques. This permits the estimation of utility values for other points on the latent variable and hence other states generated by the items. This method is explored in Chapter 3.

Naming the condition

Many condition-specific measures state the cause of the health problems being assessed in the instrument (‘Your asthma interferes with getting a good night’s sleep all of the time’). There have been a number of studies looking at the effect of naming the condition on health-state values. 5 Evidence suggests that naming the condition has an impact in some, although not all, cases.

The reason why a condition-specific measure names the medical condition is probably to improve its precision, and avoid unrelated problems. On the other hand, by knowing the name of the medical condition non-patient respondents may bring their prejudices to the valuation exercise (e.g. ‘being limited in pursuing hobbies or other leisure time activities due to cancer’). Furthermore, this relies on the correct attribution by the patient. Patients may not be able to disentangle the impact of a given health condition from other possible conditions and non-health problems in their life. The usual practice in valuing generic measures (as being generic means, by definition, that disease label is not mentioned) is to avoid anything suggesting specific diseases. In the AQL-5D valuation, however, the disease labels were maintained because it was necessary to make sure what is valued in AQL-5D is the same as what patients report about their health using AQLQ, and thus it was important to minimise changes to wording. But, because of the danger of respondents bringing their own ideas to bear, maintaining the same wording does not really guarantee that what patients mean by a given statement and what non-patients understand by the same statement will be the same, and the impact this may have had on the valuation is not known. This report will examine the impact of naming the condition in valuation studies in Chapter 4.

Side effects

A well-known concern with condition-specific measures is that they do not cover potentially important side effects of treatment. When designing a study, one solution to this problem has been to include both generic measures and condition-specific measures in a trial. The problem for economic evaluation is that it is not possible to trade-off two measures to assess overall effectiveness. Economic evaluation requires a preference-based single index measure of health that is decision specific rather than condition specific, and captures the impact on the condition and any side effects. 41 This could be achieved either by taking a condition-specific measure and adding additional dimensions to cover any known side effects or by taking a generic measure that is known to cover side effects and adding dimensions to cover aspects of the condition. In Chapter 5 we examine the former strategy, by adding an additional generic dimension to two CSPBMs and valuing the classification systems with and without the add-on dimension. For one CSPBM we then apply the preference weights for the CSPBMs with and without the add-on dimension to a patient data set and examine the impact.

Comorbidities

Even assuming there are no side effects, the achievement of comparability between specific instruments requires an additional assumption, namely that the impact of different dimensions on preferences is additive, whether or not they are included in the classification system. The impact of breathlessness on asthma health-state values, for example, must be the same whether or not the patient has other health problems not covered by the condition-specific measure, such as pain in joints. Provided the intervention only alters the dimensions covered in the specific instrument then the estimated change in health-state value would be correct. However, it assumes preference independence between dimensions included in the classification system of the specific measure and those dimensions not included (i.e. for a health state with six dimensions, in which the level of each dimension is indicated by a digit, then the difference between state XXX and YYY should be the same as the difference between XXXZ and YYYZ).

Some degree of preference interaction has been shown to exist,2,4,42 with the impact of a problem on one dimension of health being reduced or exacerbated by the existence of a problem on another dimension. This is likely to create a larger problem in condition-specific measures that focus on a narrow range of health dimensions compared with generic measures that cover a broader range of health dimensions (although the problem will also exist for generic health measures, as they too exclude many potentially important dimensions). This problem of preference interaction will also be examined in Chapter 5.

One solution to the problem of preference interaction is to keep on adding extra dimensions to the condition-specific measure. However, this will reduce the usefulness of the new preference-based measure, as it cannot be applied to existing data sets containing the condition-specific measure because it requires additional information to be collected. Furthermore, respondents in valuation studies struggle if there are too many pieces of information at once and so there is a practical constraint on the size of classification systems designed for valuation. Any classification system that is to be amenable to valuation is likely to have a limited number of dimensions of health. For use in cross-programme comparison, it is ultimately a trade-off between having measures that are relevant and sensitive to those things that matter to patients with a particular condition (including side effects) and the potential size of the preference dependence between dimensions. The relative importance of the different arguments in this trade-off will vary between conditions. In Chapter 5, we examine the extent of this problem for two conditions (asthma and common mental health problems).

Information loss compared with original measure

The derivation of health states based on a small subset of items of the original measure inevitably involves a loss of information. The original measures had multiple items in order to improve reliability and so achieve better psychometric performance in terms of validity and responsiveness. Given that the original rationale for using a condition-specific measure is to use its greater relevance and sensitivity, it is important to ensure that it retains this informational advantage in the process of becoming an index.

The extent of information loss can be examined using conventional psychometric tests of validity and responsiveness. 43 In this report we compare the preference-based condition-specific measure with the original full measure in terms of construct validity by examining scores between severity groups and responsiveness to change over time. Any loss in information needs to be balanced against the ability of preference-based measures to generate quality adjustment weights for QALYs, but a substantial loss might suggest that the whole process of selecting a few items inevitably reduces the psychometric performance of the measure. Chapter 7 looks at this empirically for four CSPBMs.

Do preference-based measures derived from condition-specific measures really offer an improvement over generic measures?

An important question is whether preference-based measures derived from condition-specific measures really do offer an improvement over existing generic measures in terms of their psychometric properties, and whether they generate sufficiently different values for differences between states and for changes over time to be important for the results of economic evaluation. This is a further development of the previous section (see Information loss compared with original measure) and is addressed in Chapter 6 using similar methods on data sets that contain the condition-specific measure and one or more of the generic preference-based measures.

What are policy implications of using condition-specific measures?

One of the reasons for health economists being reluctant to use condition-specific measures has been a view that they cannot be used to make cross-programme comparisons. In Chapter 7 of this report we review these arguments in the light of the findings of the research presented in this report.

Search strategy

Current methods for developing preference-based measures of quality of life from condition-specific measures or ‘de novo’ that were published in English were identified using a literature search conducted in December 2010. The literature search was undertaken on MEDLINE, EMBASE, Health Technology Assessment (HTA) database, Cochrane Central Register of Controlled Trials (CENTRAL), Web of Science (including Science Citation Index, Social Sciences Citation Index, Conference Proceedings), Cochrane Database of Systematic Reviews (CDSR), Cochrane Methodology Register, Database of Abstracts of Reviews of Effects (DARE). The following search strategy was used:

1. qol/hrqol/qaly/”Quality of Life”/quality of life/Quality Adjusted Life Year AND

2. utility/utilities/preference with based/index/measure within 4 words of 1 AND

3. transform*/translat*/transfer*/conver*/map*/deriv*.

The search identified 4093 papers; 104 papers remained after a title and abstract sift. The search strategy meant that the review included only journal papers published in or before December 2010.

Exclusion criteria

Of the remaining 104 papers, 78 papers were excluded at the full-paper stage for the following reasons: 30 reported the derivation and/or valuation of vignettes; 18 were either summaries or analyses of patient-valued utility data of own health, for example using standard gamble or TTO or existing utility measures in a patient group; eight reported the methodology used to develop a generic measure; five reported on measures that use patient-reported own health to produce utility scores for a selection of states, with no modelling of a tariff to produce values for all states; four were mapping studies (either mapping between measures, between valuation techniques or between a valuation technique and summary score); four were population-specific rather than condition-specific; three had no utility score (they either summarised values derived only from individuals, the scoring system was not preference based, or a tariff has not as yet been produced for the measure); two were treatment specific, not condition specific; one measured global quality of life rather than HRQoL and was not condition specific; one involved non-health aspects such as cost; and one measured relative improvement in HRQoL rather than HRQoL. This left 26 papers for inclusion in the final review.

Data extraction

The review examined the methodology used to produce the CSPBMs, focusing on the six-stage approach outlined in Chapter 1, where stages I–III produce the classification system, stage IV validates the classification system, stage V elicits health-state utility values and stage VI models the utility data to produce utilities for all health states defined by the classification system. The review was concerned with the motivation behind the study; the conditions examined; for stages I–III the method of construction and the number and composition of items and dimensions in the classification system; for stage IV, whether and how the classification system was validated; for stage V, whose values were used, how values were obtained, and whether utilities were valued on to a 1–0 full health–dead scale; and for stage VI, how utilities were estimated for every states and the accuracy of this process. Data extraction was undertaken by one member of the research team and summarised in Microsoft Excel (Microsoft Corporation, Redmond, WA, USA) using items summarised in Table 1 that were previously agreed by the team.

Included papers

A brief summary of the 26 papers included in the review is presented in Table 2. Out of these 26 papers, 17 were published in non-clinical journals, including Quality of Life Research (five papers), Health and Quality of Life Outcomes (3), Pharmacoeconomics (3), Health Economics (1), International Journal for Quality in Health Care (1), Journal of ClinicalEpidemiology (1), Medical Decision Making (1), Quality in Health Care (1) and Value in Health (1). The remaining nine papers are each published in separate clinical journals: Amyotrophic Lateral SclerosisandOther Motor Neuron Disorders, British Journal of Cancer, British Journal of Dermatology, British Journal of Obstetrics and Gynaecology, Chest, Investigative Ophthalmology and Visual Science, Journal of Oral & Maxillofacial Surgery, Ophthalmic Epidemiology and Optometry and Vision Science.

Conditions

The range of conditions is broad, covering amyotrophic lateral sclerosis (ALS), asthma and paediatric asthma, erectile (dys)functioning, diabetes, flushing, glaucoma, handicap, head and neck cancer, lung cancer, menopause, menorrhagia, minor oral surgery, overactive bladder, paediatric atopic dermatitis, Parkinson’s disease, pulmonary hypertension, rhinitis, sexual quality of life, stroke, urinary incontinence, visual impairment. However, several measures cover similar conditions: asthma (two measures), cancer (two measures), vision (three measures), bladder (two measures), and sexual functioning (two measures). The papers discuss the derivation of 22 measures (four measures are each described using two papers, one of which has two sets of preference weights derived in two different countries).

Stages of developing preference-based measure

Stage IV: validation of classification system

Few papers mention whether and how the classification system has been validated,20,21,31,45,56 and where validation of the classification system is mentioned the meaning of validation is interpreted differently depending on the study. One paper31 replicates the analysis used to construct the classification system using a different data set and different time-points of the data set used in the initial analysis and this can be interpreted as validation of the classification system. The remaining four papers20,21,45,56 do not validate the classification system but validate their measure using discriminative validity, by examining how the measure performs across subgroups of patients with different severity levels of the condition. These papers (with the exception of Burr et al. 45) also examine the agreement of their measure with a generic utility measure and a non-preference-based measure (Poissant et al. 56 used a generic measure, whereas Revicki et al. 20,21 used a condition-specific measure).

Stage V: valuation to elicit health-state values

The valuation component of each of the studies is outlined in Table 4. Three papers31,53,58 report only the classification component of deriving a measure and hence are not discussed in this section.

Stage VI: modelling health-state utility data

The methods used to obtain health-state values for all states defined by the classification system are shown in Table 5. Three papers valued all health states defined by the classification and therefore did not undertake any form of modelling of these values. One study valued all states using VAS and converted these to standard gamble using a power function. Nine studies applied a composite approach using statistical analysis involving regression analyses used to estimate an additive function. The utility value of a health state is calculated as the sum of the coefficients of the appropriate levels of each dimension. Eight papers used a decomposed approach, which uses multi-attribute utility theory as the basis for the modelling, with five papers estimating a multiplicative function and three papers estimating an additive function. The exact process used in the papers reported here differs across different studies. Another study mapped Rasch logit scores generated for health states on to mean observed health-state values using regression analysis to produce a mapping function enabling utility values for all states to be estimated using the Rasch logit scores of each health state. 60 The methodology for one study was unclear. 56

Clinical Outcomes in Routine Evaluation – Outcome Measure

The CORE-OM has been developed to assess the outcomes of psychological interventions for people with common mental health problems. 70,71 It has 34 self-report items covering the domains of subjective well-being, symptoms, function and risk, outlined in Table 6. Each item has five levels (‘not at all’ through to ‘most or all of the time’). It has been shown to be reliable, valid and sensitive to change in clinical samples. 72 It has become one of the most widely used mental health outcome measures for psychological services in the NHS and is being used in a number of clinical trials as well as in Service Development and Organisation (SDO) programme evaluations of service delivery. However the CORE-OM cannot be currently used to produce QALYs as its scoring system is not preference based. The number of items in the CORE-OM also means that it is not amenable to health-state valuation in its current form.

Developing the health-state classification

Health-state classification

Stages I–III: dimensions, items and item response levels

The results are summarised here and further details can be found elsewhere. 68 Out of the 34 CORE-OM items, 26 had disordered item thresholds, meaning that adjacent item levels were merged to obtain threshold ordering for these items. Two items (6. ‘I have been violent to others’ and 22. ‘I have threatened or intimidated another person’) were excluded from the analysis as they were judged irrelevant for a preference-based measure of HRQoL, and another item (34. ‘I have hurt myself physically or taken risks with my health’) was excluded as it was judged as being ambiguous. The application of the Rasch criteria and psychometric analysis led to the exclusion of a further 14 items (3, 5, 8, 9, 14, 18, 19, 23, 24, 27–31).

The remaining 17 items represent the items available for selection for the unidimensional emotional component of the classification system. Table 7 details the performance of these items using Rasch statistics and psychometric analysis. Following this model, items were excluded one at a time and the model was re-estimated for various combinations of five items, one from each of the remaining five conceptual domains (symptoms – anxiety; functioning – general; functioning – close relationships; functioning – social relationships; risk/harm to self) that were considered major domains for people with common mental health problems and thus requiring representation in the final classification system. The final set of items selected to represent the emotional component were items 1, 15, 16, 21 and 33, each with three-item response levels (not at all; only occasionally or sometimes; often, most or all the time). The results of the additional test proposed by Smith74 confirmed the unidimensionality of the emotional component. Figure 2 presents the item threshold map for the Rasch model estimated on the selected emotional component and is discussed further below.

TABLE 7 - Results of Rasch analysis with the 17 items of CORE-OM fitting into the Rasch model

Item	Rasch analysis			Psychometric analysis
	Residual	χ2	p-value	Standardised response mean	Missing data	Spearman’s p-value
1. I have felt terribly alone and isolated	1.415	10.118	0.072	0.99	0.4	0.714
2. I have felt tense, anxious or nervous	–0.373	2.658	0.752	1.18	0.3	0.603
4. I have felt OK about myself	–0.107	2.326	0.802	1.00	0.6	0.646
7. I have felt able to cope when things go wrong	0.371	5.829	0.323	0.78	0.6	0.594
10. Talking to people has felt too much for me	0.546	4.614	0.465	0.81	0.7	0.548
11. Tension/anxiety have prevented me doing important things	–0.191	6.021	0.304	0.89	0.8	0.642
12. I have been happy with the things I have done	0.708	1.848	0.870	0.85	0.8	0.624
13. I have been disturbed by unwanted thoughts and feelings	2.376	10.195	0.070	0.95	0.5	0.564
15. I have felt panic or terror	0.133	5.590	0.348	0.84	0.4	0.576
16. I made plans to end my life	–0.485	4.897	0.428	0.29	1.0	0.436
17. I have felt overwhelmed by my problems	–2.084	11.369	0.045	1.09	1.0	0.744
20. My problems have been impossible to put to one side	0.254	1.877	0.866	1.04	0.9	0.629
21. I have been able to do most things I needed to	1.424	3.410	0.637	0.69	0.8	0.568
25. I have felt criticised by other people	0.918	3.362	0.644	0.70	0.8	0.558
26. I have thought I have no friends	0.742	8.993	0.110	0.65	0.9	0.595
32. I have achieved the things I wanted to	0.799	1.426	0.921	0.86	1.5	0.590
33. I have felt humiliated or shamed by other people	–0.899	7.809	0.167	0.61	1.1	0.557

FIGURE 2.

Rasch item-threshold map of the emotional component of CORE-6D. Severity levels: 0, not at all; 1, only occasionally or sometimes; 2, often, most or all the time – with the exception that the response levels are reversed for the positively worded item ‘I have been able to do most things I needed to’. Source: Mavranezouli I, Brazier J, Young A, Barkham M. Using Rasch analysis to form plausible health states amenable to valuation: the development of the CORE-6D from a measure of common mental health problems (CORE-OM). Qual Life Res 2011;20:321–33.

Item 8 (‘I have been troubled by aches, pains, physical problems’) was excluded from the emotional component, as its fitting was poor, as expected, as it represents physical health rather than mental health. This item was considered important for inclusion in the classification system, as physical symptoms constitute an important dimension in their own right. Item 8 therefore was combined with the items selected for the emotional component to form an additional physical health dimension.

Health-state classification

Table 8 presents the classification system in which a health state is made up of six sentences and hence has a six-digit identifier, from best state 000000 to worst state 222222. This system generates a total of 729 health states, although it is likely that many of these health states are not plausible owing to the unidimensionality of the emotional component.

TABLE 8 - The CORE-6D classification system

Item		Level
Emotional component
1	I never feel terribly alone and isolated I feel terribly alone and isolated only occasionally or sometimes I feel terribly alone and isolated often, most or all the time	0 1 2
2	I never feel panic or terror I feel panic or terror only occasionally or sometimes I feel panic or terror often, most or all the time	0 1 2
3	I never feel humiliated or shamed by other people I feel humiliated or shamed by other people only occasionally or sometimes I feel humiliated or shamed by other people often, most or all the time	0 1 2
4	I am able to do most things I need to often, most or all the time I am able to do most things I need to only occasionally or sometimes I am not able to do the things I need to	0 1 2
5	I never make plans to end my life I make plans to end my life only occasionally or sometimes I make plans to end my life often, most or all the time	0 1 2
Physical health
6	I am never troubled by aches, pains, physical problems I am troubled by aches, pains, physical problems only occasionally or sometimes I am troubled by aches, pains, physical problems often, most or all the time	0 1 2

Stage IV: validation

The classification system was confirmed in validation analysis using another random sample of 400 patients. The Rasch model for the selected items had satisfactory model fit, item-fit and no DIF was observed.

Stage V: health-state selection and valuation study

Stage VI: modelling to produce preference weights for all states

This methodology builds on the approach undertaken for a unidimensional measure where health-state utility values for all health states defined by a classification system were produced using the relationship between Rasch model logit values and mean observed TTO values for a sample of health states. 78 This study develops the new approach further, as it contains both a unidimensional component and an additional independent dimension. The preference weights for the emotional component were estimated using the methodology outlined in Young et al. 62 and the preference weights for the additional physical health dimension were estimated using dummy variables following the standard approach outlined in Chapter 1 (e.g. Brazier et al. ,2 Yang et al. 30 and Dolan42).

Regression analysis was used to estimate the relationship between the elicited TTO values for each health state and the Rasch model logit value corresponding to the emotional component of the state (the Rasch health state) and the response level of the physical health dimension. The standard linear model specification was:

where y represents mean TTO value for state i, R represents the rescaled Rasch logit value (linearly rescaled to match the range of y), P is a dummy variable for response level j of the physical health dimension and ε is the error term. The inclusion of quadratic and cubic terms for the rescaled Rasch logit value was explored. Estimation was via ordinary least squares.

Model fit was assessed using R² and RMSE. The model with the best fit was selected and used to predict health-state utility values for all health states described by the classification system using their respective Rasch model logit value and the response level of the physical health dimension.

Stage V: selected health states and valuation study

Stage VI: modelling health-state values

The results of the regression analysis are presented in Table 12. The dummy variable for the physical health dimension at level 1 was insignificant for all models; however, the dummy variable for level 2 of the physical health dimension is significant at the 5% level in all models. Across all models the coefficients for the rescaled Rasch logit score and squared and cubic terms for this score are significant at the 5% level, suggesting the appropriateness of using model 7 as the preferred model that contains all these terms. R² and RMSE also confirm that this model performed better than the other models and was therefore used to produce estimated utility values for every state defined by the CORE-6D using the rescaled Rasch logit score for each state.

Health-state description

Health-state descriptions were generated using the classification system from EORTC-8D, a preference-based measure for cancer derived from the EORTC QLQ-C30. 34 Box 1 presents the EORTC-8D classification. There are eight dimensions each with four or five severity levels: physical functioning, role functioning, pain, emotional functioning, social functioning, fatigue and sleep disturbance, nausea, and constipation and diarrhoea. The classification system defines a total of 81,920 health states. The original valuation study used to elicit preference weights for all states defined by the classification system did not include a condition label34 and the meaning of dimensions remains unchanged if labels are added or removed from the classification system. We chose condition labels that respondents could reasonably perceive accounted for EORTC-8D health states. We chose cancer and IBS as the condition labels, as the measure is designed for cancer, and consultation with several clinicians and doctors showed that the EORTC-8D classification system would accurately describe IBS. Advantages of having cancer and IBS as the condition labels are that experience and preconceptions of these conditions is likely to differ, and whereas cancer can be terminal IBS is non-fatal.

Valuation survey

A valuation study was conducted where members of the UK general population each valued eight health states from EORTC-8D using TTO. The sample was divided into three groups: no label, IBS label and cancer label. All respondents valued the same health states, differing for each group only by the condition label used at the heading of the health state. The original valuation study asked each respondent to value eight health states, and hence eight health states were also selected in this study to be valued by each respondent and therefore by each group. Sample size was chosen to ensure sufficient power for comparison of mean health-state values across the three groups using simple t-tests. This required a total of 219 completed interviews containing 73 health-state values per state per group, assuming a power of 0.8, significance level of 0.05, SD of 0.3 and an expected difference of 0.1.

The sampling strategy used two steps to ensure that the sociodemographic characteristics of each group were the same and were representative of the UK general population. First, households were sampled using the AFD Names & Numbers database, and, using geodemographic ACORN profiles, the sample was balanced to the UK general population. Second, every unique postcode in the sample was divided across the three labelling groups to produce separate samples for each group. Respondents were interviewed in their own home by trained interviewers who had worked on previous valuation surveys including the HU1-286 and OAB-5D. 30 The project was approved by the ScHARR Research Ethics Committee at the University of Sheffield.

All interviews in the no-label group were conducted before respondents from the IBS-label group were contacted and, subsequently, all interviews in the IBS-label group were conducted before respondents from the cancer-label group were contacted. Using this design meant that respondents in the cancer and IBS-label groups could be informed of the relevant condition in the cover letter sent requesting their participation and in the participant information sheet. This design was selected owing to ethical concerns that respondents should be informed of the condition that would feature in their interview but should be unaware of the other conditions featured in this study in case this affects responses.

Selection of health states

Health states were selected to represent a range of health states for the EORTC-8D, using the results of the original valuation study to inform the selection. The original valuation study valued 85 health states across 12 ‘card blocs’, each containing the worst state plus seven other states. The best card bloc was selected for this study using: minimum prediction error per card bloc, largest range of mean TTO distribution per card bloc, smallest missing data per card bloc, and general ‘feasibility’ of states. Box 2 presents an example health state (24432411) for each label group. Level 1 represents no problems in that dimension, whereas level 4 (level 5 for physical functioning) is the most severe level for each dimension.

The interview began with respondents self-completing the EQ-5D. Respondents in the cancer and IBS-label groups were then shown an information sheet about the relevant condition (see Rowen et al. 85). Respondents completed the EORTC-8D classification system for themselves if they had the condition, or otherwise completed the classification for someone they knew/imagining someone with the condition to ensure that respondents were familiar with the system. Respondents in the no-label group self-completed the EORTC-8D for themselves. Respondents undertook a ranking exercise of the eight health states alongside ‘full health’ and ‘dead’, then valued these eight states using the MVH study version of TTO including a visual prop designed by the MVH Group (University of York)42,77 using ‘full health’ as the upper anchor. Respondents valued an additional practice state (22332322) before valuing the eight ranked health states to familiarise them with the TTO task. Finally, respondents self-completed questions covering sociodemographics, health service usage and experience of the labelled condition (where applicable).

Respondents were excluded from the TTO analysis if they valued all states as identical and less than one, valued the worst state higher than every other state or valued all states as worse than dead.

Analysis

Factorial analysis of variance (ANOVA) was estimated using a generalised linear model to determine significant differences in respondent characteristics across label groups. Simple t-tests were used to compare mean health-state values across the three label groups. The impact on elicited utility values from the inclusion of each condition label in the health-state description is analysed using regression analysis. The model specification is:

where i represents individual respondents and j represents the eight health states. The dependent variable, y, represents the TTO utility value, x represents the vector of dummies for the health states, q represents the vector of dummies to capture labelling effects, r represents the vector of interaction terms to capture labelling and severity effects, z represents the vector of sociodemographic characteristics including experience with the labelled condition, and ε_ij represents the error term. Random effects generalised-least-squares (GLS) models were used, as they are appropriate for the structure of these data for which all respondents have multiple observations87 and have been used in similar valuation surveys (see, for example, Brazier et al. 2 and Brazier and Roberts3). Stata version 9 (StataCorp LP, College Station, TX, USA) was used for all regression analysis and SPSS version 15 was used for the descriptive statistical analysis.

The data

The sample contains responses from 241 respondents from northern England, with a response rate of 39% answering their door at time of interview, and completion rate of 99% across all TTO tasks. The response rate for the cancer-label group was 38%, whereas the response rate for the no-label and IBS-label groups was larger at 40%. No respondents met the exclusion criteria. The sample is largely comparable with the population of South Yorkshire and England (see Rowen et al. 85 for further details). Sample characteristics for each label group and comparison of the characteristics using ANOVA are shown in below (see Table 13). Respondent characteristics are significantly different at the 5% level for respondents aged 18–40 years and full-time students. These differences should be taken into account when modelling the data. IBS and cancer-label groups have different proportions of respondents with experience of the relevant condition both in their family and in caring for others.

Descriptive statistics

Table 13 presents descriptive statistics of health-state values elicited for each label group and the modelled utility values estimated using regression analysis in the original valuation study. For six of the eight health states the mean value is highest for the IBS label; for six of eight states the mean value is lowest for the cancer label, and these are for the more severe health states. Health-state values for the cancer-label group were significantly different from health-state values for both the no-label group (t-test p-value = 0.01) and the IBS-label group (p-value < 0.001) but there were no significant differences between the no-label groups and IBS-label groups (p-value = 0.28).

Regression analysis

Regression analysis examining the relationship between elicited health-state utility values, health-state descriptions, condition labels and sociodemographic characteristics of respondents is presented in Table 14. Model (1) includes as predictor variables state level dummy variables, label dummy variables and sociodemographic variables; model (2), in addition, includes experience of the labelled condition; model (3) adds to model (1) interaction terms to reflect the interaction between the specific health-state and labelled condition; and model (4) adds to model (1) both interaction terms and experience of the labelled condition. Models using a range of sociodemographic and experience variables as predictors were estimated and the best models (using diagnostics, correlations and proportion of significant coefficients) are presented here. Only two sociodemographic variables are significant: where students have lower utility values and unemployed respondents have higher utility values than the employed/self-employed reference group. Goodness-of-fit measures report that models (3) and (4), which include interaction effects for cancer states, perform better than models (1) and (2), in which only experience variables and a simple additive labelling variable are included.

Health-state dummy variables are significant at the 1% level in all models and the size of coefficients is consistent, as the decrement in the elicited utility value is larger for more severe health states (as reported using the modelled utility values from the original valuation study). The IBS label is never significant, whereas the cancer label is significant in the models when it is appropriate for inclusion (where interaction effects are not also included, as this variable is perfectly collinear with interaction variable 11111111 × cancer).

The inclusion of experience variables improves the model using within and between R². Respondents in the IBS-label group with experience of IBS in themselves or in caring for others did not, on average, give significantly different responses to other respondents. In contrast, respondents in the cancer-label group with experience of cancer in themselves gave, on average, lower utilities for health states, whereas participants in this group with experience of cancer in caring for others gave higher utilities.

Interaction terms reflecting the interaction between the specific health state and the cancer label have negative coefficients, meaning that the inclusion of the interaction term reduces the utility value for that state. The inclusion of the interaction terms reduces the absolute size of coefficients for the state variables. The only exception is the positive coefficient for the interaction term for state 31212241 in model (3). Coefficients for the interaction terms were insignificant for the three mildest states and significant at the 5% level for the remaining five more severe states. Coefficients for these more severe states (44321321, 23141224, 24432411 and 51224434) have little variation, ranging from –0.128 to –0.148 in model (4) with the exception of the much larger coefficient for the most severe state (54444444) at –0.254 in model (4). Models were explored with the inclusion of interaction terms to reflect the interaction between the specific health state and the IBS label, but were never significant and did not improve the model.

CORE-6D study

The CORE-6D is a preference-based measure of health derived from the original CORE-OM outcome measure for common mental health problems. 70 Its derivation from the original CORE-OM has been summarised in Chapter 3 and described in detail elsewhere. 68 The health-state classification of the CORE-6D contains five emotional domains and a single physical-health dimension (see Table 13). The classification defines 729 states in all. Although the full CORE-6D does contain the physical health dimension, the study reported in Chapter 3 valued states with and without this dimension. The valuation of this instrument presented an opportunity to examine the impact of adding on a physical dimension to the five domains of the emotional component of the CORE-6D.

This add-on component ‘piggybacks’ on to the CORE-6D valuation survey. In the main study, there were 18 full CORE-6D states valued. In this add-on component, a further four states were valued and these were five-dimensional ‘emotional’ states that did not mention physical health at all: best state (00000), worst state (22222), 11000 and 22110. The valuation study design is detailed in Chapter 3 for all health states, although that chapter only reports the results for the 18 CORE-6D states. The 22 health states were divided into three ‘card blocs’. Card bloc 1 included the four emotional states and four full CORE-6D states. The other two card blocs only included eight full CORE-6D states each. All three blocs included the CORE-6D state 222220. This design results in the four emotional only states being matched to 12 CORE-6D states in terms of the emotional component. They differ only in having a physical dimension at either at level 0, 1 or 2. Emotional state 11000, for example, is matched to the CORE-6D states 110000, 110001 and 110002.

Bloc 1 allowed us to undertake paired t-tests between the mean values of the four emotional states and the four with the physical dimension. The sample size calculation found that a power of 0.8, significance of 0.05, a SD of 0.3 and an expected difference of 0.1 requires a sample of 73, and this was achieved for all states in the survey. However, given there are another eight independent comparisons between matched states, and a series of further comparisons between states with different levels on the physical health dimension, it was decided to undertake an ANOVA of mean health-state values in order to establish the significance of the impact of the physical health dimension.

AQL-5D study

The AQL-5D is a five-dimension five-level preference-based measure for asthma29 derived from the AQLQ24 using Rasch and conventional psychometric analysis as described in Chapter 1. 31 The five dimensions are concern about asthma, shortness of breath, weather and pollution stimuli, sleep impact, and activity limitations (Box 3). Each dimension has five levels of severity, with level 1 denoting no problems and level 5 indicating extreme problems. In the study reported in this paper, a reduced AQL-5D health-state classification system is valued where each dimension has three levels of severity: level 1 denoting no problems, level 2 denoting some problems and level 3 denoting extreme problems (see Table 13). These relate to levels 1, 3 and 5 in the original classification system. The reduced classification was chosen primarily to limit the size of the valuation survey required to address the study aim.

The pain/discomfort dimension from the EQ-5D was added at the end to the AQL-5D, which also has three levels, in effect making it AQL-6D1 (see Box 3). This extra dimension was chosen to ensure little overlap and correlation with the existing dimensions while being able to capture potential comorbidities and/or side effects.

This was a more ambitious study than the CORE study, in that the aim was to estimate full models with and without the extra dimension. Therefore, health states for the AQL-5D and AQL-6D were selected using an orthogonal array in SPSS version 15. Sixteen health states were selected for AQL-5D, one of which was a repeated state (11111). Eighteen health states were selected for AQL-6D with no repeats. The worst state for each measure (33333 and 333333) was added, taking the number of unique health states to 16 for AQL-5D and 19 for AQL-6D. These included four health states that were matched across the two classification systems in terms of the level of the non-pain dimensions: states 11111 and 111111; 12132 and 121323; 23131 and 231311; and 33333 and 333333.

Health states were divided into three ‘card blocs’ of eight states for each measure, making six blocs or combinations of states in all. Different respondents valued the AQL-5D and the AQL-6D, although they were randomly selected from the same sampling frame. The worst state appeared in all card blocs and the remaining matched health states appeared in two card blocs to improve power. Other states repeated across more than one bloc for AQL-5D and AQL-6D were chosen to reflect a range of severity (using summed levels and dimensions) and levels for each dimension. Combinations of states within card blocs were chosen to reflect a range of severity (using summed levels and dimensions) and to ensure each card bloc included each level of each dimension.

The impact of adding pain/discomfort to AQL-5D was examined in two ways. First, the mean values for the matched states were compared using independent sample t-tests. Secondly, TTO values were modelled and the asthma-specific coefficients were compared across AQL-5D and AQL-6D. Third, the significance of the pain coefficients in the AQL-6D model was examined.

Valuation surveys

The two studies used the same valuation methods.

CORE-6D

The sample contained responses from 225 respondents from South Yorkshire. The response rate was 45.7% for respondents answering their door at the time of the interview (after several attempts), with a TTO completion rate of 99.7% across all interviews. The sample is compared with the general populations of South Yorkshire and England in Table 5 (see Chapter 3). Overall, the study sample had a higher average age and a higher proportion of females, home owners and retired individuals, and a lower proportion of employed/self-employed individuals. All responses were included in the analysis, as no respondents met the exclusion criteria.

The results across all respondents are shown in Table 15. Each state was valued by between 74 and 76 respondents, except for state 222220, which was valued by all respondents. The mean TTO values for the emotional states shown in the first column are consistent with their severity: the best emotional state had a mean value of 0.95 (SD 0.15) down to the worst state at 0.14 (SD 0.48). The impact of adding the physical dimension is consistent for the best state (00000). The physical dimension has little impact at level 0 or level 1, but at level 2 it reduces the mean TTO value by 0.13. The paired t-tests found the impact of level 2 to be significant at the 5% level. For the other three emotional states, the impact of adding physical health follows a different pattern: physical problems at level 0 (i.e. saying there are no physical problems) result in an increase in TTO value of 0.07, 0.13 and 0.09 across the three states. Level 1 is also associated with increases, although they are smaller, at 0.05, 0.05 and 0.07. Only level 2 is associated with decreases of 0.07, 0.04 and 0.04. The ANOVA found the impact of physical health to have been statistically significant for the two milder states (i.e. 00000 and 11000).

AQL-5D and AQL-6D

The response rate for all eligible respondents answering their door was 45.8%. The respondents were similar to those of South Yorkshire and the UK for age and gender, but tended to have a higher proportion of retired individuals, a lower proportion of employed individuals and a lower mean EQ-5D score (0.80 vs 0.86). There were no significant differences between the samples who valued the AQL-5D and AQL-6D. 89

Just two respondents out of the 184 successfully conducted interviews were excluded. This left 1455 TTO values elicited from 180 respondents, with 727 and 728 for the AQL-5D and the AQL-6D health states, respectively. Descriptive statistics across the health states are presented in Table 16. Three pairs of matched states were each valued between 60 and 62 times, the worst states (33333 and 333333) were valued 91 times each and the remaining states were valued between 29 and 31 times. Across the four matched states, mean values for the best (0.97 vs 0.98) and worst states (0.26 vs. 0.30) of AQL-5D and AQL-6D were not found to be significantly different (see Table 17). The mean value of 12132 from the AQL-5D (0.70) was significantly higher than 121323 from AQL-6D (0.56) (p-value = 0.061). By way of contrast, the mean value for the AQL-5D state 23131 (0.64) was lower than the AQL-6D state 231311 (0.78) (p-value = 0.034).

Modelling of the preference data

For AQL-5D the coefficients across the five dimensions are consistent with the severity levels within each dimension, i.e. coefficients for level 3 > level 2 > level 1 (Table 17). The only exception is the sleep dimension, where the level 2 coefficient has the ‘wrong’ sign, although it is very small and non-significant. Level 3 of breath, weather and sleep are significant, as are levels 2 and 3 for activities. The RMSE at the individual level is quite high at 0.4, but the MAE at the state level is 0.038 and this compares very favourably with that achieved in the original model of 0.047. 21 The plot of observed and predicted mean health-state TTO values and residuals ordered by mean observed value suggests that there is no obvious pattern in the errors. 89 The N3 term was not significant in any model and so is not included in the model reported here.

For AQL-6D the pain/discomfort dimension had significant coefficients for levels 2 and 3 at the 5% level, with level 3 pain/discomfort having the largest coefficient (0.301) of any dimension in the AQL-6D for model (4). There were three inconsistencies, with levels 2 of breath (–0.001), weather (–0.016) and sleep (–0.001) being negative, but these are all < 0.02 and none was significant. Overall the model performed well in terms of MAE (0.030 vs 0.038 for AQL-5D) at the state level and again there is no obvious pattern in the errors. There was little change to the coefficients for concern and sleep compared with the AQL-5D model, but a noticeable reduction in the coefficient for level 3 of weather (which was significant in the AQL-5D model at the 5% level but non-significant in the AQL-6D model). However, there were substantial reductions in the coefficients for shortness of breath and activities, particularly for the level 3 coefficients of 0.167 compared with 0.047 and 0.307 compared with 0.150 for the AQL-5D and AQL-6D models, respectively. The results of the z-tests confirm that there were significant differences between the AQL-5D and AQL-6D models in the coefficients of level 3 for shortness of breath and activities at the 1% level. Further modelling is reported elsewhere. 89

Generic measures

Condition-specific measures

Patient data sets

Analysis

Information loss

Performance in comparison with a generic preference-based measure

Overall discussion

Overall, the CSPBMs performed better than the generic measures in terms of discriminative validity measured using effect size. They had smaller mean differences than EQ-5D yet consistently had either similar or larger effect sizes. This may be important for the power of a study, as the CSPBMs may be able to detect significant differences with smaller sample size. Responsiveness for the CSPBMs and generic preference-based measures was only able to be examined in three data sets, but indicated that AQL-5D and CORE-6D had higher standardised response means and effect sizes than EQ-5D. The reverse was found for EORTC-8D, although it was still able to detect a statistically significant change. The CSPBMs were always able to detect a statistically significant change, yet EQ-5D was unable to detect a statistically significant change in the asthma exacerbation study data set. The larger effect size and standardised response mean for CSPBMs can reduce uncertainty, which can be important for the precision estimates and probabilistic sensitivity analysis in an economic evaluation. However this does not mean the CSPBMs are necessarily a more valid preference-based measure of health. One key limitation of this analysis is the lack of available data to examine all measures using multiple data sets for both discrimination and responsiveness and is limited by the range and representativeness of each sample for each population of interest. Further research using other data sets is encouraged.

The modestly better or comparable performance between CSPBMs and generic preference-based measures is reassuring given that they are all measures capturing HRQoL, but raises the question whether or not CSPBMs offer an advantage over the generic preference-based measures. The analysis suggests that one consistent advantage of using the CSPBMs rather than EQ-5D is greater refinement of values at the upper end of HRQoL and thus greater ability to discriminate between different severity groups. There was a high level of agreement between the generic preference-based measures and CSPBMs in terms of ICC and correlation. However, contrary to what may be expected, mean change in utility scores before and after treatment and mean differences across severity groups were often larger for the generic preference-based measures than the CSPBMs. One possible explanation for the larger mean change and differences is that generic measures also capture changes in HRQoL owing to comorbidities and side effects. Another possibility is that the EQ-5D value set used here has a wider range of potential values than all other preference-based measures used here. The large range of the UK EQ-5D value set is unique to that valuation study, as it has not been found in valuation studies of other measures or in valuation studies of the same classification in other countries (see, for example, Shaw et al. 110).

Naming the condition

Many condition-specific measures name the condition in their items, and this is reflected in some of the health-state classifications, such as one for overactive bladder. 31 This is thought to improve its sensitivity. However, there are concerns that naming the condition in the health state may invite respondents valuing the state to consider their experience or preconceptions of the condition. These may be views that the condition is better or worse than being described. This would distort the values for a condition-specific health state. Another concern is the different nature of causal attribution in non-patients/hypothetical states and patients/real states. Take the statement ‘asthma interferes with getting a good night’s sleep’. If this is used in describing hypothetical states, we know that by definition that asthma is causing the sleep problem. But if this is used in patient self-report then how certain can we be that asthma is causing the sleep problem? We will not be asking for a purely factual statement, but for the patient’s guess as to what is causing the sleep problem. Maybe they have financial worries that are keeping them from sleeping well, and the worry and stress is also triggering asthma symptoms; does that count as asthma interfering with sleep?

Chapter 4 reported on a labelling study in which health states were valued by three groups of respondents that were identical, except that one group valued unlabelled states, one valued states labelled as cancer and the other valued states labelled as IBS. The inclusion of a cancer label was shown to significantly reduce the elicited health-state utility values and the impact differed depending on the severity of the state, whereas the inclusion of an IBS label had no impact on utility values. Further research is required to determine which other condition labels may potentially affect elicited utility values.

There are persuasive arguments that the difference in elicited values owing to a condition label represents greater accuracy in the value for that state. The condition label may provide more accuracy in the description of the health state that enables respondents to more accurately value the state. For example, respondents may place a different value on interference with social activities caused by needing to be near a toilet than as a result of undergoing chemotherapy. Furthermore, the condition label itself may affect the quality of life experienced in a particular health state. For example, a cancer health state may differ in its quality of life, although the health status is measured the same in terms of a classification system, such as the generic EQ-5D or cancer-specific EORTC-8D.

However, without qualitative research we cannot conclude that the differences in utility values owing to condition labels are not distortions arising from prior knowledge or preconceptions of the health state or factors irrelevant to valuing a health state. For example, the differences observed with the condition label could be caused by fear or dread associated with cancer that is not experienced by the patient (as reflected in the health-state description that will include emotional health), stigmatisation of patients with cancer or taking account of mortality. These factors would make the elicited TTO estimates biased. We therefore recommend the exclusion of condition labels from health-state descriptions, where practical, to ensure that utility values are not distorted by prior knowledge, experience or preconceptions of the condition. However, developing preference-based measures will be limited by the content of the original instrument and for many this includes the condition label. However, this may not matter for all conditions and further research is required to examine the extent of this problem. Qualitative research may demonstrate that the differences in values owing to a condition label represent greater accuracy rather than a distortion due to prior knowledge or preconceptions.

This raises the question of whether or not utility values used to inform resource allocation decisions should reflect this difference. We outlined in Chapter 4 that, as resource allocation distributes resources across different conditions and different patient groups, it is typically argued that there must be comparability in health-state descriptions irrespective of the underlying condition causing that health state. For example, the same health state experienced by a patient with cancer, IBS, heart disease, depression or diabetes should be given the same preference weighting in terms of its impact on utility. We therefore argue that health-state values used in economic evaluation should not reflect any difference in values due to a condition label. However, if qualitative research shows that differences in values due to a condition label are caused either by greater accuracy in the health-state description or the associated impact on the quality of life (rather than health per se) experienced in the condition, this implies that the utility value associated with a health state may legitimately vary by condition. These values could then be selected for use in economic evaluation on the grounds that they represent greater accuracy, and that comparability may be of secondary importance if comparability is met only by sacrificing accuracy. If it can be demonstrated using further research that the inclusion of a condition label in valuation studies enables greater accuracy in elicited utility values, this opens a new chapter in the debate of whether CSPBMs should be used in economic evaluation.

Side effects

An obvious limitation of condition-specific measures is that they may fail to pick up side effects of treatment. This could be dealt with by including additional dimensions to the condition-specific measure to cover known side effects, though this will miss any unknown side effects. This approach was examined in Chapter 5, in which a pain/discomfort dimension was added to the asthma-specific AQL-5D. Although these may not be common side effects of treatment, it provided an example of the impact of adding an extra dimension. The study found that the extra dimension had a significant impact on health-state values but that the impact was not additive. The net effect on a patient’s utility value depended on the severity of their state: the addition of pain/discomfort at level 1 (no pain/discomfort) or 2 (moderate pain/discomfort) significantly increased the mean health-state values in an asthma patient population, whereas level 3 pain/discomfort (extreme) reduced values. We also estimated a regression model of preference weights for the larger AQL-6D classification system and found that the additional dimension for pain/discomfort did impact on the condition-specific dimensions and that the impact was not consistent across dimensions. This implies that a CSPBMs would need to be completely revalued following the addition of a dimension to cover a side effect rather than simply adding the preference weights of an extra dimension to the existing value set.

Adding one or two dimensions would also reduce the practical advantage of basing a CSPBMs on an existing condition-specific measure. It would require extra data to be collected on the new dimension unless the extra dimension is based on another widely used measure (such as a generic measure).

Anchoring

Even for generic preference-based measures, the upper anchor used in the valuation task is the instrument-specific best state, and respondents may still imagine other health problems. So what is the respondent thinking about the dimensions that are not mentioned in the (condition-specific measure) health state they are being asked to value? For some dimensions they may not pay them any attention. Where they do think about them, respondents may simply assume that other dimensions are at their optimum level or that they are at the same level as their own health. However, anything other than ‘full health’ or ‘perfect health’ makes comparability between preference-based instruments problematic. Evidence from the add-on study suggests that respondents do not assume other dimensions are at full health. The addition of pain/discomfort at level 1 (no pain/discomfort) or 2 (moderate pain/discomfort) actually increased the mean health-state values in an asthma patient population; the addition of a physical health dimension at level 1 to the emotional component of the CORE-6D also increased their mean health-state value. This suggests that respondents were assuming there would be problems in other dimensions (rightly or wrongly). Our recommendation is to use a generic upper anchor in health-state valuation tasks to improve comparability between preference-based measures.

Focusing effects

Another possible concern with using CSPBMs is focusing effects. We tend to focus on those things that are placed in front of us. Respondents, therefore, will tend to focus on the problems described in the health state they are valuing rather than other aspects of their health or indeed other aspects of their life. This results in respondents exaggerating the importance of the problems associated with the condition being valued compared with other conditions. For any given health state, this suggests that CSPBMs may generate a larger decrement from any given dimension of health than a generic measure simply because the respondent is not being given the broader context. This also has important implications for comorbidities. Interestingly our research has suggested that respondents do think about other dimensions of health and so this may not be as important as initially suspected.

Comorbidities

Where there are comorbidities, the achievement of comparability between specific instruments requires the assumption that the impact of different dimensions on preferences is additive, whether or not they are included in the classification system. The impact of breathlessness on health-state values, for example, must be the same whether or not the patient has other health problems not covered by the classification system, such as joint pain. Otherwise, even in the scenario that the intervention only alters the dimensions covered in the specific instrument, the estimated change in health-state value may be incorrect due to preference dependence between dimensions included in the classification system of the specific measure and those dimensions not included. Having rheumatic pain, for example, may have an impact on the size of the health gain associated with the treatment of breathlessness.

It might have been expected that adding a comorbidity would reduce health-state values by a constant amount, regardless of the severity of the condition-specific state. The study in Chapter 5 has shown that the addition of an extra dimension at its worst level reduced the health-state values, as would be expected. However, it also showed that the addition of a dimension at the intermediate level or lowest level in most cases resulted in increases in health-state values. The AQL-5D/AQL-6D study went on to estimate the impact of the additional dimension for pain/discomfort on the coefficients of the other dimensions. The results were not consistent across dimensions and show that a simple additive adjustment would not adequately capture the effect of adding the pain/discomfort dimension to the classification system. The results from the CORE-6D study outlined in Chapter 5 were more consistent with a simple additive assumption.

Health-state utility values estimated from CSPBMs assume that the patient has only one condition. Where patients have comorbidities or where there are important side effects then our findings suggest that the values are not going to reflect the marginal impact of the condition. This is potentially a serious limitation for use within a condition as well for making cross-programme comparisons.

Conclusion

Respondents in health-state valuation studies struggle with many pieces of information at once and so there is a practical constraint on the size of classification systems designed for preference-based measures. Any classification system that is amenable to valuation will exclude some dimensions of health and so there will also be gaps in the coverage of generic measures such as EQ-5D. Generic measures were not developed to provide a complete picture of a person’s HRQoL. 112 The decision to use a CSPBMs in cross-programme comparisons is ultimately a trade-off. The advantage of a measure that is more relevant and sensitive to those things that matter to patients with the condition needs to be compared with the disadvantages from excluding side effects of treatment, distortions created by focusing effects and the potential loss of comparability from preference interactions with dimensions not covered by the narrower focus of the CSPBMs.

This trade-off will vary between condition-specific measures and CSPBMs. Some measures will be less prone to problems such as focusing effects and comorbidities, as they contain generic dimensions of HRQoL, such as the EORTC QLQ-C30 and preference-based EORTC-8D derived from it. Others will be more likely to suffer from these effects, such as those measures that focus on a narrow range of symptoms. More work is needed to explore the likely size of these effects and how they compare with any gain from using a particular measure. We believe that the priority for future research is to focus on how the focus, dimensionality and framing of the classification system impacts on health-state utility values and to provide recommendations for future studies developing preference-based measures.

Limitations

The limitations of each study are included in each chapter. There are some overall limitations of the project that are summarised here. First, it is often not possible to generalise across all CSPBMs; for example, some focus on symptoms and may be unable to capture side effects and comorbidities, whereas some focus more on HRQoL and it is realistic to expect they are able to capture side effects and comorbidities. Furthermore, for some conditions labelling may have an impact, while for others it may not. Likewise, some measures may include all important dimensions, whereas for others there may be important dimensions missing. Second, the lack of detail and clarity in the reporting of the development of CSPBMs makes comparisons across the development of measures difficult and provides significant challenges for the methodology in the derivation of the classification system to be understood and able to evolve to become more scientifically rigorous. Finally, it is important to examine the performance of CSPBMs in comparison with generic preference-based measures to determine whether they do have an advantage and to determine the impact on the results if these values are used in economic evaluation; however, there are few available data that can be used to examine this and the analysis conducted here has been constrained by this.

Unavailability of generic data

The lack of generic data arises from the failure to use a generic preference-based measures in key clinical trials or other studies. However, the lack of data collected in specific trials may not demonstrate a lack of availability for populating an economic model. There could be other related studies or routine sources that provide the necessary evidence on the values for the key states used in the model that may be undertaken to support the submission, or the values might be identified by a systematic search of the literature. In some situations, an alternative approach to deriving health-state values will be to map the condition-specific measure (and other variables) on to the relevant generic measure (see Brazier and Tsuchiya113 for an overview of mapping). Where data sets containing both measures in a relevant patient sample are available then mapping may be a quicker and cheaper solution. Mapping has its own problems and its success varies between conditions and instruments, and current practice tends to ignore the uncertainty arising from the mapping function itself. 114 Where there are no means of obtaining relevant health-state utility values using generic measures, then a CSPBM will be the only solution.

Generic measures are not appropriate

Pharmacoeconomic guidelines often recommend the use of generic measures for economic evaluation, and in the case of NICE there is a strong preference for one generic measure (i.e. EQ-5D). 6 However, no agencies rule out the use of CSPBMs. In the case of NICE, alternative methods are permitted where the EQ-5D has been shown to be inappropriate. One approach is to consider the appropriateness of a generic preference-based measure for the patient group in terms of psychometric criteria, such as practicality, reliability, validity and responsiveness. 6 Given the problems with CSPBMs, this would seem to be a sensible approach and one we will adopt here. The assessment of the appropriateness of EQ-5D has recently been addressed in some detail in a NICE DSU Technical Support Document,115 so here we summarise the key concerns.

The practicality of an instrument depends on its acceptability to respondents. All of the generic measures are quite short and for most patients are quite modest in burden compared with other questionnaires and clinical assessments that they have to endure. However, there may be concerns in certain populations with whether it is possible for patients to meaningfully respond, such as when they are extremely ill or cognitively impaired (e.g. case of dementia). In these cases, proxy responses can be used and this problem would apply as much to CSPBMs as to generic measures.

Reliability is the ability of a measure to reproduce the same value on two separate administrations when there has been no change in health. This can be over time, between methods of administration or between raters. Evidence on the reliability of the generic instruments does indicate significant random variation between assessments, but again this is no more than would be found with CSPBMs. 5

The assessment of validity is far more problematic. The primary difficulty is the lack of a gold standard measure of health-state utilities. The challenge of assessing validity pervades the measurement of all psychological phenomena and has been met in the psychometric literature by the development of various tests that have been adapted for use with preference-based measures. 13,115 The validity of a preference-based measure is the product of the classification system and the methods of valuation. Assuming the methods of valuation are acceptable, then the issue of validity is concerned with the classification system that defines the coverage and sensitivity of the instrument [although most available evidence is concerned with the index (Papaioannou et al. 11)].

The validity of the content of a measure depends on the extent to which it covers the areas of quality of life that is likely to be altered by a condition and its treatment. This can be assessed using qualitative methods such as in-depth interviews and focus groups of patients, and this is the approach recommended by the US FDA12 for patient-reported outcome measures to support labelling claims. The face validity of a classification system can be assessed using cognitive interview techniques to establish whether patients, for example, understand the descriptions in the way they are intended to be understood. Such evidence will identify a potential problem but will not be definitive, as an absent dimension may be picked up by the other dimensions, particularly generic dimensions such as well-being and social activities.

An important quantitative test or series of tests comes under the term construct validity. This is a method for testing empirically the extent to which a measure agrees with other measures or indicators of the dimensions of HRQoL considered relevant to the patient group (such as those identified by qualitative work). There are two commonly used approaches in the psychometric literature to examining construct validity of a standardised classification system. One approach is to examine whether it is able to differentiate between groups thought to differ in terms of their health (i.e. known group differences) and the other is the extent to which it correlates with another measure of health (i.e. convergent validity). These tests provide evidence on the degree to which a measure is valid at measuring the concept being tested. A related test is responsiveness, which is the ability to respond to known changes in HRQoL.

There are two weaknesses in the use of such evidence. One is that the construct variable used to define ‘known group differences’ may not be valid. This will be particularly the case where clinical measures are used such as visual acuity, respiratory function or symptoms of schizophrenia that may have only a weak relationship to HRQoL in any case. For responsiveness, the alleged change in health can come from assumed changes before and after an intervention that may not have improved patient health. Second, validity is not dichotomous but a matter of degree, and deciding whether a measure is sufficiently valid is ultimately a matter of judgement. A CSPBM may achieve a larger difference or change as indicated by a standardised effect size [where the mean change in score is divided by either the SD at baseline or the SD of the change (Katz et al. 116)], for example, but effect sizes do not indicate the value or importance of a change.

Where the classification systems of generic measures seems to fail to pick up differences or changes suggested by the condition-specific measure, or at least suggests a much smaller difference, then there needs to be some other evidence that this is likely to be important to the general public. A study, for example, could be undertaken to establish whether a dimension of HRQoL excluded from the generic measures is important to members of the general public.

This careful assessment of the evidence provides the basis for deciding whether it is worth developing a full CSPBM. The next question is which condition-specific measure should be used.

Valuation

To improve comparability between condition-specific measures, it is important that they are valued using the same methods. This is difficult to achieve, as there is no international co-ordination of this effort. However, when developing CSPBMs thought should be given to the likely use of the evidence and where it is going to be used.

Demonstrating the impact of using condition-specific preference-based measures

Given the problems with CSPBMs, it is important to demonstrate the advantages that any one measure has over existing generic measures. This would mean replicating the types of analyses performed in Chapter 6, in which we compared the performance of the CSPBMs with generic measures in terms of validity and responsiveness. The results in some cases suggested that the CSPBM was not better. In other cases, the CSPBMs did offer greater sensitivity and so policy-makers would need to decide whether this is worth the potential disadvantages in reduced comparability across programmes. This is a judgement that will be specific to the condition, the CSPBM and the advantages it appears to have over the preferred generic measures. However, it is worth noting that all CSPBMs produced similar results to EQ-5D, suggesting that the choice of measure may have minimal impact on results. In fact our results in Chapter 6 suggest that mean utility change and SD of the change may actually be smaller for the CSPBMs than EQ-5D for the measures examined in the report.

The add-on agenda

Research is under way looking at the potential role of adding dimensions to the EQ-5D to improve its relevance to specific conditions. 90 This would avoid or at least reduce some of the problems associated with CSPBMs. However, it requires the additional modules to be developed and valued and for the ‘EQ-5D-plus’ measure to be used in clinical studies. Furthermore, the number of extra dimensions that can be added at one time is limited by respondents’ ability to value large health states. There is important empirical work to be done to examine the scope for adding dimensions to common generic measures and to see whether this approach overcomes the limitations of generic measures without the need to use CSPBMs. This research suggests that it will not be possible to assume that the extra dimensions simply have an additive impact on existing EQ-5D value sets.

Health-state classification

• Use explicit criteria and methods for implementing those criteria in the development of a health-state classification system (stages I–III).

• Not to use condition labels in the health-state classification system (where possible) to avoid any potential distortions at the valuation stage due to prior knowledge, preconceptions or irrelevant information about future prognosis.

• Incorporate important side effects of treatment or comorbidities in the patient population into the health-state classification system to avoid potential inaccuracies at the valuation stage. This can be undertaken as add-ons for existing CSPBMs.

• A health-state classification system developed from an existing measure should be validated on another data set (stage IV).

Valuation

• To enhance comparability it is helpful for the CSPBMs to use the same valuation methods (stage V) as used for the generic and other CSPBMs with which it is likely to be compared.

• The Rasch vignette approach rather than a conventional statistical design should be used to generate the states for valuation where the domains of an instrument are highly correlated and form a unidimensional scale (stage VI).

For further guidance on other aspects of the development of CSPBMs – including methods of eliciting values, the sources of values and the modelling of health-state values – readers should refer to the broader literature (see Brazier et al. 5 for an overview).

All methods should be fully reported.

Stage 1: To identify and critically review the current methods for developing preference-based measures of quality of life from condition specific measures

This stage will critically review published studies of theoretical and empirical work on the development of preference-based measures from CSMs and other non-preference-based measures of health. From the review, the following will be developed: a list of questions to address to decide whether it is worth deriving a preference-based CSM; a list of the methodological challenges (in addition to those identified above); methods for deriving health states descriptions amenable to valuation studies from CSMs; and a framework for testing resultant preference-based measures.

Stage 2: To develop and test new or revised methods to compensate for the flaws of existing methods

Objectives 1 and 2 will be addressed by the review of the literature in stage 1. The remaining six objectives will be addressed by three empirical phases of work.

Valuation surveys

A common format will be used for each of the three valuation surveys. The valuation surveys will all employ the time trade-off method (TTO), where respondents are asked how many years they would be willing to sacrifice in order to be in full health.

The representative sample of the general population will be asked firstly to complete the classification for their own health state for the relevant instrument and secondly to undertake a warm-up ranking task and eight TTO valuations of health states. For the CORE-OM valuation survey the warm-up ranking task will involve four cards which will then be valued using TTO, subsequently there will be a second ranking task involving four cards which will then be valued using TTO. The MVH group version of TTO will be used to allow comparison with the EQ-5D tariff. 43 This valuation protocol was also used in the AQL-5D, OAB-5D and EORTC QLQ-C30 valuation studies. Respondents will also be asked a number of background questions covering health, demographic and socio-economic characteristics. Similar interview schedules have been successfully used in a large scale general population survey undertaken in the UK43 and by ScHARR in a number of studies. Each interview is expected to take about 30 minutes. A small pilot study of 20 respondents will be undertaken prior to each valuation study to check respondents’ understanding of the TTO and to check that they are completing each task as expected.

The sample sizes for the three surveys are as follows:

1. This study has been designed to value 24 out of the possible 54 CORE-OM states (see above), to ensure an adequate mix of mental health states with and without mobility and pain problems. These states will be divided into four blocs of eight health states, one for each respondent to value. The valuation of these 24 states has two aims. One is to produce mean estimates for each health state. The second is to compare mean values between the states with and without the addition of mobility and pain using simple t-tests. Assuming a power of 0.8, significance level of 0.05, standard deviation of 0.3 and an expected difference of 0.1, then this requires a sample of 73 valuations for each state and a total of 220 completed interviews.

2. The EORTC-8D labelling study is concerned with testing the impact on health state values of removing the name of the cause of the problem. Mean values across eight states will be compared using simple t-tests and this requires a sample of 73 valuations for each state (see justification in 1). The previous EORTC-8D valuation study had approx. 30 values per state. So this requires an additional 43 valuations of the eight states with no label and 146 interviews for health states with two different labels (73 per label). A minimum of 219 completed interviews will be required.

3. The valuation of AQL-5D and the classification enhanced with a pain dimension requires 35 health states to be valued in total (see above). Each state will be valued 30 times producing a minimum of 132 completed interviews. This number has not been selected to enable comparisons across states, but to allow the estimation of a preference model.

Description of the outcome measures

There are four condition specific measures that will form the subject of this research.

Participants in valuation surveys

The research will be carried out across the UK. It will involve: (i) random selection using the PAF register of addresses in the UK of addresses within Yorkshire; (ii) Households will be approached in writing, introducing the study, the purpose of recruitment, asking for their participation and informing that an interviewer may call; and (iii) At each address the interviewer will identify themselves and request an interview, either with the person there, or with another person in the household if they are needed to fulfil the quota, or both. The interviewer will then obtain consent ensuring the member of the public has read the information form.

Prioritisation Group

1. Director, NIHR HTA programme, Professor of Clinical Pharmacology, Department of Pharmacology and Therapeutics, University of Liverpool

2. Professor Imti Choonara, Professor in Child Health, Academic Division of Child Health, University of Nottingham

   Chair – Pharmaceuticals Panel

3. Dr Bob Coates, Consultant Advisor – Disease Prevention Panel

4. Dr Andrew Cook, Consultant Advisor – Intervention Procedures Panel

5. Dr Peter Davidson, Director of NETSCC, Health Technology Assessment

6. Dr Nick Hicks, Consultant Adviser – Diagnostic Technologies and Screening Panel, Consultant Advisor–Psychological and Community Therapies Panel

7. Ms Susan Hird, Consultant Advisor, External Devices and Physical Therapies Panel

8. Professor Sallie Lamb, Director, Warwick Clinical Trials Unit, Warwick Medical School, University of Warwick

   Chair – HTA Clinical Evaluation and Trials Board

9. Professor Jonathan Michaels, Professor of Vascular Surgery, Sheffield Vascular Institute, University of Sheffield

   Chair – Interventional Procedures Panel

10. Professor Ruairidh Milne, Director – External Relations

11. Dr John Pounsford, Consultant Physician, Directorate of Medical Services, North Bristol NHS Trust

    Chair – External Devices and Physical Therapies Panel

12. Dr Vaughan Thomas, Consultant Advisor – Pharmaceuticals Panel, Clinical

    Lead – Clinical Evaluation Trials Prioritisation Group

13. Professor Margaret Thorogood, Professor of Epidemiology, Health Sciences Research Institute, University of Warwick

    Chair – Disease Prevention Panel

14. Professor Lindsay Turnbull, Professor of Radiology, Centre for the MR Investigations, University of Hull

    Chair – Diagnostic Technologies and Screening Panel

15. Professor Scott Weich, Professor of Psychiatry, Health Sciences Research Institute, University of Warwick

    Chair – Psychological and Community Therapies Panel

16. Professor Hywel Williams, Director of Nottingham Clinical Trials Unit, Centre of Evidence-Based Dermatology, University of Nottingham

    Chair – HTA Commissioning Board

    Deputy HTA Programme Director

HTA Commissioning Board

1. Professor of Dermato-Epidemiology, Centre of Evidence-Based Dermatology, University of Nottingham

2. Department of Public Health and Epidemiology, University of Birmingham

3. Professor of Clinical Pharmacology, Director, NIHR HTA programme, Department of Pharmacology and Therapeutics, University of Liverpool

1. Professor Judith Bliss, Director of ICR-Clinical Trials and Statistics Unit, The Institute of Cancer Research

2. Professor David Fitzmaurice, Professor of Primary Care Research, Department of Primary Care Clinical Sciences, University of Birmingham

3. Professor John W Gregory, Professor in Paediatric Endocrinology, Department of Child Health, Wales School of Medicine, Cardiff University

4. Professor Steve Halligan, Professor of Gastrointestinal Radiology, Department of Specialist Radiology, University College Hospital, London

5. Professor Angela Harden, Professor of Community and Family Health, Institute for Health and Human Development, University of East London

6. Dr Martin J Landray, Reader in Epidemiology, Honorary Consultant Physician, Clinical Trial Service Unit, University of Oxford

7. Dr Joanne Lord, Reader, Health Economics Research Group, Brunel University

8. Professor Stephen Morris, Professor of Health Economics, University College London, Research Department of Epidemiology and Public Health, University College London

9. Professor Dion Morton, Professor of Surgery, Academic Department of Surgery, University of Birmingham

10. Professor Gail Mountain, Professor of Health Services Research, Rehabilitation and Assistive Technologies Group, University of Sheffield

11. Professor Irwin Nazareth, Professor of Primary Care and Head of Department, Department of Primary Care and Population Sciences, University College London

12. Professor E Andrea Nelson, Professor of Wound Healing and Director of Research, School of Healthcare, University of Leeds

13. Professor John David Norrie, Director, Centre for Healthcare Randomised Trials, Health Services Research Unit, University of Aberdeen

14. Dr Rafael Perera, Lecturer in Medical Statisitics, Department of Primary Health Care, University of Oxford

15. Professor Barney Reeves, Professorial Research Fellow in Health Services Research, Department of Clinical Science, University of Bristol

16. Professor Peter Tyrer, Professor of Community Psychiatry, Centre for Mental Health, Imperial College London

17. Professor Martin Underwood, Professor of Primary Care Research, Warwick Medical School, University of Warwick

18. Professor Caroline Watkins, Professor of Stroke and Older People’s Care, Chair of UK Forum for Stroke Training, Stroke Practice Research Unit, University of Central Lancashire

19. Dr Duncan Young, Senior Clinical Lecturer and Consultant, Nuffield Department of Anaesthetics, University of Oxford

1. Dr Tom Foulks, Medical Research Council

2. Dr Kay Pattison, Senior NIHR Programme Manager, Department of Health

HTA Clinical Evaluation and Trials Board

1. Director, Warwick Clinical Trials Unit, Warwick Medical School, University of Warwick and Professor of Rehabilitation, Nuffield Department of Orthopaedic, Rheumatology and Musculoskeletal Sciences, University of Oxford

2. Professor of the Psychology of Health Care, Leeds Institute of Health Sciences, University of Leeds

3. Director, NIHR HTA programme, Professor of Clinical Pharmacology, University of Liverpool

1. Professor Keith Abrams, Professor of Medical Statistics, Department of Health Sciences, University of Leicester

2. Professor Martin Bland, Professor of Health Statistics, Department of Health Sciences, University of York

3. Professor Jane Blazeby, Professor of Surgery and Consultant Upper GI Surgeon, Department of Social Medicine, University of Bristol

4. Professor Julia M Brown, Director, Clinical Trials Research Unit, University of Leeds

5. Professor Alistair Burns, Professor of Old Age Psychiatry, Psychiatry Research Group, School of Community-Based Medicine, The University of Manchester & National Clinical Director for Dementia, Department of Health

6. Dr Jennifer Burr, Director, Centre for Healthcare Randomised trials (CHART), University of Aberdeen

7. Professor Linda Davies, Professor of Health Economics, Health Sciences Research Group, University of Manchester

8. Professor Simon Gilbody, Prof of Psych Medicine and Health Services Research, Department of Health Sciences, University of York

9. Professor Steven Goodacre, Professor and Consultant in Emergency Medicine, School of Health and Related Research, University of Sheffield

10. Professor Dyfrig Hughes, Professor of Pharmacoeconomics, Centre for Economics and Policy in Health, Institute of Medical and Social Care Research, Bangor University

11. Professor Paul Jones, Professor of Respiratory Medicine, Department of Cardiac and Vascular Science, St George‘s Hospital Medical School, University of London

12. Professor Khalid Khan, Professor of Women’s Health and Clinical Epidemiology, Barts and the London School of Medicine, Queen Mary, University of London

13. Professor Richard J McManus, Professor of Primary Care Cardiovascular Research, Primary Care Clinical Sciences Building, University of Birmingham

14. Professor Helen Rodgers, Professor of Stroke Care, Institute for Ageing and Health, Newcastle University

15. Professor Ken Stein, Professor of Public Health, Peninsula Technology Assessment Group, Peninsula College of Medicine and Dentistry, Universities of Exeter and Plymouth

16. Professor Jonathan Sterne, Professor of Medical Statistics and Epidemiology, Department of Social Medicine, University of Bristol

17. Mr Andy Vail, Senior Lecturer, Health Sciences Research Group, University of Manchester

18. Professor Clare Wilkinson, Professor of General Practice and Director of Research North Wales Clinical School, Department of Primary Care and Public Health, Cardiff University

19. Dr Ian B Wilkinson, Senior Lecturer and Honorary Consultant, Clinical Pharmacology Unit, Department of Medicine, University of Cambridge

1. Ms Kate Law, Director of Clinical Trials, Cancer Research UK

2. Dr Morven Roberts, Clinical Trials Manager, Health Services and Public Health Services Board, Medical Research Council

Diagnostic Technologies and Screening Panel

1. Scientific Director of the Centre for Magnetic Resonance Investigations and YCR Professor of Radiology, Hull Royal Infirmary

2. Professor Judith E Adams, Consultant Radiologist, Manchester Royal Infirmary, Central Manchester & Manchester Children’s University Hospitals NHS Trust, and Professor of Diagnostic Radiology, University of Manchester

3. Mr Angus S Arunkalaivanan, Honorary Senior Lecturer, University of Birmingham and Consultant Urogynaecologist and Obstetrician, City Hospital, Birmingham

4. Dr Diana Baralle, Consultant and Senior Lecturer in Clinical Genetics, University of Southampton

5. Dr Stephanie Dancer, Consultant Microbiologist, Hairmyres Hospital, East Kilbride

6. Dr Diane Eccles, Professor of Cancer Genetics, Wessex Clinical Genetics Service, Princess Anne Hospital

7. Dr Trevor Friedman, Consultant Liason Psychiatrist, Brandon Unit, Leicester General Hospital

8. Dr Ron Gray, Consultant, National Perinatal Epidemiology Unit, Institute of Health Sciences, University of Oxford

9. Professor Paul D Griffiths, Professor of Radiology, Academic Unit of Radiology, University of Sheffield

10. Mr Martin Hooper, Public contributor

11. Professor Anthony Robert Kendrick, Associate Dean for Clinical Research and Professor of Primary Medical Care, University of Southampton

12. Dr Nicola Lennard, Senior Medical Officer, MHRA

13. Dr Anne Mackie, Director of Programmes, UK National Screening Committee, London

14. Mr David Mathew, Public contributor

15. Dr Michael Millar, Consultant Senior Lecturer in Microbiology, Department of Pathology & Microbiology, Barts and The London NHS Trust, Royal London Hospital

16. Mrs Una Rennard, Public contributor

17. Dr Stuart Smellie, Consultant in Clinical Pathology, Bishop Auckland General Hospital

18. Ms Jane Smith, Consultant Ultrasound Practitioner, Leeds Teaching Hospital NHS Trust, Leeds

19. Dr Allison Streetly, Programme Director, NHS Sickle Cell and Thalassaemia Screening Programme, King’s College School of Medicine

20. Dr Matthew Thompson, Senior Clinical Scientist and GP, Department of Primary Health Care, University of Oxford

21. Dr Alan J Williams, Consultant Physician, General and Respiratory Medicine, The Royal Bournemouth Hospital

1. Dr Tim Elliott, Team Leader, Cancer Screening, Department of Health

2. Dr Joanna Jenkinson, Board Secretary, Neurosciences and Mental Health Board (NMHB), Medical Research Council

3. Professor Julietta Patrick, Director, NHS Cancer Screening Programme, Sheffield

4. Dr Kay Pattison, Senior NIHR Programme Manager, Department of Health

5. Professor Tom Walley, CBE, Director, NIHR HTA programme, Professor of Clinical Pharmacology, University of Liverpool

6. Dr Ursula Wells, Principal Research Officer, Policy Research Programme, Department of Health

Disease Prevention Panel

1. Professor of Epidemiology, University of Warwick Medical School, Coventry

2. Dr Robert Cook, Clinical Programmes Director, Bazian Ltd, London

3. Dr Colin Greaves, Senior Research Fellow, Peninsula Medical School (Primary Care)

4. Mr Michael Head, Public contributor

5. Professor Cathy Jackson, Professor of Primary Care Medicine, Bute Medical School, University of St Andrews

6. Dr Russell Jago, Senior Lecturer in Exercise, Nutrition and Health, Centre for Sport, Exercise and Health, University of Bristol

7. Dr Julie Mytton, Consultant in Child Public Health, NHS Bristol

8. Professor Irwin Nazareth, Professor of Primary Care and Director, Department of Primary Care and Population Sciences, University College London

9. Dr Richard Richards, Assistant Director of Public Health, Derbyshire County Primary Care Trust

10. Professor Ian Roberts, Professor of Epidemiology and Public Health, London School of Hygiene & Tropical Medicine

11. Dr Kenneth Robertson, Consultant Paediatrician, Royal Hospital for Sick Children, Glasgow

12. Dr Catherine Swann, Associate Director, Centre for Public Health Excellence, NICE

13. Mrs Jean Thurston, Public contributor

14. Professor David Weller, Head, School of Clinical Science and Community Health, University of Edinburgh

1. Ms Christine McGuire, Research & Development, Department of Health

2. Dr Kay Pattison, Senior NIHR Programme Manager, Department of Health

3. Professor Tom Walley, CBE, Director, NIHR HTA programme, Professor of Clinical Pharmacology, University of Liverpool

External Devices and Physical Therapies Panel

1. Consultant Physician North Bristol NHS Trust

2. Reader in Wound Healing and Director of Research, University of Leeds

3. Professor Bipin Bhakta, Charterhouse Professor in Rehabilitation Medicine, University of Leeds

4. Mrs Penny Calder, Public contributor

5. Dr Dawn Carnes, Senior Research Fellow, Barts and the London School of Medicine and Dentistry

6. Dr Emma Clark, Clinician Scientist Fellow & Cons. Rheumatologist, University of Bristol

7. Mrs Anthea De Barton-Watson, Public contributor

8. Professor Nadine Foster, Professor of Musculoskeletal Health in Primary Care Arthritis Research, Keele University

9. Dr Shaheen Hamdy, Clinical Senior Lecturer and Consultant Physician, University of Manchester

10. Professor Christine Norton, Professor of Clinical Nursing Innovation, Bucks New University and Imperial College Healthcare NHS Trust

11. Dr Lorraine Pinnigton, Associate Professor in Rehabilitation, University of Nottingham

12. Dr Kate Radford, Senior Lecturer (Research), University of Central Lancashire

13. Mr Jim Reece, Public contributor

14. Professor Maria Stokes, Professor of Neuromusculoskeletal Rehabilitation, University of Southampton

15. Dr Pippa Tyrrell, Senior Lecturer/Consultant, Salford Royal Foundation Hospitals’ Trust and University of Manchester

16. Dr Nefyn Williams, Clinical Senior Lecturer, Cardiff University

1. Dr Kay Pattison, Senior NIHR Programme Manager, Department of Health

2. Dr Morven Roberts, Clinical Trials Manager, Health Services and Public Health Services Board, Medical Research Council

3. Professor Tom Walley, CBE, Director, NIHR HTA programme, Professor of Clinical Pharmacology, University of Liverpool

4. Dr Ursula Wells, Principal Research Officer, Policy Research Programme, Department of Health

Interventional Procedures Panel

1. Professor of Vascular Surgery, University of Sheffield

2. Consultant Colorectal Surgeon, Bristol Royal Infirmary

3. Mrs Isabel Boyer, Public contributor

4. Mr Sankaran Chandra Sekharan, Consultant Surgeon, Breast Surgery, Colchester Hospital University NHS Foundation Trust

5. Professor Nicholas Clarke, Consultant Orthopaedic Surgeon, Southampton University Hospitals NHS Trust

6. Ms Leonie Cooke, Public contributor

7. Mr Seumas Eckford, Consultant in Obstetrics & Gynaecology, North Devon District Hospital

8. Professor Sam Eljamel, Consultant Neurosurgeon, Ninewells Hospital and Medical School, Dundee

9. Dr Adele Fielding, Senior Lecturer and Honorary Consultant in Haematology, University College London Medical School

10. Dr Matthew Hatton, Consultant in Clinical Oncology, Sheffield Teaching Hospital Foundation Trust

11. Dr John Holden, General Practitioner, Garswood Surgery, Wigan

12. Dr Fiona Lecky, Senior Lecturer/Honorary Consultant in Emergency Medicine, University of Manchester/Salford Royal Hospitals NHS Foundation Trust

13. Dr Nadim Malik, Consultant Cardiologist/Honorary Lecturer, University of Manchester

14. Mr Hisham Mehanna, Consultant & Honorary Associate Professor, University Hospitals Coventry & Warwickshire NHS Trust

15. Dr Jane Montgomery, Consultant in Anaesthetics and Critical Care, South Devon Healthcare NHS Foundation Trust

16. Professor Jon Moss, Consultant Interventional Radiologist, North Glasgow Hospitals University NHS Trust

17. Dr Simon Padley, Consultant Radiologist, Chelsea & Westminster Hospital

18. Dr Ashish Paul, Medical Director, Bedfordshire PCT

19. Dr Sarah Purdy, Consultant Senior Lecturer, University of Bristol

20. Dr Matthew Wilson, Consultant Anaesthetist, Sheffield Teaching Hospitals NHS Foundation Trust

21. Professor Yit Chiun Yang, Consultant Ophthalmologist, Royal Wolverhampton Hospitals NHS Trust

1. Dr Kay Pattison, Senior NIHR Programme Manager, Department of Health

2. Dr Morven Roberts, Clinical Trials Manager, Health Services and Public Health Services Board, Medical Research Council

3. Professor Tom Walley, CBE, Director, NIHR HTA programme, Professor of Clinical Pharmacology, University of Liverpool

4. Dr Ursula Wells, Principal Research Officer, Policy Research Programme, Department of Health

Pharmaceuticals Panel

1. Professor in Child Health, University of Nottingham

2. Senior Lecturer in Clinical Pharmacology, University of East Anglia

3. Dr Martin Ashton-Key, Medical Advisor, National Commissioning Group, NHS London

4. Dr Peter Elton, Director of Public Health, Bury Primary Care Trust

5. Dr Ben Goldacre, Research Fellow, Division of Psychological Medicine and Psychiatry, King’s College London

6. Dr James Gray, Consultant Microbiologist, Department of Microbiology, Birmingham Children’s Hospital NHS Foundation Trust

7. Dr Jurjees Hasan, Consultant in Medical Oncology, The Christie, Manchester

8. Dr Carl Heneghan, Deputy Director Centre for Evidence-Based Medicine and Clinical Lecturer, Department of Primary Health Care, University of Oxford

9. Dr Dyfrig Hughes, Reader in Pharmacoeconomics and Deputy Director, Centre for Economics and Policy in Health, IMSCaR, Bangor University

10. Dr Maria Kouimtzi, Pharmacy and Informatics Director, Global Clinical Solutions, Wiley-Blackwell

11. Professor Femi Oyebode, Consultant Psychiatrist and Head of Department, University of Birmingham

12. Dr Andrew Prentice, Senior Lecturer and Consultant Obstetrician and Gynaecologist, The Rosie Hospital, University of Cambridge

13. Ms Amanda Roberts, Public contributor

14. Dr Gillian Shepherd, Director, Health and Clinical Excellence, Merck Serono Ltd

15. Mrs Katrina Simister, Assistant Director New Medicines, National Prescribing Centre, Liverpool

16. Professor Donald Singer, Professor of Clinical Pharmacology and Therapeutics, Clinical Sciences Research Institute, CSB, University of Warwick Medical School

17. Mr David Symes, Public contributor

18. Dr Arnold Zermansky, General Practitioner, Senior Research Fellow, Pharmacy Practice and Medicines Management Group, Leeds University

1. Dr Kay Pattison, Senior NIHR Programme Manager, Department of Health

2. Mr Simon Reeve, Head of Clinical and Cost-Effectiveness, Medicines, Pharmacy and Industry Group, Department of Health

3. Dr Heike Weber, Programme Manager, Medical Research Council

4. Professor Tom Walley, CBE, Director, NIHR HTA programme, Professor of Clinical Pharmacology, University of Liverpool

5. Dr Ursula Wells, Principal Research Officer, Policy Research Programme, Department of Health

Psychological and Community Therapies Panel

1. Professor of Psychiatry, University of Warwick, Coventry

2. Consultant & University Lecturer in Psychiatry, University of Cambridge

3. Professor Jane Barlow, Professor of Public Health in the Early Years, Health Sciences Research Institute, Warwick Medical School

4. Dr Sabyasachi Bhaumik, Consultant Psychiatrist, Leicestershire Partnership NHS Trust

5. Mrs Val Carlill, Public contributor

6. Dr Steve Cunningham, Consultant Respiratory Paediatrician, Lothian Health Board

7. Dr Anne Hesketh, Senior Clinical Lecturer in Speech and Language Therapy, University of Manchester

8. Dr Peter Langdon, Senior Clinical Lecturer, School of Medicine, Health Policy and Practice, University of East Anglia

9. Dr Yann Lefeuvre, GP Partner, Burrage Road Surgery, London

10. Dr Jeremy J Murphy, Consultant Physician and Cardiologist, County Durham and Darlington Foundation Trust

11. Dr Richard Neal, Clinical Senior Lecturer in General Practice, Cardiff University

12. Mr John Needham, Public contributor

13. Ms Mary Nettle, Mental Health User Consultant

14. Professor John Potter, Professor of Ageing and Stroke Medicine, University of East Anglia

15. Dr Greta Rait, Senior Clinical Lecturer and General Practitioner, University College London

16. Dr Paul Ramchandani, Senior Research Fellow/Cons. Child Psychiatrist, University of Oxford

17. Dr Karen Roberts, Nurse/Consultant, Dunston Hill Hospital, Tyne and Wear

18. Dr Karim Saad, Consultant in Old Age Psychiatry, Coventry and Warwickshire Partnership Trust

19. Dr Lesley Stockton, Lecturer, School of Health Sciences, University of Liverpool

20. Dr Simon Wright, GP Partner, Walkden Medical Centre, Manchester

1. Dr Kay Pattison, Senior NIHR Programme Manager, Department of Health

2. Dr Morven Roberts, Clinical Trials Manager, Health Services and Public Health Services Board, Medical Research Council

3. Professor Tom Walley, CBE, Director, NIHR HTA programme, Professor of Clinical Pharmacology, University of Liverpool

4. Dr Ursula Wells, Principal Research Officer, Policy Research Programme, Department of Health