Weighting and valuing quality-adjusted life-years using stated preference methods: preliminary results from the Social Value of a QALY Project

Article history

The research reported in this issue of the journal was commissioned by the National Coordinating Centre for Research Methodology (NCCRM), and was formally transferred to the HTA programme in April 2007 under the newly established NIHR Methodology Panel. The HTA programme project number is 06/91/05. The contractual start date was in October 2004. The draft report began editorial review in March 2009 and was accepted for publication in June 2009. The commissioning brief was devised by the NCCRM who specified the research question and study design. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The HTA editors and publisher have tried to ensure the accuracy of the authors’ report and would like to thank the referees for their constructive comments on the draft document. However, they do not accept liability for damages or losses arising from material published in this report.

Declared competing interests of authors

None

Copyright statement

© 2010 Queen’s Printer and Controller of HMSO. This journal is a member of and subscribes to the principles of the Committee on Publication Ethics (COPE) (http://www.publicationethics.org/). This journal may be freely reproduced for the purposes of private research and study and may be included in professional journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be addressed to: NETSCC, Health Technology Assessment, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK.

2010 Queen’s Printer and Controller of HMSO

Estimating relative weights for QALYs

Several authors have discussed the theoretical, ethical and practical issues around distributional weights for QALYs,11–13 and there have been a number of attempts to estimate weights. 11,14,15

Dolan et al. 16 provide the most comprehensive review in this field to date. Using a ‘citation pearl growing’ search strategy, they identify 78 papers, dated 2001 or earlier, of which 64 include empirical data. There is growing evidence from (mainly survey-based) studies of the general population that the number of QALYs gained is likely to be traded off against other factors. Efforts to identify these other factors have indicated a wide range of possibilities, but there remain inconsistencies and contradictory findings. The list of factors identified by Dolan et al. (drawn from both the empirical and theoretical literature) are as follows: age, severity of illness (or starting point health state), end point/final health state, culpability/responsibility for ill health, having dependants, socioeconomic characteristics, gender, ethnicity, inequalities in health, and the concentration or dispersion of the distribution of a fixed health gain. The reviewers conclude that despite a growing body of literature, there are contradictory findings, many studies involve small samples and few attempt to estimate weights. Earlier, Schwappach17 described two categories of factors that could influence social value: (1) characteristics of beneficiaries, and (2) characteristics of the intervention’s effect on patients’ health, adding the factors of prior health consumption of patients, the duration of benefit and whether the gain in health is an improvement or the prevention of a decline.

In the period since these reviews, there have been further contributions to the literature. Two of these used discrete choice data, examining attributes such as age, culpability (e.g. related to alcohol consumption), expected length of survival, time on waiting list and whether a previous transplant had been received;18 and lifestyle, socioeconomic status, age, life expectancy, quality of life (QoL) after treatment and level of past use of health care. 19 The results indicate that several factors, in addition to health gain, influence people’s choices. However, unlike the studies described in this report, these studies were either condition specific or not based on a population sample of respondents.

Projects attempting to derive QALY weights are faced with three significant challenges: identifying characteristics of beneficiaries over which weights should be derived; designing and presenting questions so that respondents can understand complexities and make choices; and elicitation of quantitative preference data from members of the general public to allow the estimation of QALY weights.

Designing questionnaires that respondents can engage with was a particular challenge in this study because our aim was to estimate the relative value of different types of decontextualised, generic QALY gains. Without context, however, questions can seem overly abstract to respondents. There is also evidence that different ‘types’ of decontextualised QALY (e.g. life-saving or QoL-enhancing QALYs) will be regarded differently,20,21 and so the presentation of questions needed to be flexible enough to incorporate different QALY types.

Valuing QALYs in monetary terms

The concept of willingness to pay (WTP) has existed for a long time. 22,23 However, not until the 1980s did government Transport Departments worldwide consider using the method to value lives saved from safety projects, rather than the gross output (‘productivity’) approach used previously. 24 Arguably, the most natural measure of the extent of a person’s preference for anything is the maximum amount that they would be willing to pay for it. Under what has naturally come to be known as the ‘willingness-to-pay’ approach to valuation of safety, one seeks to establish the maximum amounts that those affected would individually be willing to pay for (typically small) improvements in their own and others’ safety. These amounts are then aggregated across individuals to arrive at an overall value for the safety improvement concerned, thus reflecting society’s overall resource constraint.

Estimating a WTP-based monetary value of a QALY can also be viewed as a group-aggregate WTP for marginal gains in health, at least in the case of a randomly-selected sample of the public. Indeed, this argument has been used in promoting an insurance-based approach to valuing publicly-provided health care, whereby respondents are informed of the probabilities of needing care, as well as of it being successful, before providing a valuation. 25,26

The WTP method was first applied in health to value heart attack prevention. 27 Subsequently, there were few such studies in health, probably as a result of the view that such monetary valuation was unethical. In addition, the use of WTP to inform decisions about allocation of health care, which is supposed to be on the basis of (some notion of) need, may look problematic because WTP is obviously associated with ability to pay. However, it has been shown that this need not impede the use of WTP in health economic evaluation28 and that, indeed, QALYs suffer from the same phenomenon. 29 Since the early 1990s, the feasibility of using WTP in health economics has again been recognised,25,30 and more studies have been undertaken. 31,32

Thus, in health, WTP methods historically addressed decision-making dilemmas assessing relative utility of treatments at two main levels: (1) for a given group of patients (involving elicitation of values from samples of such patients), and (2) across disparate programmes funded by geographically-defined health organisations (involving elicitation from the community of WTP values for each programme at stake). Methods have been developed which work well in terms of WTP values reflecting patient preferences. 33 In the latter area, methods have been more problematic, but are improving. 34,35 As in other public sector areas, results have been mixed on how sensitive WTP responses are to the size of the good (i.e. the health change/numbers treated) on offer to respondents36–38 and to other aspects related to ‘framing’ and programme information presented to respondents. 39,40 However, innovations in valuing safety improvements derived by Carthy et al. 41 have shown promise in overcoming these issues. Methods based on these developments, consistent with the notion of starting with ‘marginal’ gains (in this case, in health), are described below. For valuing a QALY, the challenge is to start with a health gain or detriment which is not so large as to hit respondents’ budget constraints, but not so small that respondents are unwilling to trade it off against a risky situation in the standard gamble (SG) type question, also involved in a procedure of the sort devised by Carthy et al. 41

Through the 1990s, development of national-level technology assessment agencies led to calls for monetary values of a QALY to aid decision making at a national level. 42,43 In the UK, there has been significant debate about the empirical basis of the cost-per-QALY threshold above/below which NICE would recommend rejection/adoption of a therapy by the NHS. For example, the proceedings of the 2007 House of Commons Health Select Committee criticise the current NICE threshold on the bases that it ‘. . . is not based on empirical research and is not directly related to the NHS budget, nor is it at the same level as that used by PCTs [primary care trusts] in providing treatments not assessed by NICE.’ Following previous literature, the implication is that, once a budget is set for the NHS (such budget setting not being the responsibility of NICE), we can infer a threshold by observing the cost per QALY of treatments which are funded vis-à-vis those that are not. 44

Two related responses to these arguments can be made: (1) it is well known that the NHS at the local level is not systematic in how it makes such decisions, at least in economics terms;45 and (2) because the NHS is not good at curtailing existing therapies which are poor value for money, it is not really known whether the marginal cost per QALY within the rest of the NHS is indeed out of line with (i.e. lower than) that used by NICE.

Given these significant challenges to ‘discovering’ a threshold, an alternative is to ask members of the public about their WTP for health gains. It may be thought that asking individuals about their WTP for such health gains from their own pockets would not be relevant to the issue of establishing a threshold value contingent on the size of the health-care budget. Indeed, Culyer et al. 44 (p. 57) state ‘Therefore, information about how much an individual or society values improvements in health (i.e. their WTP for a QALY) is not at all relevant to the NICE remit. These values could only be used as the appropriate threshold by NICE if it were also given responsibility to set the NHS budget.’

This is an internally consistent position. However, it does not diminish the importance of trying to establish what value(s) people actually do place on QALY gains. There are two main reasons for this.

First, in a democratic society there is a case to be made for ensuring that the government’s budget-setting process should, as far as possible, be informed by the preferences of members of the public. While it is reasonable to expect negotiations between the Treasury and Department of Health to take account of a number of factors, information regarding the public’s WTP for health care should arguably constitute an important consideration.

Second, a theoretical argument is that, when assessing WTP questions in surveys, if respondents think of the NHS as being at full efficiency and unable to provide more services (or QALYs) without extra payments being made, then expressed WTP amounts would be a reasonable representation of a value of a QALY at the margin for the NHS and not far removed from what a budget-holder, like a PCT, might say is the value (if PCTs used QALYs and if they behaved in an economically rational and QALY-maximising fashion!).

If the present study were to suggest that eliciting a robust monetary value of a QALY is feasible, and if a suitably representative sample survey were then undertaken, it would make a significant contribution to policy with respect to thresholds.

However, suitable existing evidence is scant and of variable quality. Some estimates have been made of the value of a QALY based either on modelling approaches or on survey research. 46,47 Modelling studies have been reviewed elsewhere and values of a QALY vary greatly depending on how the data are manipulated. 48 Moreover, survey work on the value of a QALY has been limited. Typically, individuals have been asked about their WTP for health gains for which quality adjustment factors have been obtained from another sample without fully adjusting for uncertainty (i.e. by presenting scenarios involving certain gains in QoL) and, in some cases, eliciting values from patients and not from members of the general public. 47,49,50 Only one such estimate exists for a European country. 47 The research undertaken in this study, therefore, represents a significant advance in the methods in this area.

Introduction

The first objective of the relativities study, ‘to identify characteristics of beneficiaries of health care over which relative weights are to be derived’, requires qualitative enquiry. This precedes the quantitative estimation of the relative importance placed on those characteristics, once established. While the quantitative study is reported in two subsequent sections, reflecting two methodological approaches, these approaches share a common empirical foundation based on in-depth exploratory and developmental research.

The selection of attributes for inclusion in the relativities study is crucial. The inclusion of an attribute without a robust rationale for doing so, or the omission of an important attribute, will lead to misleading conclusions. Qualitative techniques are increasingly used to establish appropriate attributes, particularly for discrete choice studies. 51 In this study we have taken a predominantly qualitative approach and supplemented conventional methods with other techniques. This exploratory phase of the project was an iterative process, involving three waves of focus groups with members of the general public, and use of a range of methods including: open-ended discussion; simple ranking procedures; experimentation with sample questions; and a more complex ranking task involving card sorting (Q methodology). Qualitative findings were interpreted alongside the results of the other methods used.

Most of this chapter is taken up by reporting on the three main methods used to identify the most important characteristics of beneficiaries, followed by a description of the development of methods used to present information to respondents using innovative diagrammatic representations, before, finally, describing the format of the questionnaires, presented using a computer-assisted personal interview (CAPI).

Methods

Focus groups were facilitated by two or more members of the research team depending on the size of the group. Discussions were introduced and guided by the focus group leader. Other researchers were available to help distribute materials and answer questions during individual exercises. Group discussions were recorded using a digital voice recorder and transcribed verbatim. All focus groups received an introductory description of the project and the problem at hand. This took the form of a brief presentation followed by the opportunity to ask questions. Participants were then guided through two or more tasks.

During the first wave of focus groups, we adopted open-ended qualitative techniques to elicit views and to probe responses. Respondents were advised that resources to provide health services are constrained and, as such, difficult choices must be made about the types of treatments and interventions that are provided by the NHS and, by implication, those that are not. The focus group leader introduced notions of scarcity of resources, carefully, and in simple terms (see Appendix 1). Essentially respondents were asked to accept the inevitability of rationing (although that term was not used) and that with or without the views of the general public, priority setting will happen. There was positive acceptance of these facts and respondents were comfortable proceeding on that basis. They were asked to suggest what sorts of things should be taken into account when such decisions are made.

Despite interesting discussions (a summary of which is presented below), participants often had difficulty absorbing and expressing opinions about the concepts we wanted them to explore. Generally, respondents readily proposed issues such as the size of the health gain, cost, QoL and life extension, but when asked to ‘go beyond’ these concepts there was difficulty (or perhaps reluctance). If the group discussion then stalled, examples of possible issues for discussion were suggested by group facilitators, but the results of this approach raised concerns about leading respondents and endowing particular attributes with validity simply by mentioning them.

In the second wave of focus groups, we included a simple ranking task both as a means of generating some crude data and to stimulate discussion. Respondents were asked to rank order a set of 10 cards printed with issues that might be considered in priority setting, such as ‘quality of life of patients before treatment’ or ‘the social class of patients typically affected’. Participants in these focus groups were also presented with some examples of the types of questions that would be used in the quantitative study, stimulating debate about the inclusion of different attributes as well as providing valuable information on the appropriateness of different modes of presentation.

In the third wave of focus groups, we introduced Q sort techniques which are sufficiently distinct to warrant a separate section and this follows the general findings. Q sorting involves arranging a number of cards, printed with statements about the topic, according to an instruction such as ‘from most agree to most disagree’. Allowing focus groups to begin with individual Q sort activities and following this with discussion enabled respondents to express their views (via the Q sort) before engaging in discussion with others. They also entail a common stimulus set (in this case 46 cards printed with different statements about the issue at hand). These results are, therefore, unencumbered by the input of the focus group leader, or by the views of others during discussion.

Focus groups were conducted with groups of between 4 and 10 respondents and organised in three waves of data collection in Newcastle upon Tyne and Norwich between March and November 2005. Participants in the Newcastle groups were recruited through a social research company (NWA Social and Market Research) based in the north-east of England and £20 was paid to each participant in recognition of their travel expenses and time. Participants in Norwich were recruited from an existing university database of members of the general public who had consented to be contacted for research. A total of 126 respondents (42 in Norwich and 84 in Newcastle) took part in focus groups.

Qualitative findings based on open-ended discussion

As already stated, the use of open-ended qualitative techniques was only a partial success. This may reflect the fact that respondents do not necessarily have a readily articulated account of their views on such complex issues, or that they were unsure about the kinds of things we were asking them to consider. In general, respondents were far more comfortable talking about health-related characteristics than they were discussing social or personal characteristics of the beneficiaries of health care. We do not report a complete, formal qualitative analysis because of the nature of these data, our own objectives and word limits. Instead, a summary is given of the nature of the discussion on each issue, including some brief illustrative quotes, and the results of the simple ranking exercise are included.

Results based on simple ranking

A subgroup of 19 respondents (aged 20–62, 10 female) rank ordered a set of possible attributes according to their importance and discussed their rankings. They were also invited to add any additional attributes (writing them onto blank cards provided) and incorporate those into their ranking. Table 1 presents the average ranking of each item. This is only illustrative; respondents’ rankings are not intended to be interpreted in isolation of their comments and the results of other methods.

Respondents’ comments during this task revealed multiple understandings of the attributes as well as a small number of common views. Respondents were comfortable and confident talking about health-related attributes and less so when discussing non-health-related attributes. Socioeconomic status and gender were considered irrelevant to issues of prioritisation by all respondents. Whether or not patients had had a lot of health care in the past was often construed as the health service having failed them and issues of ‘orphan drugs’ were difficult for respondents to appreciate and were not covered by the lack of any other treatment.

While the qualitative data and simple ranking data provide a good grounding in the issues in question, we felt it was insufficient for the selection of attributes and so incorporated a third method, Q methodology. This is relatively unfamiliar to most and so requires separate explanation below.

Q methodology

The basic features of Q methodology

Q methodology52–54 is used to study the nature of views, opinions and beliefs. It is a useful addition to qualitative methods, especially where respondents do not necessarily have readily articulated accounts of their views on a topic.

The two main features of a Q study are the data collection method – which is based on a card sorting technique (the ‘Q sort’) – and a form of factor analysis which is used to analyse patterns between the card sorts to reveal a small number of underlying perspectives. There are several key terms that are used in Q studies. The ‘Q sort’ provides the primary data source in Q methodology. Respondents sort a set of statement cards, known as the ‘Q set’. The Q set comprises a number of statements which cover the range of viewpoints and opinions on the particular topic of interest. Respondents consider each card in turn and assign it in a quick, initial sort, to one of three piles: agree, disagree or neutral. A more detailed arrangement of cards then follows, using a grid such as the one reproduced in Figure 1. Factors are the result of Q analysis, the aim of which is to identify shared views and meanings that exist around a topic (via correlations between the positioning of cards by respondents).

FIGURE 1.

Q sort response grid.

Each space in the grid indicates the positioning of a card on the continuum from –5 to +5. Two items are placed in the ‘+/–5’ positions, four items in the ‘4’ and ‘3’ positions and so on. Making use of the three initial piles, respondents are asked to consider the cards in their ‘agree’ pile, select two cards that they ‘most agree’ with and place these in the +5 column. Next, selecting from the cards that they disagree with, respondents are asked to select the two cards that they ‘most disagree’ with, and place those cards in the –5 column. This process is repeated until all cards are placed (46 in this example), finishing at the centre of the distribution. Often the Q data (i.e. the positioning of the cards in the Q sort) are supplemented by a brief interview. In this study the Q sorts were followed by group discussion.

Summary

Based on our qualitative and Q methodological enquiry, we rejected both lifestyle and socioeconomic status as attributes. Age, QoL and length of life were clearly issues of importance. The issue of whether beneficiaries of health care have dependants was slightly more difficult to resolve, but ultimately we took account of considerations of policy relevance, in consultation with representatives of NICE. It is difficult to conceive of a situation where NICE would recommend an intervention be made available only to people with dependants.

Arriving at a list of key attributes for inclusion in quantitative survey questions is a difficult process. Increasingly, qualitative methods are built into the early stages of study designs. In the context of this study, the use of open-ended qualitative methods alone proved insufficient to determine the attributes, perhaps because of the complexity of the subject matter. Respondents appeared to be led by the suggestions of the focus group facilitator, and there remained uncertainties around particular attributes (such as socioeconomic status) following analysis of the qualitative data. In this case, Q methodology provided additional structure and the opportunity to use a standardised stimulus in both the generation of data and the analysis. With respect to socioeconomic status, for example, we were able to divide the broad issue into sub-issues that could be described in discrete statements. Factor analysis showed a good deal of consensus that these issues should not be part of health-care priority setting, and we selected age and severity as the attributes to bring forward into the quantitative analysis. Future research investigating the views of the public around complex issues should consider using Q methodology in addition to more typical qualitative methods.

Diagrammatic questions

The presentation of the discrete choice and matching questions, including a detailed introductory explanation, was developed iteratively in focus groups. By far the most successful method was presentation of concepts of health (QoL), age, and health gains using diagrams. These diagrams were first explained by building them in small steps for respondents using an animated powerpoint presentation (reproduced in Appendix 4), which was ultimately incorporated into a CAPI. The diagrams were then presented either as choice questions or as matching questions, examples of which are shown in Figures 2 and 3. Choice questions (e.g. Figure 2) present respondents with a one-off choice between option A and option B, which differ in terms of health gains, ages of patients and levels of QoL. The two options are presented both diagramatically and descriptively in the accompanying text. Choice questions are explained in more detail in Chapter 4.

FIGURE 2.

Examples of choice questions. (a) Screen 1: treatment for condition A results in a gain in life-years; treatment for condition B results in a gain in both quality of life (QoL) and length of life. (b) Screen 2: treatment for either condition A or condition B results in a gain in quality of life. (c) Screen 3: treatment for condition A results in a gain in quality of life; treatment for condition B results in a gain in length of life.

FIGURE 3.

Example of iterations within a single matching question based on responses shown. (a) Screen 1: untreated scenarios. (b) Screen 2: ‘with treatment’ scenarios showing health gains – condition A is chosen by the respondent. (c) Screen 3: when 40 people with condition A or 100 people with condition B can be treated, the respondent chooses B. (d) Screen 4: when 80 people with condition A or 100 people with condition B can be treated, the respondent chooses B. (e) Screen 5: when 95 people with condition A or 100 people with condition B can be treated, the respondent chooses A. (f) Screen 6: finally, when 90 people with condition A or 100 people with condition B can be treated, the respondent chooses B.

Matching questions (e.g. Figure 3) present respondents with a series of iterative choices where the numbers of people in Groups A and B are varied until a point of equivalence is reached. The same size of health gain is presented in Group A as in Group B for each set of iterative choices. These questions are explained in more detail in Chapter 5.

The questionnaire instrument was then piloted using cognitive interviews with 42 respondents (27 in Norwich and 15 in Newcastle) and was well received. Respondents followed the introduction and understood the questions. Their comments resulted in only minor adjustments to the wording and number of questions in each version of the questionnaire.

Survey sample and administration

The discrete choice and matching questions were part of a longer questionnaire (incorporating attitudinal questions and sociodemographics), which was administered face to face, using a CAPI, by interviewers from NatCen. The interview began with some basic demographic and household questions, following which an animated powerpoint presentation explained the meaning of the diagrams step by step. Next the respondents answered six matching questions followed by eight discrete choice questions. There were four attitudinal questions, before some more detailed sociodemographic and health questions. On average, interviews lasted 41 minutes.

The survey was administered by NatCen to a random sample (n = 587) of the population in England from February to April 2007. The sample was generated by NatCen from the population of adults (aged 18 and over) living in England. Thirty addresses were selected from each of 40 postcode areas, which were stratified by Government Office Region (nine regions) and the proportion of manual/non-manual households. Within each household, only one adult was eligible for inclusion in the study. In households with more than one eligible adult present, interviewers randomly selected one interviewee. A total of 243 (41%) were male, the mean age of the whole sample being 52 years and, thus, females and older people are slightly over-represented.

Basic design

In a discrete choice study, respondents are presented with a series of choice sets (usually pairwise). Each scenario in the set is defined according to some predefined attributes (ours being predetermined from the research reported in Chapter 3) and levels of such attributes. The attribute levels vary across scenarios and choice sets. In each choice set, the respondent is asked which scenario they prefer or would choose. Faced with a series of such choices, respondents essentially reveal how much weight they attach to each of the attributes, the actual weights being derived through statistical analysis of the data (see below).

Table 5 lists the attributes introduced in the preceding chapter along with the levels for each that were used in the discrete choice study.

The levels for the age variable were chosen to represent a range of stages of life: a very young child, a fully grown child and a young adult, followed by two or three further stages of adulthood into old age. The QoL if untreated variable again ranges from death through a series of better (though some still quite serious) states to 90%. The variables representing gains in life expectancy and QoL follow in large part from what was chosen for age and QoL if untreated. For example, to give a full number of life-years to a 1-year-old who would otherwise die, the gain in life expectancy would be 79, and to return someone in a health state valued at 60% back to full health, a gain of 40% would be offered. Some levels of the life expectancy and QoL gain variables also reflect the desire to have some scenarios where people might get an improvement but not to full health or maximum life-years (the maximum here being 80).

Using these variables and imposing some assumptions (such as people being in full health before the onset of illness), we constructed diagrams of the sort shown in Figure 2. QALY gains (shown as a dark shaded area in the diagrams) are calculated from the five attributes listed in Table 5.

A full factorial design using the attributes listed in Table 5 would have resulted in (6)(7)(8)(4)(6) = 8064 possible profiles but there are many combinations of levels on these attributes that result in implausible scenarios. For example, ‘age at death’ must be greater than ‘age at onset’ of illness, and the gain in life expectancy added to the age at death if untreated cannot exceed some reasonable maximum age, which was constrained to be 80 years. The full list of constraints is listed in Appendix 5. After imposing these constraints, 6572 of the possible 8064 profiles were implausible, leaving 1492 profiles (19% of the total).

The experimental design software sas, which allows for constraints, was initially used to select choice sets from the 1492 profiles. This resulted in a design with over 200 profiles describing ‘age at onset’ as 1-year-olds and only 12 profiles describing 70-year-olds. At this stage the design was altered manually to improve the balance of questions about different age groups. Clearly, such a severely-reduced set of available profiles, together with the manual alteration of the design to achieve greater balance, negatively affect the design properties that are desirable in discrete choice experiments (DCEs). Nevertheless, before administering the questionnaire, data were simulated to ensure that a model could be estimated on the basis of the amended design.

Functional form and empirical approach

Response data in DCEs are modelled within a random utility framework of the general form:

in which utility, U, is separated into parts which are explainable, in this case V, and unexplainable, ε. In this study, we are concerned with estimating V, which represents an underlying continuous and latent variable which is nevertheless unobservable.

If the standard QALY model is true, V would simply be a function of QALYs. If individuals are concerned about other characteristics, then these will also be part of the utility function. In this simple case we assume that utility is a function of age at onset (AO), age at death without treatment (AD), QoL lost without treatment (QL) and QALYs gained from treatment (QALY). This gives:

Quality of life lost (QL) is transformed from a variable in Table 5, by subtracting ‘quality of life if untreated’ from 1. This is done in order to facilitate the log transformation required below – with some scenarios involving instant death, and thus a QoL of zero, which could not otherwise have been log transformed. Note, therefore, that when interpreting this variable, the larger the ‘quality of life lost’ at the onset of illness, the more ‘severe’ the health state.

The first two terms on the right-hand side of (2) detect age effects, QL detects severity and QALY is the health gain. If expressed as an additive function, this would mean that gains in utility could be experienced even if QALY gains were zero, i.e. age and severity would have effects on utility irrespective of whether or not QALY gains are incurred. The alternative, therefore, was to use a multiplicative form of the utility function. The QALY itself is a multiplicative function of life-years and QoL gained. By extension, in a multiplicative form of the above function, utility is derived from QALYs multiplied by the magnitudes of the other variables, ensuring that with zero QALYs gained there is a zero impact on utility. Empirically, the multiplicative models presented below consistently outperformed those based on an additive functional form, which were also investigated. Although, it may appear that a multiplicative model of the form QALYs × AGE × SEVERITY, with just one age-related variable, would make more theoretical sense, we took the more pragmatic view that this would leave too much riding on the ‘age at onset’ variable in terms of what respondents might be thinking about in relation to age, and so we included age at death as well. In addition, this was the best performing model empirically, which, it could be argued, is important for estimating weights.

If we assume such a multiplicative underlying model, we may use a log-linear model of the form:

This is a standard log-linear utility function where the βs are parameters to be estimated. Given that discrete choice response data are based on choices over alternative combinations of the dependent variables, then, assuming that the β parameters are identical across all individuals, a simple model of the following form can be estimated:

where Δ represents the differences in levels of any given attribute reflected in the pairwise choices presented.

Equation (4) was estimated using a logit model (see ‘simple’ model under Discrete choice results), allowing for clustering of the individual standard errors to account for the fact that each individual responded to several questions. Essentially, this amounts to a conditional logit model.

A flexible functional form (referred to below as the ‘powered’ model) was also specified to allow for any non-linear relationships between choice and the included variables. While the standard log model allows for non-linearities, these functions are monotonic, this restriction possibly being too strong and resulting in biased estimates. Including higher order terms, in a fashion analogous to the popular translog model,56 allows for modelling of more complex non-linearities. The use of such flexible functional forms provides more robust estimation of the coefficients and reduces the potentially confounding problems of omitted variable bias. This also allows us to investigate the non-linearities detected in the matching data (see Chapter 5). All models were tested using the Akakie and Bayesian information criteria in order to aid model selection. Those which performed best on these criteria are reported below.

Discrete choice results

A total of 587 respondents yielded 4696 useable responses to the discrete choice questions. The estimated models are shown in Table 6. We did also take a more conventional econometric approach of converting covariates into categorical variables (i.e. sets of dummy variables) within the basic multiplicative framework. However, these models did not perform as well as those reported and are more challenging for calculation of weights.

The coefficients for the simple model suggest that increasing age at onset reduces the probability of choice, as does increasing the age at death, although the former is not statistically significant. This suggests that the young are preferred to the old. The coefficient on QoL lost is also negative, suggesting that as the health state is more severe the respondents are less likely to choose that group. Finally, the impact of QALY is positive, as we would expect. The impact of severity, as it appears in the regression results, appears to contradict earlier literature. However, a more accurate picture of the impact of all of the variables is provided through examination of the shapes of the functional forms, which, for the simple model, are given in Figures 4–7. These diagrams show the shapes of the functional relationships when measured on a single scale, as represented by the ‘predicted utility’ axes. Here, it can be seen that that the general impact of the QoL lost variable on utility is very small, the inference being that the severity–utility relationship is essentially flat and nearly so for age.

FIGURE 4.

Age at onset vs utility.

FIGURE 5.

Age at death vs utility.

FIGURE 6.

Quality of life (QoL) lost vs utility.

FIGURE 7.

QALY vs utility.

The equivalent diagrams for the powered models are shown in Figures 8–11. Once again, the sign of the coefficients in the regression model inform us of the direction of impact on utility for age at onset (individuals who are aged around 10–40 years are slightly preferred to the very young and the very old), age at death (slight preferences to save those who die young and those who will die old rather than the middle aged) and severity (a preference for individuals with lower severity, with the maximum at 0.4, after which predicted utility slopes downwards showing less preference for purely life-saving interventions). However, Figures 8–10 demonstrate, once again, the relationship between each of these variables and utility to be essentially flat. This is reinforced by the lack of statistical significance on some of the coefficients, especially for age at onset and, this time, QoL lost. Nevertheless, weights were calculated using the point estimates from the powered regression, regardless of statistical significance. This approach is used for a number of reasons: first, the model is the best performing in diagnostic tests; second, we cannot rule out joint significance; and third, and most importantly, when calculating weights from any model, the point estimates from that model (regardless of which approach is used) are our most informed values. While it is possible to use hypothesis testing to determine whether our estimates are significantly different to zero, this does not provide better information than the point estimates. Also, because of what was revealed by the directions of the coefficients in the regression models, the results on severity are further investigated (see Further investigation of severity). Figure 11 shows that increasing QALYs are always preferred. Concavity is still observed, but less so than in the simple model.

FIGURE 8.

Age at onset vs utility (powered functions).

FIGURE 9.

Age at death vs utility (powered functions).

FIGURE 10.

Quality of life (QoL) lost vs utility (powered models).

FIGURE 11.

QALY vs utility (powered model).

Weighting QALYs using discrete choice data

Two novel ways of estimating weights from the above models have been devised, one based on predicted probabilities of choice and the other based on a compensating variation approach. As indicated above, for each, the weights presented are based on the powered model only.

Predicted probability of choice approach

The weights are estimated using the estimated probability of choice, with a base case of AO = 40, AD = 60, QL = 0.7, and QALY = 4 (i.e. individuals fall ill at 40, lose 0.7 of their QoL, will die at 60 without treatment, and are then given four QALYs with treatment). The way in which the four QALYs have been allocated is unspecified in this model. This choice of a base case is challenging for interpretation of subsequent weights, in that it may be thought better to choose an extreme position and then measure weights for every other scenario in one direction relative to that. However, using an extreme as a base case is problematic too; the most obvious example being use of the highest age group from which it is not possible to gain any QALYs. The choice of a four-QALY gain arose to reflect a reasonable-sized gain and also to correspond with the four QALY gains which were offered to respondents in the matching study (see Chapter 5).

In order to calculate the weights we compare our base case to an alternative scenario. We then vary the number of QALYs being offered in the alternative scenario until the estimated probability of choosing the base case equals 0.5 (i.e. the individual is indifferent between the two scenarios). This is more easily demonstrated by using an example. We have our base case, AO = 40, AD = 60, QL = 0.7 and QALY = 4. We now take a scenario for which we want to find a weight, for example AO = 1, AD = 1 and QL = 1 (i.e. individuals fall ill and die at age 1), this being comparison 1 in Table 7. We adjust the number of QALYs from treatment available to the comparison group until the probability of choosing the base case equals 0.5. This probability to calculated as:

Pr BASECASE = exp ( log ( u BASECASE ) ) exp ( log ( u BASECASE ) ) + exp ( log ( u COMPARISON ) )

TABLE 7 - Weights based on powered model

QoL, quality of life.

Base	Age at onset	Age at death	QoL lost	QALYs gained	Weight
	40	60	0.7	4
Comparison
1	1	1	1	4.1	1.025
2	1	10	0.7	1.85	0.463
3	1	10	0.1	1.52	0.38
4	1	20	0.7	3.8	0.95
5	1	20	0.1	3.16	0.79
6	10	10	1	2.48	0.62
7	10	20	0.7	3.5	0.875
8	10	20	0.1	2.9	0.725
9	10	40	0.7	4.6	1.15
10	10	40	0.1	3.95	0.9875
11	40	40	1	6.15	1.54
12	40	60	0.7	4	1
13	40	60	0.1	3.38	0.845
14	70	70	1	6.45	1.61
15	70	80	0.7	4.2	1.05
16	70	80	0.1	3.55	0.89

where is the predicted utility for the ith choice from the powered model regression results in Table 6. In this case the probability of choosing the base case is 0.5 when the comparison group is offered 4.1 QALYs. The weight itself is found by taking the ratio of the QALYs offered in both cases, so:

weight = QAL Y COMPARISON QAL Y BASECASE

which in this case gives 4.1/4 = 1.025. This demonstrates that the base case is slightly preferred to the alternative, with one QALY to the base case being equal to 1.025 QALYs to the comparison. This process is repeated for each comparison scenario to generate weights. More generally, we try to illustrate this in Figure 12. Weights closer to 0 show a stronger preference for the comparison, weights equal to 1 show indifference between the two groups and weights greater than 1 show preference for the base case.

FIGURE 12.

Preferences and weights in the discrete choice study.

The weights from this procedure are given in Table 7. (In this table, the variables given earlier as percentages are now presented on a 0–1 scale in line with the more common representation in the QALY literature. Percentages were used earlier because that is how QoL was presented to survey respondents.) While these represent weights arising from assessing the respective comparator group against the base case, it is possible to generate weights for comparing different comparison groups. For example, if we wished to compare scenarios 5 and 8, we could do this indirectly by comparing the weights given in Table 7. The weight for comparison 5 is 0.79 and the weight for comparison 8 is 0.725 – from this we can conclude that scenario 8 is preferred to scenario 5. Alternatively, we could take the ratio of these weights, to generate a new weight for scenario 8 of 0.92 (0.725/0.79), which demonstrates that 0.92 of a QALY to scenario 8 is worth one QALY to scenario 5.

The weights presented in Table 7 are sometimes difficult to interpret because a number of factors vary at once when comparing scenarios. We can see that most weights are less than 1, suggesting that the comparison scenarios used here are generally preferred to the base case. The only exceptions are comparisons 1, 9, 11, 14 and 15 (and comparison 12 which is the same as the base case). Apart from comparison 1, these scenarios are towards the top end of age at death and are quite severe.

For those scenarios which are purely life saving, where age at onset equals age at death (comparisons 1, 6, 11 and 14), we see that there is a preference for those with an age at onset and age at death of 10 (comparison 6). The weight for this scenario is 0.62. The weights for the older groups (comparisons 11 and 14) are greater than 1, indicating a preference for the base case. The weight for the youngest group (comparison 1) is also greater than 1, suggesting that individuals do not choose to treat those who are very young. These results suggest a non-linear relationship between age and weights, with the most preferred being the younger, but not youngest, groups.

For severity we can see that, for the same age at onset and age at death, as QoL lost decreases, moving from 0.7 to 0.1, the weight moves towards 0. The less severe comparisons (where QoL lost equals 0.1) have a weight of less than 1, showing that they are the preferred groups. This can be seen for comparisons 2 and 3, 4 and 5, 7 and 8, 9 and 10, 12 and 13, and 15 and 16.

It is difficult to make generalisations about age at onset and age at death using Table 7, where both variables often vary at the same time. An alternative approach to presenting the weights is to hold one of the factors constant and allow the others to vary. This is the approach used in Tables 8 and 9.

TABLE 8 - Weights by age at onset and age at death

Age at death	Age at onset
	1	10	20	40	60	70
1
10	0.47
20	0.95	0.87
40	1.25	1.16	1.04
60	1.09	1.00	0.90	1.00
70	0.95	0.88	0.78	0.86	1.09
80	0.80	0.74	0.65	0.73	0.93	1.05

TABLE 9 - Weights by age at onset and severity

Severity	Age at onset
	1	10	20	40	60
0.1	0.925	0.850	0.750	0.840
0.4	0.875	0.805	0.713	0.800
0.7	1.088	1.000	0.900	1.000
1					1.625

In Table 8, QoL lost is fixed at 0.7. This clearly shows that as age at onset increases the weights move towards 0 up to age 20 but then move towards 1. Weights for age at death move towards 1 up to age 40, but then fall again. This shows that there is a clear preference for treating people who fall ill at ages between 10 and 40 rather than the very old or the very young, and a preference for treating those who will die either very young or very old. In this, and similar tables to follow, blank cells are simply representative of unfeasible comparisons.

In Table 9, age at death is fixed at 60 and the other attributes are allowed to vary. Again we can see that the weights for age at onset are quadratic, starting close to 1 for an age at onset of 1, moving towards 0 up to age at onset of 20 before moving towards 1 as age at onset increases further. For severity we can see a preference for treating those who are closer to the middle of the severity range. The weights are nearest 0 for a severity of 0.4 and closer to 1 for severity scores of 0.1 (least severe) and 1 (most severe).

Compensating variation approach

Another approach to calculating the relative weights attached to different types of QALYs, or beneficiaries of QALYs, is to use the Hicksian compensating variation approach to welfare measurement. The method for calculating the compensating variation using discrete data is due to Small and Rosen57 and was introduced to the health economics literature in the context of DCEs by Lancsar and Savage. 58

In general, the compensating variation (CV) is calculated by valuing in monetary terms the change in expected utility due to a policy change (e.g. change in price or quality of a good/service, or, in our study, a change in health state) as the change in income required to return the individual to their initial level of utility, that is, to compensate them for the change.

The CV for discrete choice data takes the following form:

⁽⁵⁾ CV = − 1 λ [ ln ∑ j = 1 J e V J 0 − ln ∑ j = 1 J e V J 1 ]

where VJ0 and VJ1 are the value of the indirect utility function for each choice option j before and after the policy change respectively; J is the number of options in the choice set; and λ is the marginal utility of income, or its proxy.

While a monetary value is the most convenient to turn the change in expected utility into a common metric, in fact any quantitative metric could be used as the numeraire. In the current study, instead of using income it is possible to calculate the CV for a move from one health state to another using the marginal utility of QALY gains as the numeraire. That is, we calculate the number of additional QALYs required to be given or taken away in the new health state to equalise the utility derived from the base and new health states.

Like the probability-of-choice approach described previously, the CV approach can be used to value changes in, and create weights for, whole scenarios (i.e. entire health states). In addition, the CV can also be calculated to value and derive weights for individual characteristics such as age at onset, age at death and severity. To calculate both types of weights, we value changes from an initial health state, namely the same reference case as used in the calculation of the probability weights: AO = 40, AD = 60, QL = 0.7 and QALY = 4.

From this initial health state we can calculate the CV, measured in numbers of QALYs, associated with a move to a new health state, described by new levels on the above attributes. This can involve an entire scenario or the valuation of each attribute one at a time. We use these CV measures to calculate weights for entire health states and for individual characteristics that describe the health states, by taking the ratio of the total QALYs required in the new health state to equalise utility and the original number of QALYs in the base model. This welfare theoretic approach also allows investigation of the strength of preference, as indicated by the magnitude of the CV.

As with the probability-of-choice approach, the CV and relative weights reported in this section used the conditional logit results for the powered model reported in Table 6. The CV and relative weights for the entire health states described in Table 7 are reported in Table 10. In each of these, QALY gains are held constant at 4.

TABLE 10 - Weights based on compensating variation approach (version 1)

CV, compensating variation; QoL, quality of life.

Base	Age at onset 40	Age at death 60	QoL lost 0.7	QALY gain with treatment 4	CV	Weight per scenario
Comparison
1	1	1	1	4	0.0224	1.0056
2	1	10	0.7	4	–0.5788	0.8553
3	1	10	0.1	4	–0.7280	0.8180
4	1	20	0.7	4	–0.0435	0.9891
5	1	20	0.1	4	–0.1787	0.9553
6	10	10	1	4	–0.3650	0.9087
7	10	20	0.7	4	–0.1097	0.9726
8	10	20	0.1	4	–0.2468	0.9383
9	10	40	0.7	4	0.1190	1.0297
10	10	40	0.1	4	–0.0112	0.9972
11	40	40	1	4	0.3591	1.0898
12	40	60	0.7	4	0	1
13	40	60	0.1	4	–0.1339	0.9665
14	70	70	1	4	0.4050	1.1013
15	70	80	0.7	4	0.0390	1.0098
16	70	80	0.1	4	–0.0937	0.9766

Before discussing the results in Table 10, it is important to note that ‘QALY gain with treatment’ represents the number of QALYs in the health-state scenarios presented to respondents in the survey. This is not the same as the ‘QALY gain’ in the probability-of-choice tables, which is the result of the calculation of the number of QALYs required to equalise the probability of choosing the base case and alternative health states.

Both the CV and weights are interpreted relative to the reference case. The same reference case is included as comparison 12, for which the CV is 0, as would be expected since the health state has not changed, and the weight is 1. In Figure 13, relative to this base case, a negative CV indicates that the new state is preferred to the base case and that individuals are ‘willing to pay’ to secure the new health state – see for example, comparison 2. The consequent weight of 0.855 indicates that a QALY given in the base state is worth only 0.855 of a QALY given in the new state. A positive CV indicates that individuals require compensation (in terms of being given additional QALYs) in the new health state – see for example, comparison 1. A weight of greater than 1 indicates preference for the base state.

FIGURE 13.

Preferences and weights in the discrete choice study.

For the four comparison health states that involve instant death without treatment (comparisons 1, 6, 11 and 14) there is a preference for giving the QALYs to the base case, with the exception of comparison 6 which favours the alternative. This is in accordance with the underlying utility values derived from each health state. That is, the utility derived from the base case is greater than the utility derived from the alternatives with the exception of comparison 6 (a 10-year-old who dies instantly without treatment), which yields a higher utility than the base case.

Still looking at the magnitude of the CV relative to the base case, respondents were willing to pay the largest amount (in terms of QALYs) for comparison 3, namely a health state in which the individual becomes unwell at age 1, without treatment QoL drops by 90% and they die at age 10, whereas with treatment they gain four QALYs. The least preferred beneficiary of four QALYs is someone who becomes unwell at age 70 and without treatment dies instantly (comparison 14).

Welfare measures and relative weights can also be calculated for the same complete health states described in Table 10, but where QALYs, in addition to the other attributes, are allowed to change between the initial and new health states. These results are presented in Table 11.

TABLE 11 - Weights based on compensating variation approach (version II)

CV, compensating variation; QoL, quality of life.

Base	Age at onset 40	Age at death 60	QoL lost 0.7	QALY gain with treatment 4	CV	Weight per scenario
Comparison
1	1	1	1	1	0.8568	1.2142
2	1	10	0.7	1	0.3853	1.0963
3	1	10	0.1	1	0.2642	1.0660
4	1	20	0.7	1	0.8065	1.2016
5	1	20	0.1	1	0.7022	1.1756
6	10	10	1	1	0.5561	1.1390
7	10	20	0.7	1	0.7557	1.1889
8	10	20	0.1	1	0.6491	1.1623
9	10	40	0.7	1	0.9299	1.2325
10	10	40	0.1	1	0.8312	1.2078
11	40	40	1	1	1.1082	1.2771
12	40	60	0.7	1	0.8398	1.2100
13	40	60	0.1	1	0.7370	1.1842
14	70	70	1	1	1.1417	1.2854
15	70	80	0.7	1	0.8695	1.2174
16	70	80	0.1	1	0.7680	1.1920
17	1	1	1	2	0.4895	1.1224
18	1	10	0.7	2	–0.0460	0.9885
19	1	10	0.1	2	–0.1813	0.9547
20	1	20	0.7	2	0.4316	1.1079
21	1	20	0.1	2	0.3122	1.0780
22	10	10	1	2	0.1462	1.0366
23	10	20	0.7	2	0.3733	1.0933
24	10	20	0.1	2	0.2517	1.0629
25	10	40	0.7	2	0.5741	1.1435
26	10	40	0.1	2	0.4601	1.1150
27	40	40	1	2	0.7821	1.1955
28	40	60	0.7	2	0.4699	1.1175
29	40	60	0.1	2	0.3519	1.0880
30	70	70	1	2	0.8215	1.2054
31	70	80	0.7	2	0.5041	1.1260
32	70	80	0.1	2	0.3874	1.0969
33	1	1	1	10	–0.9594	0.7602
34	1	10	0.7	10	–1.6568	0.5858
35	1	10	0.1	10	–1.8256	0.5436
36	1	20	0.7	10	–1.0374	0.7407
37	1	20	0.1	10	–1.1961	0.7010
38	10	10	1	10	–1.4122	0.6469
39	10	20	0.7	10	–1.1153	0.7212
40	10	20	0.1	10	–1.2754	0.6811
41	10	40	0.7	10	–0.8442	0.7890
42	10	40	0.1	10	–0.9992	0.7502
43	40	40	1	10	–0.5535	0.8616
44	40	60	0.7	10	–0.9859	0.7535
45	40	60	0.1	10	–1.1437	0.7141
46	70	70	1	10	–0.4971	0.8757
47	1	1	1	15	–1.6162	0.5960
48	1	10	0.7	15	–2.3566	0.4109
49	1	10	0.1	15	–2.5338	0.3666
50	1	20	0.7	15	–1.6997	0.5751
51	1	20	0.1	15	–1.8691	0.5327
52	10	10	1	15	–2.0985	0.4754
53	10	20	0.7	15	–1.7829	0.5543
54	10	20	0.1	15	–1.9535	0.5116
55	10	40	0.7	15	–1.4924	0.6269
56	10	40	0.1	15	–1.6588	0.5853
57	40	40	1	15	–1.1777	0.7056
58	40	60	0.7	15	–1.6446	0.5889
59	40	60	0.1	15	–1.8132	0.5467
60	1	1	1	30	–3.2013	0.1997
61	1	10	0.7	30	–4.0036	–0.0009
62	1	10	0.1	30	–4.1926	–0.0482
63	1	20	0.7	30	–3.2930	0.1768
64	1	20	0.1	30	–3.4780	0.1305
65	10	10	1	30	–3.7265	0.0684
66	10	20	0.7	30	–3.3840	0.1540
67	10	20	0.1	30	–3.5696	0.1076
68	10	40	0.7	30	–3.0648	0.2338
69	10	40	0.1	30	–3.2481	0.1880
70	40	40	1	30	–2.7139	0.3215
71	40	60	0.7	30	–3.2325	0.1919
72	40	60	0.1	30	–3.4171	0.1457

Again, a weight greater (less) than 1 indicates a preference for the base (alternative) health state. Looking at these results in aggregate, the number of QALYs gained is driving the preference between the base and comparison health states. Without exception, for QALY gains in the new health state of 10, 15 and 30, the new health state is always preferred to the base (which only contains four QALYs). For all comparison health states involving only one QALY gain, the base case (containing four QALYs) is always preferred. For comparison health states with two QALY gains, the base case is generally preferred to the comparison case, except when those two QALYs are given to 1-year-olds whose QoL drops by 0.1 and who die at age 10, or 1-year-olds whose QoL drops by 0.7 and who die at age 20.

The CV and relative weights associated with the individual attributes (age at onset, age at death and severity) are reported in Table 12. These calculations used the same initial base health state of age at onset of 40, age at death of 60, QoL lost of 0.7 and four QALYs. The CV and weights per attribute level were calculated by changing one attribute at a time in the new health state, holding all else constant.

TABLE 12 - Compensating variations per attribute

CV, compensating variation; QoL, quality of life.

Attribute	Level	CV	Weight
Age at onset	1	0.0688	1.0172
	10	0.0043	1.0011
	20	–0.0862	0.9784
	40	0	1
	60	0.1741	1.0435
	70	0.2675	1.0669
Age at death	1	–0.3173	0.9207
	10	–0.6568	0.8358
	20	–0.1141	0.9715
	40	0.1148	1.0287
	60	0	1
	70	–0.1108	0.9723
	80	–0.2438	0.9390
Severity (QoL loss)	1	0.2517	1.0629
	0.7	0	1
	0.4	–0.1730	0.9568
	0.1	–0.1339	0.9665

Age at onset of 20 and 40 appear to be the most important, which could relate to considerations of the ages at which individuals are most productive to society (and also more likely to have dependants), followed by the very young, with least importance given to the eldest two age groups of 60- and 70-year-olds. Higher preference is given to those who die young (aged 10, 1 and 20).

When interpreting the severity weights, we first note that the base case of QoL lost = 0.7 is one of the middle severity levels, so movements from this base case can represent an improvement in QoL (a loss of only 0.4 or 0.1 rather than 0.7 QoL) for which respondents were willing to pay or a detriment (QoL lost = 1) for which respondents required compensation. As movements from the base are in both directions, absolute values are used. Relative to the base case, again, more severe states are less preferred, as would be expected given the results reported above. In particular, moving from the base to the worst level of severity had the largest impact in terms of the CV; that is, respondents would have to be compensated a relatively large amount in absolute value for a health state that involved instant death. Willingness to pay (in terms of QALYs) to improve the initial level of severity – that is, to lose only 10% or 40% of QoL rather than the base case of 70% was similar across these two levels, producing similar weights (0.9567 and 0.9665 respectively).

Further investigation of severity

In the above regression analyses, the severity variable (QoL lost) appeared to behave in the opposite direction to how it had been weighted in previous studies. Although the relationship is close to being flat, this different result is nevertheless reflected to some extent in the regressions and in the weights presented in the previous section. This issue was, therefore, thought worthy of further investigation. In particular, the question to address is why QALYs gained by those who would otherwise die instantly (i.e. whose QoL lost would be 1) were not weighted higher than other types of QALY gain.

To investigate this further, we isolated choice sets in which one scenario involved QoL lost being equal to 1 (i.e. age at onset = age at death), referred to here as the ‘life-saving’ option. By selecting choices with just one life-saving option, by definition, the comparator scenario within any such choice would be any of all other types (e.g. QoL enhancing only and L-shaped gains which enhance QoL, but also add years to life). This amounted to 1253 choice sets in total.

On 446 occasions, the life-saving option was chosen. On these occasions, the mean QALY gain being offered in the life-saving option was 12.8 (median = 7) as opposed to a mean QALY gain of 10.7 (median = 7.8) in the alternative (which was not chosen). The life-saving option, by definition, presents QoL lost as 1, and the mean QoL lost presented in the rejected scenario was 0.54. On these occasions it seems that, on average, QALY gains are larger and beneficiaries are more severe in life-saving options, so it is not surprising that this was chosen.

On the 809 occasions when life saving was not chosen, the mean QALY gain being offered in the life-saving option was 5.43 (median = 4) as opposed to 13.4 (median = 7.9) in the alternative; the alternative having a mean QoL lost of 0.53. On these occasions it seems that, on average, QALY gains are much larger when beneficiaries are in a less severe state. This seems to have driven the choices made, making severity appear less important than it might be if choices were offered in which the scenarios were closer together in terms of numbers of QALYs gained. This may explain why results of matching-type studies, which essentially set QALYs equal and highlight differences in equity variables of interest (such as severity), tend to reveal more enhanced weights for such variables with beneficiaries in more severe states being more highly valued. An improved design might have meant that the interaction between QALYs and severity could have been allowed for more comfortably, which may have shown that the shape of the relationship between severity and utility was positive when treatment provides large QALY gains – a preference shown for the most severe – and negative when treatment provides small QALY gains – a preference for the least severe. That is, when small amounts of QALYs are being offered to someone who is already in a very bad state (or who will die imminently) it might seem reasonable not to place additional value or weight on that more severe state. Indeed, this is one interpretation of the results from one other study we are aware of, which provides a result that runs counter to the usual view of severity;59 however, our data are not able to show this one way or the other. Other studies on severity, based on matching-style approaches, are discussed in Chapter 5, Concluding remarks.

Concluding remarks

The discrete choice part of the relativities study has exposed the reader to three things: the challenges of deriving relative weights for QALYs using discrete choice methods; two novel approaches to eliciting relative QALY weights from a discrete choice experiment; and some initial empirical weights.

Overall, the estimated weights are similar across different regression models and different weighting procedures. There are obviously differences between the simple model and that including higher-order polynomials due to the imposed restriction of monotonicity in the former. Comparing the CV weights with those from the probability-of-choice approach, we can see that the range of weights is smaller (0.8–1.1) for the former, and that they are generally nearer to 1, suggesting no difference between the base case and the comparison scenario. More generally, this would indicate that age and severity have little impact on QALY gains. The range for the probability-of-choice approach is 0.38–1.61. As the weights are based on the same regression model, it is unsurprising that the same scenarios result in the lowest and highest weights in both methods.

The narrow range of weights, especially using the CV approach, and the lack of statistical significance of age at onset and severity in the regression models, raise a number of issues. It might be that, when presenting information on such variables within a context in which health gains also vary, the impacts of age and severity are diminished. Indeed, this could be seen as an advantage of a discrete choice approach in which equity attributes are embedded within scenarios in which health gains (or QALYs) are also allowed to vary. However, it could also be the case that this result is driven by the compromises made necessary in the experimental design.

There are some clear patterns. For example, as the age at onset increases, the weights move towards 0 (until age 20) and then move towards 1. This shows a preference for those who fall ill between the ages of 10 and 40, with 20 being the most preferred. As age at death increases, the weights move towards 0 (until age 10) and then move towards 1 (up to age 40), before moving back towards 0. This shows a preference for individuals who will die at 1, 10 and 80 and less of a preference for those between 20 and 70. As severity increases, there is a preference for those who are slightly ill, a QoL loss of 0.4 and 0.1, compared with those who lose 0.7 and 1. Of course, the advantage with these approaches is that it is possible that age effects outweigh severity effects, or vice versa, meaning that generalisations are harder to make unless we can invoke ceteris paribus assumptions. In other words, different weights on QALYs are required for a variety of contexts.

With respect to more methodological issues, we have demonstrated two novel approaches to deriving weights from our empirical data. First, what issues arise from people’s perceived similarities or differences in the weights? At a conceptual level, the CV and probability-of-choice approaches to deriving relative weights are similar. However, instead of equalising the probability of choice between the two health states, the CV method equalises the utility associated with the two health states – a subtle but important difference. Second, does this difference matter? One possibility, of course, is that using the CV approach overcomes one limitation of the probability approach in that the CV can be used to calculate weights per attribute in addition to weight for entire health-state scenarios.

The controversy of these results lies in the small impact of the equity variables of interest (age and severity). However, this should not necessarily be taken as an indication that weighting of QALYs is not desirable. In the short term, the results need to be reinforced by calculation of confidence intervals around the weights that have been presented. This is a significant task; hence such data have not been presented in this report. In addition, despite the controversial nature of this result, as mentioned above, it may have arisen due to one potential advantage of a discrete choice approach, whereby variations in equity attributes are considered alongside variations in health gains. Our main longer-term recommendation would, therefore, be for more research, as replication of such a result would be crucial prior to any subsequent policy recommendations. This research should also pursue alternative experimental design strategies in order to address the issue of implausible scenarios while still maintaining desirable design properties, so ensuring we can estimate the effects of interest with improved efficiency. Further important potential challenges, such as the nature of the (multiplicative) functional form used in the statistical analysis and the transformation of variables from the original forms in which they were presented in Table 6, are discussed further in Chapter 5, Concluding remarks.

Basic approach

We report here the ‘matching’ relativities study which, along with the DCE study (see Chapter 4), set out to determine whether the UK public wish to attach more weight to some QALYs than to others. To recap, a QALY is 1 year in full health and years spent in less than full health are ‘quality adjusted’ in order to take account of the morbidity associated with disability, disease or illness. As QALYs combine morbidity (QoL) and mortality (length of life) on a single scale, they allow comparisons to be made across interventions with different types of health outcomes (e.g. QoL enhancing versus life extending). In the standard ‘unweighted’ QALY model, all QALYs are of equal value. For example, the age of recipients does not matter, as long as the QALY gain is the same. Likewise, the standard model assumes that equal QALY gains are of equal value regardless of how severely ill the patients are prior to treatment. The aim of the matching – and DCE studies – is then to address the question of whether a QALY is a QALY is a QALY. Or put another way, is the ‘standard’ model correct?

While each respondent in the survey answered a set of matching and discrete choice questions, we concentrate here on the results of the matching study, the main aims of which were:

• to estimate the relative weights to be attached to a QALY according to the age of recipients

• to estimate the relative weights to be attached to a QALY according to the severity of illness of recipients.

The secondary aims of the matching study were to assess the impact of certain other attributes considered to be of interest to respondents and which featured in the design such as gain in life expectancy and gain in QoL.

Methods

The QALY grid approach

Recall that the animated powerpoint introduction to the CAPI used a graph with QoL (from 0% to 100%) on the vertical axis and Age (from 0 to 80) on the horizontal axis. Clearly, all questions posed in the relativities studies – both matching and DCE – must involve health gains that fall within this defined space. Further, all questions must be ‘logical’ in that there must be some positive gain in terms of either QoL or length of life, or both. While it was necessary to incorporate such considerations explicitly into the DCE design (the underlying principle of which is to allow all attributes to vary independently), ‘illogical’ scenarios were ruled out a priori in the matching design. There remain, however, an enormous number of potential pairings that can be made, and the challenge was to come up with the most parsimonious design that allowed all the research questions to be addressed. We were keen that the design be capable of testing the standard ‘unweighted’ QALY model, i.e. where all relative weights would be equal to one, as well as being able to detect systematic departures from this baseline assumption with respect to the age of patients and their ‘untreated’ QoL. To that end we developed the ‘QALY grid’ approach, summarised briefly below.

Consider again the graph with QoL (from 0% to 100%) on the vertical axis and Age (from 0 to 80) on the horizontal axis. By partitioning the vertical into five 20% ranges and dividing the horizontal into four 20-year intervals, we get 20 ‘cells’ (numbered 1–20 in Table 13).

TABLE 13 - The QALY grid

QoL, quality of life

QoL	A	B	C	D
80–100%	17	18	19	20	P
60–80%	13	14	15	16	Q
40–60%	9	10	11	12	R
20–40%	5	6	7	8	S
0–20%	1	2	3	4	T
Age	0–20	20–40	40–60	60–80

Improving a person’s health by any one cell, i.e. by 20% per year for 20 years, would, under the ‘standard’ model, give four QALYs. But are all QALYs weighted equally, irrespective of where they are located in the grid? For example, suppose that type A people were those whose QoL would, if untreated, be 60% at (and after) age 20 and treatment could raise that QoL to 80% for the next 20 years, whereas type B people would be at 80% from the age of 60 but could receive treatment that would restore them to 100% for the subsequent 20 years. Both types would gain four ‘standard’ QALYs; but do respondents give both treatments equal priority? If not, which group of 100 do they give priority to? And if the number of beneficiaries in that group is reduced, at what value of X < 100 do the two treatments receive equal priority? For example, if a respondent finds it hard to prioritise when the same total expenditure of resources could treat either 40 people of type A or 100 people of type B, that would suggest that such a respondent gives two-and-a-half times as much weight to a QALY in cell 14 as in cell 20.

If we have data on respondents’ relative values for the 20 cells, and the value of a ‘reference’ cell is set equal to 1 (suppose we call that v1), we can derive a set of relative weights (v2–v20) for the remaining 19 cells in the grid. Our basic aim was to estimate these relative values. If each cell is paired with every adjacent cell, we can estimate ratios such as v1/v2 and v1/v5. This requires 31 pairings. An example of a question respondents were actually presented with in the matching study is reproduced in Figure 14.

FIGURE 14.

Matching question from CAPI.

While this is a parsimonious design, extrapolating from these 31 ‘single cell’ questions requires a number of strong assumptions about the nature of respondents’ preferences; assumptions we did not feel able to make without first testing their validity.

So another 41 questions in the overall design are intended to embody a number of such tests. Some of these questions look at blocks of cells: for example, each row P–T is paired with every adjacent row, and each column with every adjacent column. One important difference between the ‘whole column’ and ‘single cell’ pairings is that the former necessarily involve ‘life-extending’ QALYs (as they offer an additional 20 years of life expectancy). For example, pairing cells 10 and 11 asks respondents to consider an additional four QALYs (moving from 40% to 60% health for 20 years) either between the ages of 20 and 40 or between the ages of 40 and 60. In contrast, pairing columns B and C asks respondents to consider extending life (at 100% health) for 20 years at the age of either 20 or 40. If respondents were to have preferences over ‘life-extending’, rather than ‘life-enhancing’ QALYs, we might expect the results of the questions involving the whole column to be somewhat different than implied by the ‘sum of the parts’, i.e. those pairings of the single cells that make up the columns.

The design also allows us to test for consistency via chaining with equal numbers of steps, e.g. (v₁/v₅)*(v₅/v₉) should equal (v₁/v₉). To test for chaining with different numbers of steps, we have built in some non-adjacent pairings. This has been done for single cells (e.g. compare the pairs {1, 2}, {2, 3} and {1, 3}), for whole columns, e.g. compare {A, B}, {B, C} and {A, C} and for whole rows, e.g. compare {R, S}, {S, T} and {R, T}.

Finally, there are the tests for independence of the profile in which a cell appears. Consider the different ways in which cell 14 may appear in a scenario. In Figure 15, 14S shows the patient to be in good health until aged 20 when – without treatment – their QoL falls from 100% to 60% and they die at the age of 40.

FIGURE 15.

The profile tests (cell 14).

This depicts the ‘standard’ representation that applies to all other single cell comparisons used here. In 14X the patient is at 80% of good health until the age of 20, when their QoL falls to 60%, and again they die at the age of 40. In 14Z, the patient is at 80% of good health until the age of 20, when their QoL falls to 60%, but now they live until the age of 80. Finally, in 14Y the patient is in good health from birth and lives to the age of 80. Each profile test compares the ‘standard’ representation with one of the three other representations of that cell (i.e. 14S versus 14X, 14S versus 14Z, 14S versus 14Y). Three different sets of profile tests were carried out, involving cells 7, 11 and 14. ‘Fair innings’ type arguments would suggest a tendency, all else being equal, to favour those patients whose total lifetime QALYs were lower.

For each of the three cells used in the profile tests, a ‘fair innings’ type argument would predict that more weight would be given to S than to Y (as the total lifetime QALYs is less in S than in Y), but S would be given less weight than X (as the total lifetime QALYs is greater in S than in X). Although not shown in detail here, S would be given less weight than Z in the cases of cells 7 and 11, but S would be given more weight than Z in the case of cell 14.

The 72 questions were spread evenly across 12 versions of the matching questionnaire, each respondent answering six matching questions (along with eight DCEs).

Results

Concluding remarks

We reported here the results of a computer-assisted ‘matching’, or person trade-off, valuation study carried out in a sample of 587 members of the public in the UK. The results showed that, when asked questions of the type posed here, respondents do differentiate between ‘types’ of QALYs and are willing to trade off (sometimes considerable) numbers of patients treated in order to prioritise according to age and severity of illness. The matching data show a general tendency to give more weight to younger patients and to those in poorer health, but the pattern is not a simple one. In particular, less weight is given to the youngest patients and to those in poorest health, reversing the pattern found elsewhere in the data. As these patterns may be observed in the simple choice data from the first matching iteration (when there were 100 patients in each group), they are robust to differences in methods used to aggregate the data.

The qualitative findings reported in Chapter 3 may offer some explanation for the pattern of results uncovered here. Recall that age was found to be important to focus group participants for a variety of reasons including life expectancy, which would explain the general tendency to favour the young over the old. Recall also, however, that age was found to be used as a ‘proxy’ for a range of other characteristics including economic activity and the existence of dependants. One possible explanation for the tendency to favour 20- to 40-year-olds over younger patients, therefore, is that such considerations outweigh the impact of life expectancy around those ages. If we were to represent the 20 discrete choice weights with the 20 cells of the QALY grid, the peak (most preferred cell) would be cell 14. This is consistent with the most preferred age group from the matching data (20–40), but implies that less-severe states are preferred (60% to 80%), although the magnitude of the weights is obviously very different.

Returning specifically to the matching data, it is also reported in Chapter 3 that respondents expressed the view that an improvement in QoL for people in poor health would be more important than an identical improvement in QoL for people in relatively good health. Again, this would seem to be consistent with the general tendency to favour those in more severe health states initially. Some concern was also expressed that prolonging life in a really bad health state was not desirable. One possible explanation for why respondents favoured a move from 20% to 40% health over one from 0% to 20% is that 20% was still considered to be not worth living. Indeed, one of the respondents quoted in Chapter 3 almost makes this point explicitly: ‘whereas 20% is pretty close to death.’ This raises issues about how the QoL axis – which necessarily relied on a ‘visual analogue scale (VAS)-like’ representation in order to put the concept across to respondents – was being interpreted, particularly towards the lower end.

It is also possible that the tendency for the weights to ‘dip’ towards the top right-hand corner of the grid may be linked to how respondents were interpreting the QoL scale. While the diagrammatic representation of QoL over the patient’s lifetime made it clear that the QoL related to the relevant stage of the patient’s life, such that 100% health at the age of 70 meant in good health for a 70-year-old, it may be that certain respondents were considering QoL as an absolute measure. If this were the case, then it might have been considered that 80% health, for example, was the best that old people could reasonably expect to achieve and little was to be achieved by trying to improve their health beyond this. If there were a certain degree of this going on when respondents were answering the matching questions, this raises the possibility that the weights estimated here are perhaps penalising the ‘healthy old’ rather too heavily.

While the general pattern of weights estimated for the QALY grid is robust to the different methods of aggregating the data, clearly the magnitude of the weights differs markedly. The ratio of means method of aggregating the data is the more conservative of the two methods deployed here and has been used to derive relative weights elsewhere in the economy. 60 It is important to acknowledge, however, that theoretical arguments exist in favour of other methods of aggregating the data. It is not our intention to go into these issues further here as decisions regarding the appropriate magnitude of any weights used for policy purposes will clearly be guided by additional considerations, such as acceptability and practicality.

We conclude this section by comparing how the magnitude of the weights estimated here compares with those reported in previous studies. The review paper by Dolan and colleagues16 cites a number of studies that have set out to estimate age-weights for life-years gained that have come up with a range of values. For example, Johannesson and Johanesson61 report weights for life-years gained at ages 30, 50 and 70 years of 1.0, 0.22 and 0.1 respectively, indicating a 10:1 differential between the most preferred and least preferred age groups. Adjusting for QoL affected these estimates somewhat, with three QALYs gained for a 50-year-old or nine QALYs gained for a 70-year-old judged equivalent to one QALY gained for a 30-year-old. Similarly, Nord and colleagues11 derived weights for life-years gained at ages 10, 20, 60 and 80 of 1.1, 1.0, 0.4 and 0.1 respectively, again indicating a more than 10:1 differential between the most preferred and least preferred age groups. This pattern was very similar when life-improving, rather than life- extending, interventions were considered. Finally, Busschbach and colleagues62 compared QoL improvements at ages 5, 10, 35, 60 and 70 and estimated weights for the utility of health at these ages to be 0.2, 1.5, 1.0, 0.7 and 0.7 respectively, indicating a 7.5:1 differential between the most preferred and least preferred age groups. It is interesting to note that these weights are more extreme than even the less conservative of our two methods of aggregating the data.

In addition, a number of studies have looked at the impact of severity of health state and found it to be an important factor, although the results are mixed. Nord and colleagues11 and Ubel and colleagues63 asked subjects to compare improvements on a disability scale (where lower numbers indicate better functioning) with approximately equal distances between the levels. Respondents were asked to indicate how many patients moving from level 5 to level 1 was equivalent to moving a smaller number of patients from level 6 to level 4. Both studies reported a marked preference for treating the more severely ill patients. In contrast, a recent study by Dolan and Tsuchiya64 found that respondents gave consistently higher priority to patients with better prospects without treatment, indicating that the more severely ill attract lower priority than the better off, a finding which remains unexplained and is perhaps more in line with the result on severity in Chapter 4.

Basic strategy

Other Government departments/agencies, e.g. the Department for Transport (DfT), the Department for Environment, Food and Rural Affairs (DEFRA) and the HSE, have drawn on the stated preferences of representative cross-sections of the population to assign monetary values to the prevention of injury, illness and premature death. If it were possible to obtain a comparable preference-based monetary value for a QALY, this would offer the prospect of being able to apply cost–benefit analysis to an even broader range of public health and safety policies in a more coherent and consistent manner – and in a way which better reflects the values of the people who are paying for, and benefiting from, those policies.

So the objective of this part of the study was to explore the feasibility of eliciting such a value and, in the process, raise and address the theoretical and practical issues involved in trying to do so.

As a starting point, consider the way that preference-based monetary values have been estimated for use in road safety appraisal. Typically, surveys have sought to elicit the maximum amounts that respondents would individually be willing to pay for (usually small) reductions in their own (and possibly others’) risk of being killed or injured in road accidents. These amounts are then aggregated across individuals to arrive at an overall value for the safety improvement concerned. The resultant figure thus aims to show what the safety improvement is ‘worth’ to the affected group, relative to the alternative ways in which that group might have spent its money.

For example, suppose that the members of a representative sample of the population were each asked for their WTP to reduce their risk of death on the roads during the next year by 1 in 100,000. If the mean response were £12, the implication would be that 100,000 members of the population would, between them, be willing to pay a total of £1.2M to achieve an overall reduction in risk that would, on average, reduce the number of fatalities on the roads by 1. Hence, the preference-based component of the value of preventing a (statistical) fatality (VPF) would be set at £1.2M. [While this is the largest element in the overall VPF, it is not the only one: it is usually supplemented by figures which represent the loss of net consumption entailed by premature death and some estimate of the costs (medical, police, etc.) of dealing with the consequences of a typical fatal accident.]

A similar approach could be envisaged for estimating a societal WTP-based monetary value of a QALY. The simplest case would be if people had a clear idea of what a QALY is, and could then be asked for their WTP to reduce the risk of suffering the loss of 1 QALY by 1 in 1000 (for example). If a representative sample stated a mean WTP of, say, £18 for such a risk reduction, the implication would be that 1000 members of the population would, between them, be willing to pay a total of £18,000 for some intervention that would, on average, prevent the loss of one QALY.

Other closely-related variants are possible. For example, each respondent could be asked to imagine that they face a 1 in 1000 risk of an illness which, if untreated (or treated conventionally), would entail the loss of one QALY. They could then be offered an ‘insurance policy’ which would ensure that they received treatment (or treatment over and above the conventional provision) that would restore them to their current/normal health and avoid that QALY loss. Again, if the average WTP for this insurance were £18, the aggregate value of the treatment that generates a one-QALY benefit would be £18,000.

That is the basic principle. In the remainder of this part of the report, we describe how we explored different ways of trying to put that principle into practice, and we discuss the issues raised both by the process and by the responses elicited.

Overview of the questionnaire

In practice, very few members of the public know what a QALY is. One possibility might have been to begin any interview by trying to explain the concept in a general way (as was attempted in the ‘relativities’ part of the project) and then ask individuals to think about how they would envisage personal QALY gains or losses and how much they would be willing to pay to achieve such gains or avoid such losses.

We do not rule this out as one possible way of proceeding; indeed, in the ‘relativities’ study there appeared to be a reasonable degree of receptiveness for the initial broad explanation of what QALYs are. However, such an explanation would have required quite a lot of time – especially if supplemented by some attempt to get respondents to relate the concept to their own present and future lives – and our focus was upon testing some of the assumptions underlying QALYs and their possible monetisation, rather than accepting them as uncontested. So instead of asking people to value (chances of) QALY gains/losses directly, we adopted the strategy of asking people to value avoidance of, or reductions in the risk of, illness states described quite naturalistically; and we then tried to elicit in a standard way some QoL index measure of those states. Putting the two together would – if certain assumptions hold – enable us to assign money values to QALYs; but separating the two tasks would also allow us to examine more closely whether the necessary assumptions do in fact hold.

To be more specific, the questionnaire centred around two illness states – one involving recurrent stomach/bowel problems, the other involving recurrent episodes of head pain. For each state, there were three possible durations: 3 months, 12 months and ‘the rest of your life’. And for various combinations of these, there were questions about how much the respondent would be willing to pay to prevent the certainty of the illness and how much they would be willing to pay to eliminate some risk of the illness. Having obtained these monetary responses, the questionnaire went on to elicit information about trade-offs the respondent would make in terms of the relative probabilities of different health states, including some measure of the QoL index assigned to the illness state. In theory, this trade-off information makes it possible to compute for each individual the QALY losses entailed by the various illness descriptions; and this information, combined with the individual’s WTP to avoid/reduce the risk of the various illness scenarios, should (if the underlying assumptions hold) enable us to take readings of that individual’s monetary value of a QALY.

The questionnaire can be viewed as divided into four sections. Part A asked questions about each individual’s current self-assessed state of health and then introduced them to the particular illness descriptions that constituted the focus of subsequent questions. Part B introduced the general idea of WTP for health benefits before eliciting a series of WTP values for different scenarios. These scenarios were varied between four different subsamples in order to provide broader coverage and allow certain tests of conformity (or otherwise) with standard assumptions. Respondents were allocated at random to one of the four subsamples, and the fact that Part A and the first question in Part B were common to all respondents allowed us to check for the comparability of those different subsamples. Part C consisted of four questions eliciting various risk trade-offs designed both to test the standard assumptions invoked for this part of the QALY computation and to provide a QoL/health-state index measure for the illness state under consideration. The final part of the questionnaire consisted of a series of questions collecting socioeconomic/demographic information. This allowed further checks of comparability between subsamples.

Implementation and results

There were four versions of the questionnaire, one for each of the four subsamples. A copy of Version 2, together with the relevant supporting material, is included in Appendix 6. The ways in which the other versions differed from Version 2 will be explained in the text that follows.

But first, the sample. In order to make the most of the project budget, sampling for this part of the project involved revisiting many of those who had previously participated in the ‘relativities’ part of the study and then enlarging the sample opportunistically from that base. Thus, this was a ‘convenience’ sample; and while it covered the full range of educational level, income and social class, it was not intended to be representative of the population as a whole. What is important for present purposes is that within this sample, respondents were allocated at random between the four versions of the questionnaire: our objective was to examine within-subject consistency and make between-subsample comparisons. If it is judged that the survey instruments pass the tests conducted on this basis, it will require a larger and more representative sample to provide estimates suitable for public policy.

The four versions were as follows. Versions 1 and 2 focused on scenarios about stomach illness while Versions 3 and 4 were centred on scenarios about head pain. As indicated earlier, within each version there were three durations of illness: 3 months followed by return to respondent’s current state of health; 12 months followed by return to respondent’s current state of health; and a chronic condition where the illness lasted for the rest of the respondent’s life. Each scenario was described on a separate card: the six cards are reproduced as part of the supporting materials in Appendix 7.

The key way in which Version 1 differed from Version 2, and likewise the way in which Version 3 differed from Version 4, related to the questions involving the risk of an illness: Versions 1 and 3 asked about WTP to eliminate 10% risks of the three durations of illness, while Versions 2 and 4 asked about eliminating 5% risks.

The total number of questionnaires was 409. However, after a check, six of these were excluded on the grounds that they contained very few answers to the main questions and offered little or no possibility of contributing usefully to the analysis.

The distribution of the 403 between the four versions is shown Table 26.

We now turn to the main results from the key parts of the questionnaire, starting with Part A.

Using the willingness-to-pay and standard gamble data to derive a monetary value of a QALY

In order to get an estimate of each individual’s monetary value of a QALY, our procedure involves converting an illness description into a QALY loss by using the individual’s response to Question E, and then combining that with their money value for avoiding that loss, taken from the WTP questions.

There are various ways we could combine individuals’ responses to different questions, but perhaps the simplest to explain is to take the 12-month illness, compute the QALY loss involved in that, and combine that with WTP to avoid that loss as stated in response to B5. It will be seen that the issues raised by this route are so fundamental that it is really rather superfluous to pursue the other variants, none of which circumvent those fundamental problems. So we focus on this route, which works as follows.

Consider someone who is just willing to accept a 5% risk of death in Question E. This is taken to signify that the illness state is indexed at 0.95 on the scale that runs from 1 (normal health) to 0 (dead, or as bad as being dead). In other words, each year spent in that state is rated at 0.95 of a normally healthy year and thus represents a loss of 0.05 (i.e. one-twentieth) of a QALY. If that same individual states in Question B5 that they are willing to pay £800 to avoid that loss, then 20 people like this respondent would, collectively, be prepared to pay £16,000 to avoid health losses that add up to one QALY (i.e. 20 × 0.05). Thus, this individual’s responses amount to saying that for a population consisting of similar people, the money value of a QALY is judged to be £16,000. By calculating the ‘value of a QALY’ implied by each individual’s responses combined in this way, we can derive mean and median values for this sample.

The potential problem, of course, is that if someone says they are only willing to take a 1 in 100,000 risk (or less) of death in Question E, their WTP to avoid the 12-month illness is multiplied by 100,000, potentially giving an astronomical figure for the value of a QALY. (Someone who says they are only willing to take a 1 in 100,000 risk of death to avoid the chronic illness state is taken to be indexing that health state at 0.99999, i.e. a year spent in that state is taken to amount to the loss of 0.00001 of a QALY. So if 100,000 such people were each willing to pay, say, £300 to avoid the 12-month illness, they would between them be paying £30M and their combined benefit would add up to just one QALY.) Sure enough, we find that 115 respondents in total – 59 in the S subsample and 56 in the H subsample – give responses to Questions E and to B5 which, in combination, imply values of a QALY of more than £1M. With some of these combinations generating values of thousands of millions of pounds, the mean values for a QALY are £3 × 108 from the S subsample and £7 × 108 from the H subsample. The medians are somewhat more terrestrial: £26,666.67 and £57,142.86 respectively. On the grounds that there are no statistically significant differences between the two subsample distributions, we might pool them to obtain an overall mean of £5 × 108 and an overall median of £40,000.

It has been widely accepted that when such values are being used to guide public policy, it is the mean figure which should be used as the best indicator of social welfare. However, clearly, a mean value for a QALY of £5 × 108, or even a figure one-thousandth as big as that (i.e. £500,000), would be totally anomalous in a world where the VPF is about £1.5M and where this, in the context of road accident fatalities, represents preventing a death which on average entails the loss of about 40 years of life expectancy.

Nevertheless, applying the procedure in the way described does generate the kinds of extremely skewed distributions reported, and that does produce means which are pulled up by some extremely high individual figures.

This raises the question of whether there is something that could be done – some other way of managing the procedure or analysing the data, perhaps – that might give figures more compatible with those in use in other areas of public health and safety policy?

As a start to answering this question, recall what is involved in a study to elicit the value of preventing a (statistical) fatality or the value of preventing a particular (statistical) injury. Typically, respondents are presented with a description of a specified physical outcome – death, or a particular injury description – and are asked for their WTP to reduce their risk of experiencing this outcome by some given amount. We then effectively sum the responses over a representative population big enough that their individual risk reductions add up to preventing the (statistical) expectation of one death or one injury of the particular type.

In the course of this procedure, no attempt is made at the individual level to ascertain the number of QALYs each individual perceives they would lose in the event of being killed or injured. In principle, this could be done. Were things to be done that way, it might very well be that the combination of some individuals giving high WTP with responses by them implying very small individual QALY scores would produce a similar highly-skewed distribution and upward pull on means. But as things stand, it has not been the usual practice. So, in the light of the way in which things actually have been done, what might be the implication for the present study?

Take the case of a non-fatal road injury, which is a closer analogue than a fatality to the illness descriptions used in the present study. Call this road injury ‘J’. Typically, respondents have been given descriptions of what J involves, have been asked to think how such an injury would impact upon their life, and have then either been asked for their WTP to prevent/reduce the risk of such an injury or expressed some relativity between the injury and death. In this latter case, the value of preventing each J is then ‘pegged’ against the VPF according to the average relativity expressed between J and death. But however it is arrived at, for the sake of example, let us suppose that the process generates an average money value for preventing injury J of £60,000.

If one were to want to go further and infer a value of a QALY from the established value of preventing injury J, the most obvious way of doing so would be to undertake a survey eliciting from a cross-section of the population their judgement of the QALY loss they would suffer if they sustained injury J. Again, for the sake of example, suppose that responses to such a survey ranged quite widely from one individual to another but that the average judged QALY loss turned out to be 2.5 QALYs.

On this basis, a public body undertaking a road safety scheme costing £600,000 which is expected to prevent 10 cases of injury J could (just) justify that expenditure on cost–benefit grounds (assuming all other projects yielding an excess of benefit over cost were also undertaken). And although this body could not say in advance which particular individuals would benefit from the scheme by avoiding injuries they would otherwise have suffered, it could assert that on average the benefit would amount to preventing a loss of 25 QALYs, translating to an average value of £24,000 per QALY.

We can mimic that kind of calculation on the basis of the data collected in the feasibility study if it is processed in the following way.

First, we have reported in Table 29 the WTP-based mean values for avoiding the 12-month durations of S and H: respectively, £1867.37 and £3252.35. Let us round these figures to £1870 and £3250. Had our sample been a large, representative cross-section of the population, it could have been argued that health-care resources allocated to preventing/curing such illnesses could (just) be justified if they cost those amounts for every 12-month episode prevented/cured.

Then we might ask what QALY gains would result from such expenditure. We cannot know which particular individuals will benefit, but on the basis of our sample members’ responses to Question E as summarised in Table 37, together with the standard QALY procedure for combining health-state indices with periods of time spent in a health state, we could say that, on average, each avoided case of 12-month S would avoid a loss of 0.104 of a QALY, while each avoided case of 12-month H would generate an average benefit of 0.144 of a QALY.

Putting the relevant figures together, we would have an expenditure of £1870 on 12-month S, bringing an average QALY benefit of 0.104, which translates to a money value of about £17,980 per QALY, while each £3250 spent preventing 12-month H would ‘save’ 0.144 QALYs, implying a value per QALY of approximately £22,570.

Processed this way, the data obtained via two different health-state descriptions produce figures reasonably close to each other on either side of £20,000 per QALY. But why are these figures so very different from the astronomical means based on the value-per-QALY figures generated by combining Question E and B5 responses at the level of each individual? And which – if either – is the correct basis for deriving an estimate?

The answer to the first of these questions is fairly straightforward and can be illustrated by an example (which may also be helpful when trying to answer the second question).

Consider a subsample of 10 people. In order to keep things simple, suppose each of them states the same WTP value to avoid 12 months of H – let us say £250 (which works out at about £10 for each fortnightly episode avoided). Suppose also that in response to Question E, 8 of these 10 would take a 1% risk of death in order to get a 99% chance of avoiding H for the rest of their lives, while one person would take a 2% risk and the remaining member of the subsample would only take a 0.1% chance. Averaging those responses gives an average QALY loss of 0.0101 per 12-month illness which, combined with the average WTP of £250 to avoid it, generates a value per QALY of just over £24,750.

Contrast this with the figure given by first combining each individual’s WTP with their Question E response and then averaging. For eight respondents, the implied value of a QALY is £250 ÷ 0.01, which gives £25,000. For the respondent prepared to take a 2% risk, the figure is £250 ÷ 0.02, which gives £12,500. And for the respondent who will only accept a risk of 1 in 1000, the calculation is £250 ÷ 0.001, which produces £250,000. On this basis, the subsample mean is £46,250, i.e. almost double the figure arrived at by the other method.

The reason for the difference is this. Under the first method of computation, the two responses which differed from 0.01 were added to each other and to the eight 0.01s; and as they diverged from 0.01 in different directions but by almost the same amount, (+0.01 in one case, –0.009 in the other), this adding and averaging more or less cancelled them out, so that the subsample average diverged from 0.01 by just 0.0001. The net effect of this was to nudge the computed value of a QALY down to a little below £25,000.

However, under the second computation method, the (reciprocals of the) probabilities operate multiplicatively on the WTP responses before any averaging occurs. Thus the difference between 0.01 and 0.02 acts to halve the implied value of a QALY for that individual from £25,000 to £12,500 – a money difference of –£12,500. At the same time, the slightly smaller difference in the opposite direction between 0.01 and 0.001 acts to multiply by 10 the implied value – producing £250,000 as compared with £25,000 – a money difference of +£225,000. When added and averaged at this stage, the two ‘outliers’ nowhere near cancel each other out: on the contrary, one individual’s implied value comes to more than all of the other nine put together, and this has the effect of almost doubling the mean.

So that is why the two methods of computation produce very different results; and the divergences in the example, striking though they may be, are dwarfed by the divergences in the actual sample, where still smaller probabilities had the effect of multiplying WTP responses by tens and hundreds of thousands, and even by millions.

So which is the correct method to use? In theory, if our analysis is based on the premise of each member of the population valuing the same QALY gain, it is the second method, i.e. the method used initially in the analysis of the survey data which generates stratospheric means. But, as with any theory, the validity and usefulness of the results depend crucially on the extent to which the underlying assumptions are valid. If we subscribe to the conventional precepts of welfare economics and if we could be confident that individuals have values and preferences which conform with standard assumptions and that their responses to our questions reveal those values with total accuracy and precision, the second method would be the appropriate one to use. In the light of the example which shows how one person’s value can outweigh the values of nine others, this may seem a surprising conclusion; but if all 10 really were reporting their true values accurately, and if the guiding principles entail giving each person and their values equal weight, then taking the mean of that distribution, however skewed, is the appropriate thing to do.

But what if certain assumptions do not hold perfectly? In particular, what if people are not always able to report their values with total accuracy, but are liable to give responses which contain elements of ‘noise’, error and/or bias? We are still some way from having very good models of the noise, error, bias and imprecision in human judgement; but we know enough to appreciate (a) that such things exist and (b) that they may not always be neutral or ‘white’ in their effects. So although we cannot say categorically how to model these factors, let us consider how they might impact upon the results of the present feasibility study.

Consider again the example set out a few paragraphs earlier. Suppose that all 10 respondents not only have the same ‘true’ WTP to avoid the illness, but also would have their QoL diminished by the illness to exactly the same degree – that is, by 1%. If they each reported their values and judgements with complete accuracy, we should infer a mean value of a QALY of £25,000.

However, suppose that they do not all process probability judgements with unerring accuracy. To keep things simple, suppose that just two of them give erroneous responses – one reporting a willingness to accept a 2% risk, the other setting the ‘break-even’ risk of failure at 0.1%. Of course, this is only an illustration: we do not know enough about judgemental error to say exactly which and how many deviations from the underlying ‘true’ values are likely to occur. The point is, however, that if those two responses involve (seemingly small) errors, they can throw the estimates off in different ways and to very different degrees, depending on how they are processed. In the case of the first (and theoretically vulnerable) method of taking the arithmetic mean WTP separately from the arithmetic mean of the failure risks and then deriving a value of a QALY on the basis of the combination of these two means, the errors act to produce an estimate which is roughly £250 (and thus about 1%) below the true value; whereas in the case of the second (and theoretically preferable) method, the errors act multiplicatively before any averaging process occurs, and thereby produce an estimate which is £21,250 (and thus 85%) above the true value.

No general claim is being made here about the relative performance of the two methods: this is just a simple and convenient, albeit stylised, example. But what we do know is that people generally find probabilities quite challenging to manipulate, and their responses are liable to deviate a good deal, and in ways we do not well understand, from what is assumed by standard decision theories. (Recall the evidence relating to the sums of responses to Questions D and F, as reported in Tables 35 and 36.) So a method of calculation which is especially liable to magnify ‘errors’ by including them multiplicatively – and which we know in this case is liable to produce phenomenally large and implausible values – seems hard to justify as a basis for public policy.

Unfortunately, there is no way of knowing with any great confidence whether any serious distortions occur as the result of using the other method outlined here. It may be, as in the example, that this method has a tendency to understate the ‘true’ value. But equally, it is possible to produce examples where that is not the case; and even if it were the case in this study, we know too little about the nature of the imprecision/error in people’s responses to say by how much any estimates diverge from the ‘truth’. Perhaps the most that can be said is that this method, flawed though it may be, seems broadly compatible with the ways in which other values used by government departments have been derived; and to the extent that there is a desire to have a monetary value for a QALY established on a similar footing to the values used in DfT, DEFRA, etc., there appears to be some argument for this method of estimation, while keeping all the caveats in mind.

Concluding remarks

Speaking of caveats, it is important to reiterate that even if the data were unproblematic and the calculation of the monetary value of a QALY were straightforward, no value generated by this feasibility study could be regarded as a sound basis for policy because the sample used here was not representative of the population. The question is, then, whether the present study gives grounds for believing that (at least some of) the techniques explored here could be used in conjunction with a large and genuinely representative sample to produce a value robust enough to be used to guide policy, or whether the evidence suggests that no sufficiently robust value is likely (ever) to emerge.

There were a number of encouraging features. Interviewers generally found that respondents were engaged and interested, and very few interviews were aborted or resulted in large numbers of missing values. And even though many people in the UK are still somewhat resistant to the idea of paying for health care over and above tax contributions, the very small number of zero responses to B4 and especially B5 suggests that the ‘protest’ element was small and that people were willing to ‘play the game’. Moreover, although the responses to B4 and B5 did not display the ideal 4:1 ratio that full sensitivity to duration might have elicited, there was a degree of sensitivity that could arguably have been compatible with the effect of budget constraints.

However, there are also a number of discouraging features. Although by no means new or unexpected, there was serious between-sample insensitivity to the size of the risk being eliminated in questions B6–B8. And although responses to these questions should have been less affected by budget constraints and should therefore have shown greater sensitivity to duration, they did, if anything, exhibit rather less sensitivity to the 3-month:12-month difference; and also seemed to greatly underweight the ‘remaining lifetime’ duration relative to the 12-month scenario, with mean responses to B8 never more than 5 times higher than the corresponding means for B7, even though average remaining life expectancy was in the region of 30 years. It appeared that the use of probabilistic questions added complexity and dulled sensitivity.

Thus it may come as no surprise that some of the Part C questions, which used probabilities as their main ‘currency’, were also problematic. Although responses to Question D showed greater sensitivity to duration than was exhibited in the WTP questions, the mean response still appeared to greatly underweight remaining life expectancy relative to the next year. The summation of responses to Questions D and F showed a very substantial and seemingly systematic departure from the behaviour that would be consistent with standard assumptions.

It is hard to say whether the responses to Question E are ‘reasonable’ or not. Certainly, the distributions are heavily skewed: for both S and H the measure of skewness is greater than 2. Between a quarter and a third of respondents gave a ‘failure risk’ greater than the mean, while more than a third were unwilling to accept a risk of failure of 1 in 1000. Focusing on this latter statistic, does it really seem plausible that a condition involving significant disruption of at least some activities for between 8 hours and 3 days every couple of weeks entails a loss of welfare of less than 0.1%? Or is it that eliciting a response by means of a question involving the risk of immediate death induces ultraconservative responses from a significant minority of respondents? This is an open question. But if such responses are combined at the individual level with average WTP responses, they imply huge values of a QALY of the kind that generate the implausibly high means reported in this study.

On the other hand, if we compute means for B5 and for Question E separately and then use the ratio of these means to estimate the value of a QALY, the two figures obtained via different illness state descriptions are reasonably similar and not obviously outlandish.

Overall, then, this feasibility study sounds many notes of caution and points to a number of issues – particularly concerning the way respondents process probabilities and the extent to which their answers are sensitive to key dimensions – that would require further investigation before investing in a large representative national survey. Such questions are potentially amenable to investigation using qualitative methods and entailing considerable cognitive testing during piloting, and it would be unwise to embark on a large-scale study to generate policy values without first undertaking extensive (and probably expensive) preparatory research of this kind.

Weighting QALYs

The main results of the discrete choice and matching approaches show very different sets of weights for age and severity. In broad terms, the discrete choice results might be taken as suggesting that it is not worth weighting QALYs at all. By contrast, significant weights can be inferred from matching data, with gains for some groups being weighted up to 2.75 times higher than for others when using the more conservative aggregation method, the ratios going up to 4:1 using the less conservative method. Despite such differences in magnitude, discrete choice and matching weights did show similar patterns across age (but not severity) ranges.

There are two perspectives to take on the differences in results. First, it could be said that it would be premature to propose any particular set of QALY weights at this point in time: before that point is reached, there is scope for both further reconciliation and replication. Second, it might be argued that there is no scope for reconciliation and we need to choose between the results in light of the caveats of each.

Reconciliation and replication

With respect to the reconciliation of our findings it could, of course, be argued that it is not surprising that the discrete choice and matching methods led to different results. One factor has already been pointed out. The matching study involved holding health gains constant between the two options in any pairwise choice, and varying the age and severity attributes, while the discrete choice questions presented a pair of scenarios with different health gains as well as levels of age and severity. It might be, therefore, that the matching procedure highlights age and severity while the discrete choice method dampens down their impact when the size of health gain is also varied. Indeed, if respondents have lexicographic preferences, whereby health gain matters above all else, it could be argued that the results of the discrete choice and matching approaches are entirely consistent. If this were the case, then the policy implications would require very careful thinking through and would pose major challenges to matching-based approaches in which choices presented hold QALY gains constant. Nevertheless, continuing to focus on the differences, there are additional limitations in the design of the discrete choice study which indicate caution around interpretation of results.

Other potentially important differences between the methods are listed in more detail in Appendix 9. Despite these, given that each respondent was asked a set of discrete choice questions and a set of matching questions, further detailed analysis may shed light on reasons for the differences.

Decision heuristics provides another possible avenue to reconciliation. First we have mentioned, in Chapter 5, that the lowest weight attached to the 60- to 80-year-olds in full health may have resulted from respondents thinking that 80% health is good enough for someone in such an age group. The lack of graphical representation of numbers of people in the matching study may have detracted respondents from the ‘brutal’ nature of trade-offs between persons, which, if dealt with otherwise, may have led to smaller trade-offs. With respect to the discrete choice approach, the simplest decision heuristic would be to compare the size of the areas representing the health gain, i.e. to maximize health and ignore age and severity. If each of these arguments had some validity, the ‘true’ result would be somewhere between the two.

It may be significant that the discrete choice and matching studies use different functional forms for age and severity weights.

In the discrete choice study, utility is modelled as a function of ‘age at onset’ (AO), ‘age at death if untreated’ (AD), ‘quality of life lost without treatment’ (QL), and ‘QALYs gained from treatment’ (QALY). Thus, the discrete choice functional form measures the health gain from an intervention by the size of the dark shaded area in Figures 2 and 3, i.e. it measures the health gain as the total number of QALYs. The functional form then treats this measure of ‘total QALY gain’ as one variable. The other three variables describe the lifetime health profile without treatment (the light shaded area). So, the weights produced by the discrete choice study are based on the properties of the light shaded area and the total area of the dark shaded area.

In contrast, the matching study focuses on the properties of the ‘dark shaded area’ of health gains. Recall that this study uses a ‘QALY grid’ of 20 cells, defined and numbered as in Figure 16. Each cell is assumed to have a subjective weight, which is applied to any health gains that occur in that cell. The subjective value of an intervention is modelled as the sum of the weighted health gains that are generated.

FIGURE 16.

A health gain in the QALY grid.

For example, consider the following case. In the absence of treatment, the lifetime health profile is given by AO = 10, AD = 20 and QL = 0.7 (i.e. the patient is in full health to age 10, drops to 30% health from ages 10 to 20, and then dies). With treatment, the patient is maintained at 50% health from age 10 to age 30, and then dies. The health profile without treatment is represented by the light shaded area in Figure 16; the health gain from treatment is represented by the dark shaded area. The discrete choice study treats the effect of treatment as an undifferentiated gain of seven QALYs (i.e. an increase of 20 percentage points for 10 years, plus an increase of 50 percentage points for 10 years). The weight given to these QALYs is determined by the values of AO, AD and QL. In contrast, the matching study treats the effect of treatment as the creation of two QALYs in cell 2, one QALY in cell 5, two QALYs in cell 6 and one QALY in each of cells 9 and 10; each of these QALYs is given the weight of the cell in which it is located. Denoting the weight of each cell i by wi, and the QALY gain in each cell i by qi, a simple additive functional form can be specified as follows:

⁽⁶⁾ U = w 1 q 1 + … + w 20 q 20 .

It is possible that the difference between the functional forms used in the two studies is responsible for the differences in their results. We tried to test this explanation by estimating equation (6) using the discrete choice data. One might expect that if the differences between the results of the two studies were primarily due to the different functional forms used, this estimation exercise would generate weights similar to those found in the matching study. Alternatively, if it generated weights with some other systematic pattern, that might point towards some other explanation of the differences. (For example, if the weights were approximately equal in all cells, that would give further support to the conclusion that, when answering to discrete choice questions, respondents do not weight QALYs according to age or severity.) In fact, the estimated weights were highly variable, with no apparent pattern. The Bayesian information criterion clearly favoured the original model (described in Chapter 4, Predicted probability of choice approach) rather than equation (6). The lack of structure in these estimates is probably the result of econometric problems caused by multicollinearity in the data. The basic problem is that, in a typical discrete choice question, the health gains created by each intervention occur in cells that are adjacent to one another, creating strong positive correlation between the qi values for adjacent cells. This reflects the fact that the questions used in the discrete choice study had been designed to estimate a very different model.

Procedurally, too, the exercises are very different (the results of the matching questions are derived from numbers being treated, which is always constant in the discrete choice) and discrete choice is perhaps a more complicated task (there are more things varying at any one moment, such as the health gain). Other such differences are outlined below. As pointed out in Chapter 1, these further differences represent concerns that each group within the Team, largely Newcastle-led and UEA-led, have with the piece of relativities work on which they did not lead. The Newcastle-led group think that both exercises have their merits, whereas the UEA-led group stands by the methods and results from the matching study.

Replication is also important. Aspects of each of these approaches are new and their applications to deriving relative weights for QALYs have generated significant challenges. Further survey work is required to inform the debate on weighting of QALYs with more confidence. With respect to discrete choice methods, as we have said, we would recommend pursuit of alternative experimental design strategies in order to address the issue of implausible scenarios while still maintaining desirable design properties, so ensuring we can estimate the effects of interest with improved efficiency.

The feasibility of estimating a monetary value of a QALY

It was never the intention in the valuation study to produce a monetary value of a QALY for use in policy deliberations. The purpose was simply to assess the feasibility of estimating such a value.

There is scope to use the current data set to further explore different ways of combining WTP and SG data to arrive at a value of a QALY, and to examine the sensitivity of results to different assumptions about noise in the data – and especially in the probability responses generated by the SG questions. Further consideration needs to be given to aggregation issues: for example, the weight to attach to means versus medians, and whether to use means of individual WTP/SG combinations or combinations of mean WTP and mean SG values.

The feasibility study has demonstrated that there are considerable challenges involved in trying to elicit a robust monetary value of a QALY. However, the low number of protest responses demonstrates that respondents are comfortable with these types of question. Any future national sample survey should be preceded by further extensive qualitative research and cognitive testing to resolve the main questions identified in the present study.

Implications for research and practice

Contribution of authors

Rachel Baker contributed to the design of the study, analysis and interpretation of data, and drafting and editing of the report. She led the Q methodology and qualitative analysis and drafted Chapter 3. Ian Bateman contributed to the conception and design of the study, analysis and interpretation of data, revision of the report and approval of the final version. Cam Donaldson (lead author and principal investigator) contributed to the conception and design of the study, analysis and interpretation of data, and drafting and editing of the report. Michael Jones-Lee, Jose Luis Pinto Prades, Mandy Ryan, Phil Shackley, Richard Smith and Robert Sugden contributed to the conception and design of the study, analysis and interpretation of data, and revision and editing of the report. Emily Lancsar and John Wildman contributed to the design of the study, analysis and interpretation of discrete choice data, and drafting and revision of the report. Graham Loomes contributed to the conception and design of the study, analysis and interpretation of data, and drafting and editing of the report. He led the valuation study and drafted Chapter 6. Helen Mason contributed to the design of the study, analysis and interpretation of data, and revision and editing of the report. Maria Odejar contributed to the analysis and interpretation of discrete choice data. Angela Robinson contributed to the conception and design of the study, analysis and interpretation of data, and drafting and editing of the report. She led the analysis of person trade-off relativities study data and drafted Chapter 5.

Disclaimers

The views expressed in this publication are those of the authors and not necessarily those of the HTA programme or the Department of Health.

Health Technology Assessment reports published to date

1. Home parenteral nutrition: a systematic review.

   By Richards DM, Deeks JJ, Sheldon TA, Shaffer JL.

2. Diagnosis, management and screening of early localised prostate cancer.

   A review by Selley S, Donovan J, Faulkner A, Coast J, Gillatt D.

3. The diagnosis, management, treatment and costs of prostate cancer in England and Wales.

   A review by Chamberlain J, Melia J, Moss S, Brown J.

4. Screening for fragile X syndrome.

   A review by Murray J, Cuckle H, Taylor G, Hewison J.

5. A review of near patient testing in primary care.

   By Hobbs FDR, Delaney BC, Fitzmaurice DA, Wilson S, Hyde CJ, Thorpe GH, et al.

6. Systematic review of outpatient services for chronic pain control.

   By McQuay HJ, Moore RA, Eccleston C, Morley S, de C Williams AC.

7. Neonatal screening for inborn errors of metabolism: cost, yield and outcome.

   A review by Pollitt RJ, Green A, McCabe CJ, Booth A, Cooper NJ, Leonard JV, et al.

8. Preschool vision screening.

   A review by Snowdon SK, Stewart-Brown SL.

9. Implications of socio-cultural contexts for the ethics of clinical trials.

   A review by Ashcroft RE, Chadwick DW, Clark SRL, Edwards RHT, Frith L, Hutton JL.

10. A critical review of the role of neonatal hearing screening in the detection of congenital hearing impairment.

    By Davis A, Bamford J, Wilson I, Ramkalawan T, Forshaw M, Wright S.

11. Newborn screening for inborn errors of metabolism: a systematic review.

    By Seymour CA, Thomason MJ, Chalmers RA, Addison GM, Bain MD, Cockburn F, et al.

12. Routine preoperative testing: a systematic review of the evidence.

    By Munro J, Booth A, Nicholl J.

13. Systematic review of the effectiveness of laxatives in the elderly.

    By Petticrew M, Watt I, Sheldon T.

14. When and how to assess fast-changing technologies: a comparative study of medical applications of four generic technologies.

    A review by Mowatt G, Bower DJ, Brebner JA, Cairns JA, Grant AM, McKee L.

1. Antenatal screening for Down’s syndrome.

   A review by Wald NJ, Kennard A, Hackshaw A, McGuire A.

2. Screening for ovarian cancer: a systematic review.

   By Bell R, Petticrew M, Luengo S, Sheldon TA.

3. Consensus development methods, and their use in clinical guideline development.

   A review by Murphy MK, Black NA, Lamping DL, McKee CM, Sanderson CFB, Askham J, et al.

4. A cost–utility analysis of interferon beta for multiple sclerosis.

   By Parkin D, McNamee P, Jacoby A, Miller P, Thomas S, Bates D.

5. Effectiveness and efficiency of methods of dialysis therapy for end-stage renal disease: systematic reviews.

   By MacLeod A, Grant A, Donaldson C, Khan I, Campbell M, Daly C, et al.

6. Effectiveness of hip prostheses in primary total hip replacement: a critical review of evidence and an economic model.

   By Faulkner A, Kennedy LG, Baxter K, Donovan J, Wilkinson M, Bevan G.

7. Antimicrobial prophylaxis in colorectal surgery: a systematic review of randomised controlled trials.

   By Song F, Glenny AM.

8. Bone marrow and peripheral blood stem cell transplantation for malignancy.

   A review by Johnson PWM, Simnett SJ, Sweetenham JW, Morgan GJ, Stewart LA.

9. Screening for speech and language delay: a systematic review of the literature.

   By Law J, Boyle J, Harris F, Harkness A, Nye C.

10. Resource allocation for chronic stable angina: a systematic review of effectiveness, costs and cost-effectiveness of alternative interventions.

    By Sculpher MJ, Petticrew M, Kelland JL, Elliott RA, Holdright DR, Buxton MJ.

11. Detection, adherence and control of hypertension for the prevention of stroke: a systematic review.

    By Ebrahim S.

12. Postoperative analgesia and vomiting, with special reference to day-case surgery: a systematic review.

    By McQuay HJ, Moore RA.

13. Choosing between randomised and nonrandomised studies: a systematic review.

    By Britton A, McKee M, Black N, McPherson K, Sanderson C, Bain C.

14. Evaluating patient-based outcome measures for use in clinical trials.

    A review by Fitzpatrick R, Davey C, Buxton MJ, Jones DR.

15. Ethical issues in the design and conduct of randomised controlled trials.

    A review by Edwards SJL, Lilford RJ, Braunholtz DA, Jackson JC, Hewison J, Thornton J.

16. Qualitative research methods in health technology assessment: a review of the literature.

    By Murphy E, Dingwall R, Greatbatch D, Parker S, Watson P.

17. The costs and benefits of paramedic skills in pre-hospital trauma care.

    By Nicholl J, Hughes S, Dixon S, Turner J, Yates D.

18. Systematic review of endoscopic ultrasound in gastro-oesophageal cancer.

    By Harris KM, Kelly S, Berry E, Hutton J, Roderick P, Cullingworth J, et al.

19. Systematic reviews of trials and other studies.

    By Sutton AJ, Abrams KR, Jones DR, Sheldon TA, Song F.

20. Primary total hip replacement surgery: a systematic review of outcomes and modelling of cost-effectiveness associated with different prostheses.

    A review by Fitzpatrick R, Shortall E, Sculpher M, Murray D, Morris R, Lodge M, et al.

1. Informed decision making: an annotated bibliography and systematic review.

   By Bekker H, Thornton JG, Airey CM, Connelly JB, Hewison J, Robinson MB, et al.

2. Handling uncertainty when performing economic evaluation of healthcare interventions.

   A review by Briggs AH, Gray AM.

3. The role of expectancies in the placebo effect and their use in the delivery of health care: a systematic review.

   By Crow R, Gage H, Hampson S, Hart J, Kimber A, Thomas H.

4. A randomised controlled trial of different approaches to universal antenatal HIV testing: uptake and acceptability. Annex: Antenatal HIV testing – assessment of a routine voluntary approach.

   By Simpson WM, Johnstone FD, Boyd FM, Goldberg DJ, Hart GJ, Gormley SM, et al.

5. Methods for evaluating area-wide and organisation-based interventions in health and health care: a systematic review.

   By Ukoumunne OC, Gulliford MC, Chinn S, Sterne JAC, Burney PGJ.

6. Assessing the costs of healthcare technologies in clinical trials.

   A review by Johnston K, Buxton MJ, Jones DR, Fitzpatrick R.

7. Cooperatives and their primary care emergency centres: organisation and impact.

   By Hallam L, Henthorne K.

8. Screening for cystic fibrosis.

   A review by Murray J, Cuckle H, Taylor G, Littlewood J, Hewison J.

9. A review of the use of health status measures in economic evaluation.

   By Brazier J, Deverill M, Green C, Harper R, Booth A.

10. Methods for the analysis of quality-of-life and survival data in health technology assessment.

    A review by Billingham LJ, Abrams KR, Jones DR.

11. Antenatal and neonatal haemoglobinopathy screening in the UK: review and economic analysis.

    By Zeuner D, Ades AE, Karnon J, Brown J, Dezateux C, Anionwu EN.

12. Assessing the quality of reports of randomised trials: implications for the conduct of meta-analyses.

    A review by Moher D, Cook DJ, Jadad AR, Tugwell P, Moher M, Jones A, et al.

13. ‘Early warning systems’ for identifying new healthcare technologies.

    By Robert G, Stevens A, Gabbay J.

14. A systematic review of the role of human papillomavirus testing within a cervical screening programme.

    By Cuzick J, Sasieni P, Davies P, Adams J, Normand C, Frater A, et al.

15. Near patient testing in diabetes clinics: appraising the costs and outcomes.

    By Grieve R, Beech R, Vincent J, Mazurkiewicz J.

16. Positron emission tomography: establishing priorities for health technology assessment.

    A review by Robert G, Milne R.

17. The debridement of chronic wounds: a systematic review.

    By Bradley M, Cullum N, Sheldon T.

18. Systematic reviews of wound care management: (2) Dressings and topical agents used in the healing of chronic wounds.

    By Bradley M, Cullum N, Nelson EA, Petticrew M, Sheldon T, Torgerson D.

19. A systematic literature review of spiral and electron beam computed tomography: with particular reference to clinical applications in hepatic lesions, pulmonary embolus and coronary artery disease.

    By Berry E, Kelly S, Hutton J, Harris KM, Roderick P, Boyce JC, et al.

20. What role for statins? A review and economic model.

    By Ebrahim S, Davey Smith G, McCabe C, Payne N, Pickin M, Sheldon TA, et al.

21. Factors that limit the quality, number and progress of randomised controlled trials.

    A review by Prescott RJ, Counsell CE, Gillespie WJ, Grant AM, Russell IT, Kiauka S, et al.

22. Antimicrobial prophylaxis in total hip replacement: a systematic review.

    By Glenny AM, Song F.

23. Health promoting schools and health promotion in schools: two systematic reviews.

    By Lister-Sharp D, Chapman S, Stewart-Brown S, Sowden A.

24. Economic evaluation of a primary care-based education programme for patients with osteoarthritis of the knee.

    A review by Lord J, Victor C, Littlejohns P, Ross FM, Axford JS.

1. The estimation of marginal time preference in a UK-wide sample (TEMPUS) project.

   A review by Cairns JA, van der Pol MM.

2. Geriatric rehabilitation following fractures in older people: a systematic review.

   By Cameron I, Crotty M, Currie C, Finnegan T, Gillespie L, Gillespie W, et al.

3. Screening for sickle cell disease and thalassaemia: a systematic review with supplementary research.

   By Davies SC, Cronin E, Gill M, Greengross P, Hickman M, Normand C.

4. Community provision of hearing aids and related audiology services.

   A review by Reeves DJ, Alborz A, Hickson FS, Bamford JM.

5. False-negative results in screening programmes: systematic review of impact and implications.

   By Petticrew MP, Sowden AJ, Lister-Sharp D, Wright K.

6. Costs and benefits of community postnatal support workers: a randomised controlled trial.

   By Morrell CJ, Spiby H, Stewart P, Walters S, Morgan A.

7. Implantable contraceptives (subdermal implants and hormonally impregnated intrauterine systems) versus other forms of reversible contraceptives: two systematic reviews to assess relative effectiveness, acceptability, tolerability and cost-effectiveness.

   By French RS, Cowan FM, Mansour DJA, Morris S, Procter T, Hughes D, et al.

8. An introduction to statistical methods for health technology assessment.

   A review by White SJ, Ashby D, Brown PJ.

9. Disease-modifying drugs for multiple sclerosis: a rapid and systematic review.

   By Clegg A, Bryant J, Milne R.

10. Publication and related biases.

    A review by Song F, Eastwood AJ, Gilbody S, Duley L, Sutton AJ.

11. Cost and outcome implications of the organisation of vascular services.

    By Michaels J, Brazier J, Palfreyman S, Shackley P, Slack R.

12. Monitoring blood glucose control in diabetes mellitus: a systematic review.

    By Coster S, Gulliford MC, Seed PT, Powrie JK, Swaminathan R.

13. The effectiveness of domiciliary health visiting: a systematic review of international studies and a selective review of the British literature.

    By Elkan R, Kendrick D, Hewitt M, Robinson JJA, Tolley K, Blair M, et al.

14. The determinants of screening uptake and interventions for increasing uptake: a systematic review.

    By Jepson R, Clegg A, Forbes C, Lewis R, Sowden A, Kleijnen J.

15. The effectiveness and cost-effectiveness of prophylactic removal of wisdom teeth.

    A rapid review by Song F, O’Meara S, Wilson P, Golder S, Kleijnen J.

16. Ultrasound screening in pregnancy: a systematic review of the clinical effectiveness, cost-effectiveness and women’s views.

    By Bricker L, Garcia J, Henderson J, Mugford M, Neilson J, Roberts T, et al.

17. A rapid and systematic review of the effectiveness and cost-effectiveness of the taxanes used in the treatment of advanced breast and ovarian cancer.

    By Lister-Sharp D, McDonagh MS, Khan KS, Kleijnen J.

18. Liquid-based cytology in cervical screening: a rapid and systematic review.

    By Payne N, Chilcott J, McGoogan E.

19. Randomised controlled trial of non-directive counselling, cognitive–behaviour therapy and usual general practitioner care in the management of depression as well as mixed anxiety and depression in primary care.

    By King M, Sibbald B, Ward E, Bower P, Lloyd M, Gabbay M, et al.

20. Routine referral for radiography of patients presenting with low back pain: is patients’ outcome influenced by GPs’ referral for plain radiography?

    By Kerry S, Hilton S, Patel S, Dundas D, Rink E, Lord J.

21. Systematic reviews of wound care management: (3) antimicrobial agents for chronic wounds; (4) diabetic foot ulceration.

    By O’Meara S, Cullum N, Majid M, Sheldon T.

22. Using routine data to complement and enhance the results of randomised controlled trials.

    By Lewsey JD, Leyland AH, Murray GD, Boddy FA.

23. Coronary artery stents in the treatment of ischaemic heart disease: a rapid and systematic review.

    By Meads C, Cummins C, Jolly K, Stevens A, Burls A, Hyde C.

24. Outcome measures for adult critical care: a systematic review.

    By Hayes JA, Black NA, Jenkinson C, Young JD, Rowan KM, Daly K, et al.

25. A systematic review to evaluate the effectiveness of interventions to promote the initiation of breastfeeding.

    By Fairbank L, O’Meara S, Renfrew MJ, Woolridge M, Sowden AJ, Lister-Sharp D.

26. Implantable cardioverter defibrillators: arrhythmias. A rapid and systematic review.

    By Parkes J, Bryant J, Milne R.

27. Treatments for fatigue in multiple sclerosis: a rapid and systematic review.

    By Brañas P, Jordan R, Fry-Smith A, Burls A, Hyde C.

28. Early asthma prophylaxis, natural history, skeletal development and economy (EASE): a pilot randomised controlled trial.

    By Baxter-Jones ADG, Helms PJ, Russell G, Grant A, Ross S, Cairns JA, et al.

29. Screening for hypercholesterolaemia versus case finding for familial hypercholesterolaemia: a systematic review and cost-effectiveness analysis.

    By Marks D, Wonderling D, Thorogood M, Lambert H, Humphries SE, Neil HAW.

30. A rapid and systematic review of the clinical effectiveness and cost-effectiveness of glycoprotein IIb/IIIa antagonists in the medical management of unstable angina.

    By McDonagh MS, Bachmann LM, Golder S, Kleijnen J, ter Riet G.

31. A randomised controlled trial of prehospital intravenous fluid replacement therapy in serious trauma.

    By Turner J, Nicholl J, Webber L, Cox H, Dixon S, Yates D.

32. Intrathecal pumps for giving opioids in chronic pain: a systematic review.

    By Williams JE, Louw G, Towlerton G.

33. Combination therapy (interferon alfa and ribavirin) in the treatment of chronic hepatitis C: a rapid and systematic review.

    By Shepherd J, Waugh N, Hewitson P.

34. A systematic review of comparisons of effect sizes derived from randomised and non-randomised studies.

    By MacLehose RR, Reeves BC, Harvey IM, Sheldon TA, Russell IT, Black AMS.

35. Intravascular ultrasound-guided interventions in coronary artery disease: a systematic literature review, with decision-analytic modelling, of outcomes and cost-effectiveness.

    By Berry E, Kelly S, Hutton J, Lindsay HSJ, Blaxill JM, Evans JA, et al.

36. A randomised controlled trial to evaluate the effectiveness and cost-effectiveness of counselling patients with chronic depression.

    By Simpson S, Corney R, Fitzgerald P, Beecham J.

37. Systematic review of treatments for atopic eczema.

    By Hoare C, Li Wan Po A, Williams H.

38. Bayesian methods in health technology assessment: a review.

    By Spiegelhalter DJ, Myles JP, Jones DR, Abrams KR.

39. The management of dyspepsia: a systematic review.

    By Delaney B, Moayyedi P, Deeks J, Innes M, Soo S, Barton P, et al.

40. A systematic review of treatments for severe psoriasis.

    By Griffiths CEM, Clark CM, Chalmers RJG, Li Wan Po A, Williams HC.

1. Clinical and cost-effectiveness of donepezil, rivastigmine and galantamine for Alzheimer’s disease: a rapid and systematic review.

   By Clegg A, Bryant J, Nicholson T, McIntyre L, De Broe S, Gerard K, et al.

2. The clinical effectiveness and cost-effectiveness of riluzole for motor neurone disease: a rapid and systematic review.

   By Stewart A, Sandercock J, Bryan S, Hyde C, Barton PM, Fry-Smith A, et al.

3. Equity and the economic evaluation of healthcare.

   By Sassi F, Archard L, Le Grand J.

4. Quality-of-life measures in chronic diseases of childhood.

   By Eiser C, Morse R.

5. Eliciting public preferences for healthcare: a systematic review of techniques.

   By Ryan M, Scott DA, Reeves C, Bate A, van Teijlingen ER, Russell EM, et al.

6. General health status measures for people with cognitive impairment: learning disability and acquired brain injury.

   By Riemsma RP, Forbes CA, Glanville JM, Eastwood AJ, Kleijnen J.

7. An assessment of screening strategies for fragile X syndrome in the UK.

   By Pembrey ME, Barnicoat AJ, Carmichael B, Bobrow M, Turner G.

8. Issues in methodological research: perspectives from researchers and commissioners.

   By Lilford RJ, Richardson A, Stevens A, Fitzpatrick R, Edwards S, Rock F, et al.

9. Systematic reviews of wound care management: (5) beds; (6) compression; (7) laser therapy, therapeutic ultrasound, electrotherapy and electromagnetic therapy.

   By Cullum N, Nelson EA, Flemming K, Sheldon T.

10. Effects of educational and psychosocial interventions for adolescents with diabetes mellitus: a systematic review.

    By Hampson SE, Skinner TC, Hart J, Storey L, Gage H, Foxcroft D, et al.

11. Effectiveness of autologous chondrocyte transplantation for hyaline cartilage defects in knees: a rapid and systematic review.

    By Jobanputra P, Parry D, Fry-Smith A, Burls A.

12. Statistical assessment of the learning curves of health technologies.

    By Ramsay CR, Grant AM, Wallace SA, Garthwaite PH, Monk AF, Russell IT.

13. The effectiveness and cost-effectiveness of temozolomide for the treatment of recurrent malignant glioma: a rapid and systematic review.

    By Dinnes J, Cave C, Huang S, Major K, Milne R.

14. A rapid and systematic review of the clinical effectiveness and cost-effectiveness of debriding agents in treating surgical wounds healing by secondary intention.

    By Lewis R, Whiting P, ter Riet G, O’Meara S, Glanville J.

15. Home treatment for mental health problems: a systematic review.

    By Burns T, Knapp M, Catty J, Healey A, Henderson J, Watt H, et al.

16. How to develop cost-conscious guidelines.

    By Eccles M, Mason J.

17. The role of specialist nurses in multiple sclerosis: a rapid and systematic review.

    By De Broe S, Christopher F, Waugh N.

18. A rapid and systematic review of the clinical effectiveness and cost-effectiveness of orlistat in the management of obesity.

    By O’Meara S, Riemsma R, Shirran L, Mather L, ter Riet G.

19. The clinical effectiveness and cost-effectiveness of pioglitazone for type 2 diabetes mellitus: a rapid and systematic review.

    By Chilcott J, Wight J, Lloyd Jones M, Tappenden P.

20. Extended scope of nursing practice: a multicentre randomised controlled trial of appropriately trained nurses and preregistration house officers in preoperative assessment in elective general surgery.

    By Kinley H, Czoski-Murray C, George S, McCabe C, Primrose J, Reilly C, et al.

21. Systematic reviews of the effectiveness of day care for people with severe mental disorders: (1) Acute day hospital versus admission; (2) Vocational rehabilitation; (3) Day hospital versus outpatient care.

    By Marshall M, Crowther R, Almaraz- Serrano A, Creed F, Sledge W, Kluiter H, et al.

22. The measurement and monitoring of surgical adverse events.

    By Bruce J, Russell EM, Mollison J, Krukowski ZH.

23. Action research: a systematic review and guidance for assessment.

    By Waterman H, Tillen D, Dickson R, de Koning K.

24. A rapid and systematic review of the clinical effectiveness and cost-effectiveness of gemcitabine for the treatment of pancreatic cancer.

    By Ward S, Morris E, Bansback N, Calvert N, Crellin A, Forman D, et al.

25. A rapid and systematic review of the evidence for the clinical effectiveness and cost-effectiveness of irinotecan, oxaliplatin and raltitrexed for the treatment of advanced colorectal cancer.

    By Lloyd Jones M, Hummel S, Bansback N, Orr B, Seymour M.

26. Comparison of the effectiveness of inhaler devices in asthma and chronic obstructive airways disease: a systematic review of the literature.

    By Brocklebank D, Ram F, Wright J, Barry P, Cates C, Davies L, et al.

27. The cost-effectiveness of magnetic resonance imaging for investigation of the knee joint.

    By Bryan S, Weatherburn G, Bungay H, Hatrick C, Salas C, Parry D, et al.

28. A rapid and systematic review of the clinical effectiveness and cost-effectiveness of topotecan for ovarian cancer.

    By Forbes C, Shirran L, Bagnall A-M, Duffy S, ter Riet G.

29. Superseded by a report published in a later volume.

30. The role of radiography in primary care patients with low back pain of at least 6 weeks duration: a randomised (unblinded) controlled trial.

    By Kendrick D, Fielding K, Bentley E, Miller P, Kerslake R, Pringle M.

31. Design and use of questionnaires: a review of best practice applicable to surveys of health service staff and patients.

    By McColl E, Jacoby A, Thomas L, Soutter J, Bamford C, Steen N, et al.

32. A rapid and systematic review of the clinical effectiveness and cost-effectiveness of paclitaxel, docetaxel, gemcitabine and vinorelbine in non-small-cell lung cancer.

    By Clegg A, Scott DA, Sidhu M, Hewitson P, Waugh N.

33. Subgroup analyses in randomised controlled trials: quantifying the risks of false-positives and false-negatives.

    By Brookes ST, Whitley E, Peters TJ, Mulheran PA, Egger M, Davey Smith G.

34. Depot antipsychotic medication in the treatment of patients with schizophrenia: (1) Meta-review; (2) Patient and nurse attitudes.

    By David AS, Adams C.

35. A systematic review of controlled trials of the effectiveness and cost-effectiveness of brief psychological treatments for depression.

    By Churchill R, Hunot V, Corney R, Knapp M, McGuire H, Tylee A, et al.

36. Cost analysis of child health surveillance.

    By Sanderson D, Wright D, Acton C, Duree D.

1. A study of the methods used to select review criteria for clinical audit.

   By Hearnshaw H, Harker R, Cheater F, Baker R, Grimshaw G.

2. Fludarabine as second-line therapy for B cell chronic lymphocytic leukaemia: a technology assessment.

   By Hyde C, Wake B, Bryan S, Barton P, Fry-Smith A, Davenport C, et al.

3. Rituximab as third-line treatment for refractory or recurrent Stage III or IV follicular non-Hodgkin’s lymphoma: a systematic review and economic evaluation.

   By Wake B, Hyde C, Bryan S, Barton P, Song F, Fry-Smith A, et al.

4. A systematic review of discharge arrangements for older people.

   By Parker SG, Peet SM, McPherson A, Cannaby AM, Baker R, Wilson A, et al.

5. The clinical effectiveness and cost-effectiveness of inhaler devices used in the routine management of chronic asthma in older children: a systematic review and economic evaluation.

   By Peters J, Stevenson M, Beverley C, Lim J, Smith S.

6. The clinical effectiveness and cost-effectiveness of sibutramine in the management of obesity: a technology assessment.

   By O’Meara S, Riemsma R, Shirran L, Mather L, ter Riet G.

7. The cost-effectiveness of magnetic resonance angiography for carotid artery stenosis and peripheral vascular disease: a systematic review.

   By Berry E, Kelly S, Westwood ME, Davies LM, Gough MJ, Bamford JM, et al.

8. Promoting physical activity in South Asian Muslim women through ‘exercise on prescription’.

   By Carroll B, Ali N, Azam N.

9. Zanamivir for the treatment of influenza in adults: a systematic review and economic evaluation.

   By Burls A, Clark W, Stewart T, Preston C, Bryan S, Jefferson T, et al.

10. A review of the natural history and epidemiology of multiple sclerosis: implications for resource allocation and health economic models.

    By Richards RG, Sampson FC, Beard SM, Tappenden P.

11. Screening for gestational diabetes: a systematic review and economic evaluation.

    By Scott DA, Loveman E, McIntyre L, Waugh N.

12. The clinical effectiveness and cost-effectiveness of surgery for people with morbid obesity: a systematic review and economic evaluation.

    By Clegg AJ, Colquitt J, Sidhu MK, Royle P, Loveman E, Walker A.

13. The clinical effectiveness of trastuzumab for breast cancer: a systematic review.

    By Lewis R, Bagnall A-M, Forbes C, Shirran E, Duffy S, Kleijnen J, et al.

14. The clinical effectiveness and cost-effectiveness of vinorelbine for breast cancer: a systematic review and economic evaluation.

    By Lewis R, Bagnall A-M, King S, Woolacott N, Forbes C, Shirran L, et al.

15. A systematic review of the effectiveness and cost-effectiveness of metal-on-metal hip resurfacing arthroplasty for treatment of hip disease.

    By Vale L, Wyness L, McCormack K, McKenzie L, Brazzelli M, Stearns SC.

16. The clinical effectiveness and cost-effectiveness of bupropion and nicotine replacement therapy for smoking cessation: a systematic review and economic evaluation.

    By Woolacott NF, Jones L, Forbes CA, Mather LC, Sowden AJ, Song FJ, et al.

17. A systematic review of effectiveness and economic evaluation of new drug treatments for juvenile idiopathic arthritis: etanercept.

    By Cummins C, Connock M, Fry-Smith A, Burls A.

18. Clinical effectiveness and cost-effectiveness of growth hormone in children: a systematic review and economic evaluation.

    By Bryant J, Cave C, Mihaylova B, Chase D, McIntyre L, Gerard K, et al.

19. Clinical effectiveness and cost-effectiveness of growth hormone in adults in relation to impact on quality of life: a systematic review and economic evaluation.

    By Bryant J, Loveman E, Chase D, Mihaylova B, Cave C, Gerard K, et al.

20. Clinical medication review by a pharmacist of patients on repeat prescriptions in general practice: a randomised controlled trial.

    By Zermansky AG, Petty DR, Raynor DK, Lowe CJ, Freementle N, Vail A.

21. The effectiveness of infliximab and etanercept for the treatment of rheumatoid arthritis: a systematic review and economic evaluation.

    By Jobanputra P, Barton P, Bryan S, Burls A.

22. A systematic review and economic evaluation of computerised cognitive behaviour therapy for depression and anxiety.

    By Kaltenthaler E, Shackley P, Stevens K, Beverley C, Parry G, Chilcott J.

23. A systematic review and economic evaluation of pegylated liposomal doxorubicin hydrochloride for ovarian cancer.

    By Forbes C, Wilby J, Richardson G, Sculpher M, Mather L, Riemsma R.

24. A systematic review of the effectiveness of interventions based on a stages-of-change approach to promote individual behaviour change.

    By Riemsma RP, Pattenden J, Bridle C, Sowden AJ, Mather L, Watt IS, et al.

25. A systematic review update of the clinical effectiveness and cost-effectiveness of glycoprotein IIb/IIIa antagonists.

    By Robinson M, Ginnelly L, Sculpher M, Jones L, Riemsma R, Palmer S, et al.

26. A systematic review of the effectiveness, cost-effectiveness and barriers to implementation of thrombolytic and neuroprotective therapy for acute ischaemic stroke in the NHS.

    By Sandercock P, Berge E, Dennis M, Forbes J, Hand P, Kwan J, et al.

27. A randomised controlled crossover trial of nurse practitioner versus doctor-led outpatient care in a bronchiectasis clinic.

    By Caine N, Sharples LD, Hollingworth W, French J, Keogan M, Exley A, et al.

28. Clinical effectiveness and cost – consequences of selective serotonin reuptake inhibitors in the treatment of sex offenders.

    By Adi Y, Ashcroft D, Browne K, Beech A, Fry-Smith A, Hyde C.

29. Treatment of established osteoporosis: a systematic review and cost–utility analysis.

    By Kanis JA, Brazier JE, Stevenson M, Calvert NW, Lloyd Jones M.

30. Which anaesthetic agents are cost-effective in day surgery? Literature review, national survey of practice and randomised controlled trial.

    By Elliott RA Payne K, Moore JK, Davies LM, Harper NJN, St Leger AS, et al.

31. Screening for hepatitis C among injecting drug users and in genitourinary medicine clinics: systematic reviews of effectiveness, modelling study and national survey of current practice.

    By Stein K, Dalziel K, Walker A, McIntyre L, Jenkins B, Horne J, et al.

32. The measurement of satisfaction with healthcare: implications for practice from a systematic review of the literature.

    By Crow R, Gage H, Hampson S, Hart J, Kimber A, Storey L, et al.

33. The effectiveness and cost-effectiveness of imatinib in chronic myeloid leukaemia: a systematic review.

    By Garside R, Round A, Dalziel K, Stein K, Royle R.

34. A comparative study of hypertonic saline, daily and alternate-day rhDNase in children with cystic fibrosis.

    By Suri R, Wallis C, Bush A, Thompson S, Normand C, Flather M, et al.

35. A systematic review of the costs and effectiveness of different models of paediatric home care.

    By Parker G, Bhakta P, Lovett CA, Paisley S, Olsen R, Turner D, et al.

1. How important are comprehensive literature searches and the assessment of trial quality in systematic reviews? Empirical study.

   By Egger M, Jüni P, Bartlett C, Holenstein F, Sterne J.

2. Systematic review of the effectiveness and cost-effectiveness, and economic evaluation, of home versus hospital or satellite unit haemodialysis for people with end-stage renal failure.

   By Mowatt G, Vale L, Perez J, Wyness L, Fraser C, MacLeod A, et al.

3. Systematic review and economic evaluation of the effectiveness of infliximab for the treatment of Crohn’s disease.

   By Clark W, Raftery J, Barton P, Song F, Fry-Smith A, Burls A.

4. A review of the clinical effectiveness and cost-effectiveness of routine anti-D prophylaxis for pregnant women who are rhesus negative.

   By Chilcott J, Lloyd Jones M, Wight J, Forman K, Wray J, Beverley C, et al.

5. Systematic review and evaluation of the use of tumour markers in paediatric oncology: Ewing’s sarcoma and neuroblastoma.

   By Riley RD, Burchill SA, Abrams KR, Heney D, Lambert PC, Jones DR, et al.

6. The cost-effectiveness of screening for Helicobacter pylori to reduce mortality and morbidity from gastric cancer and peptic ulcer disease: a discrete-event simulation model.

   By Roderick P, Davies R, Raftery J, Crabbe D, Pearce R, Bhandari P, et al.

7. The clinical effectiveness and cost-effectiveness of routine dental checks: a systematic review and economic evaluation.

   By Davenport C, Elley K, Salas C, Taylor-Weetman CL, Fry-Smith A, Bryan S, et al.

8. A multicentre randomised controlled trial assessing the costs and benefits of using structured information and analysis of women’s preferences in the management of menorrhagia.

   By Kennedy ADM, Sculpher MJ, Coulter A, Dwyer N, Rees M, Horsley S, et al.

9. Clinical effectiveness and cost–utility of photodynamic therapy for wet age-related macular degeneration: a systematic review and economic evaluation.

   By Meads C, Salas C, Roberts T, Moore D, Fry-Smith A, Hyde C.

10. Evaluation of molecular tests for prenatal diagnosis of chromosome abnormalities.

    By Grimshaw GM, Szczepura A, Hultén M, MacDonald F, Nevin NC, Sutton F, et al.

11. First and second trimester antenatal screening for Down’s syndrome: the results of the Serum, Urine and Ultrasound Screening Study (SURUSS).

    By Wald NJ, Rodeck C, Hackshaw AK, Walters J, Chitty L, Mackinson AM.

12. The effectiveness and cost-effectiveness of ultrasound locating devices for central venous access: a systematic review and economic evaluation.

    By Calvert N, Hind D, McWilliams RG, Thomas SM, Beverley C, Davidson A.

13. A systematic review of atypical antipsychotics in schizophrenia.

    By Bagnall A-M, Jones L, Lewis R, Ginnelly L, Glanville J, Torgerson D, et al.

14. Prostate Testing for Cancer and Treatment (ProtecT) feasibility study.

    By Donovan J, Hamdy F, Neal D, Peters T, Oliver S, Brindle L, et al.

15. Early thrombolysis for the treatment of acute myocardial infarction: a systematic review and economic evaluation.

    By Boland A, Dundar Y, Bagust A, Haycox A, Hill R, Mujica Mota R, et al.

16. Screening for fragile X syndrome: a literature review and modelling.

    By Song FJ, Barton P, Sleightholme V, Yao GL, Fry-Smith A.

17. Systematic review of endoscopic sinus surgery for nasal polyps.

    By Dalziel K, Stein K, Round A, Garside R, Royle P.

18. Towards efficient guidelines: how to monitor guideline use in primary care.

    By Hutchinson A, McIntosh A, Cox S, Gilbert C.

19. Effectiveness and cost-effectiveness of acute hospital-based spinal cord injuries services: systematic review.

    By Bagnall A-M, Jones L, Richardson G, Duffy S, Riemsma R.

20. Prioritisation of health technology assessment. The PATHS model: methods and case studies.

    By Townsend J, Buxton M, Harper G.

21. Systematic review of the clinical effectiveness and cost-effectiveness of tension-free vaginal tape for treatment of urinary stress incontinence.

    By Cody J, Wyness L, Wallace S, Glazener C, Kilonzo M, Stearns S, et al.

22. The clinical and cost-effectiveness of patient education models for diabetes: a systematic review and economic evaluation.

    By Loveman E, Cave C, Green C, Royle P, Dunn N, Waugh N.

23. The role of modelling in prioritising and planning clinical trials.

    By Chilcott J, Brennan A, Booth A, Karnon J, Tappenden P.

24. Cost–benefit evaluation of routine influenza immunisation in people 65–74 years of age.

    By Allsup S, Gosney M, Haycox A, Regan M.

25. The clinical and cost-effectiveness of pulsatile machine perfusion versus cold storage of kidneys for transplantation retrieved from heart-beating and non-heart-beating donors.

    By Wight J, Chilcott J, Holmes M, Brewer N.

26. Can randomised trials rely on existing electronic data? A feasibility study to explore the value of routine data in health technology assessment.

    By Williams JG, Cheung WY, Cohen DR, Hutchings HA, Longo MF, Russell IT.

27. Evaluating non-randomised intervention studies.

    By Deeks JJ, Dinnes J, D’Amico R, Sowden AJ, Sakarovitch C, Song F, et al.

28. A randomised controlled trial to assess the impact of a package comprising a patient-orientated, evidence-based self- help guidebook and patient-centred consultations on disease management and satisfaction in inflammatory bowel disease.

    By Kennedy A, Nelson E, Reeves D, Richardson G, Roberts C, Robinson A, et al.

29. The effectiveness of diagnostic tests for the assessment of shoulder pain due to soft tissue disorders: a systematic review.

    By Dinnes J, Loveman E, McIntyre L, Waugh N.

30. The value of digital imaging in diabetic retinopathy.

    By Sharp PF, Olson J, Strachan F, Hipwell J, Ludbrook A, O’Donnell M, et al.

31. Lowering blood pressure to prevent myocardial infarction and stroke: a new preventive strategy.

    By Law M, Wald N, Morris J.

32. Clinical and cost-effectiveness of capecitabine and tegafur with uracil for the treatment of metastatic colorectal cancer: systematic review and economic evaluation.

    By Ward S, Kaltenthaler E, Cowan J, Brewer N.

33. Clinical and cost-effectiveness of new and emerging technologies for early localised prostate cancer: a systematic review.

    By Hummel S, Paisley S, Morgan A, Currie E, Brewer N.

34. Literature searching for clinical and cost-effectiveness studies used in health technology assessment reports carried out for the National Institute for Clinical Excellence appraisal system.

    By Royle P, Waugh N.

35. Systematic review and economic decision modelling for the prevention and treatment of influenza A and B.

    By Turner D, Wailoo A, Nicholson K, Cooper N, Sutton A, Abrams K.

36. A randomised controlled trial to evaluate the clinical and cost-effectiveness of Hickman line insertions in adult cancer patients by nurses.

    By Boland A, Haycox A, Bagust A, Fitzsimmons L.

37. Redesigning postnatal care: a randomised controlled trial of protocol-based midwifery-led care focused on individual women’s physical and psychological health needs.

    By MacArthur C, Winter HR, Bick DE, Lilford RJ, Lancashire RJ, Knowles H, et al.

38. Estimating implied rates of discount in healthcare decision-making.

    By West RR, McNabb R, Thompson AGH, Sheldon TA, Grimley Evans J.

39. Systematic review of isolation policies in the hospital management of methicillin-resistant Staphylococcus aureus: a review of the literature with epidemiological and economic modelling.

    By Cooper BS, Stone SP, Kibbler CC, Cookson BD, Roberts JA, Medley GF, et al.

40. Treatments for spasticity and pain in multiple sclerosis: a systematic review.

    By Beard S, Hunn A, Wight J.

41. The inclusion of reports of randomised trials published in languages other than English in systematic reviews.

    By Moher D, Pham B, Lawson ML, Klassen TP.

42. The impact of screening on future health-promoting behaviours and health beliefs: a systematic review.

    By Bankhead CR, Brett J, Bukach C, Webster P, Stewart-Brown S, Munafo M, et al.

1. What is the best imaging strategy for acute stroke?

   By Wardlaw JM, Keir SL, Seymour J, Lewis S, Sandercock PAG, Dennis MS, et al.

2. Systematic review and modelling of the investigation of acute and chronic chest pain presenting in primary care.

   By Mant J, McManus RJ, Oakes RAL, Delaney BC, Barton PM, Deeks JJ, et al.

3. The effectiveness and cost-effectiveness of microwave and thermal balloon endometrial ablation for heavy menstrual bleeding: a systematic review and economic modelling.

   By Garside R, Stein K, Wyatt K, Round A, Price A.

4. A systematic review of the role of bisphosphonates in metastatic disease.

   By Ross JR, Saunders Y, Edmonds PM, Patel S, Wonderling D, Normand C, et al.

5. Systematic review of the clinical effectiveness and cost-effectiveness of capecitabine (Xeloda^®) for locally advanced and/or metastatic breast cancer.

   By Jones L, Hawkins N, Westwood M, Wright K, Richardson G, Riemsma R.

6. Effectiveness and efficiency of guideline dissemination and implementation strategies.

   By Grimshaw JM, Thomas RE, MacLennan G, Fraser C, Ramsay CR, Vale L, et al.

7. Clinical effectiveness and costs of the Sugarbaker procedure for the treatment of pseudomyxoma peritonei.

   By Bryant J, Clegg AJ, Sidhu MK, Brodin H, Royle P, Davidson P.

8. Psychological treatment for insomnia in the regulation of long-term hypnotic drug use.

   By Morgan K, Dixon S, Mathers N, Thompson J, Tomeny M.

9. Improving the evaluation of therapeutic interventions in multiple sclerosis: development of a patient-based measure of outcome.

   By Hobart JC, Riazi A, Lamping DL, Fitzpatrick R, Thompson AJ.

10. A systematic review and economic evaluation of magnetic resonance cholangiopancreatography compared with diagnostic endoscopic retrograde cholangiopancreatography.

    By Kaltenthaler E, Bravo Vergel Y, Chilcott J, Thomas S, Blakeborough T, Walters SJ, et al.

11. The use of modelling to evaluate new drugs for patients with a chronic condition: the case of antibodies against tumour necrosis factor in rheumatoid arthritis.

    By Barton P, Jobanputra P, Wilson J, Bryan S, Burls A.

12. Clinical effectiveness and cost-effectiveness of neonatal screening for inborn errors of metabolism using tandem mass spectrometry: a systematic review.

    By Pandor A, Eastham J, Beverley C, Chilcott J, Paisley S.

13. Clinical effectiveness and cost-effectiveness of pioglitazone and rosiglitazone in the treatment of type 2 diabetes: a systematic review and economic evaluation.

    By Czoski-Murray C, Warren E, Chilcott J, Beverley C, Psyllaki MA, Cowan J.

14. Routine examination of the newborn: the EMREN study. Evaluation of an extension of the midwife role including a randomised controlled trial of appropriately trained midwives and paediatric senior house officers.

    By Townsend J, Wolke D, Hayes J, Davé S, Rogers C, Bloomfield L, et al.

15. Involving consumers in research and development agenda setting for the NHS: developing an evidence-based approach.

    By Oliver S, Clarke-Jones L, Rees R, Milne R, Buchanan P, Gabbay J, et al.

16. A multi-centre randomised controlled trial of minimally invasive direct coronary bypass grafting versus percutaneous transluminal coronary angioplasty with stenting for proximal stenosis of the left anterior descending coronary artery.

    By Reeves BC, Angelini GD, Bryan AJ, Taylor FC, Cripps T, Spyt TJ, et al.

17. Does early magnetic resonance imaging influence management or improve outcome in patients referred to secondary care with low back pain? A pragmatic randomised controlled trial.

    By Gilbert FJ, Grant AM, Gillan MGC, Vale L, Scott NW, Campbell MK, et al.

18. The clinical and cost-effectiveness of anakinra for the treatment of rheumatoid arthritis in adults: a systematic review and economic analysis.

    By Clark W, Jobanputra P, Barton P, Burls A.

19. A rapid and systematic review and economic evaluation of the clinical and cost-effectiveness of newer drugs for treatment of mania associated with bipolar affective disorder.

    By Bridle C, Palmer S, Bagnall A-M, Darba J, Duffy S, Sculpher M, et al.

20. Liquid-based cytology in cervical screening: an updated rapid and systematic review and economic analysis.

    By Karnon J, Peters J, Platt J, Chilcott J, McGoogan E, Brewer N.

21. Systematic review of the long-term effects and economic consequences of treatments for obesity and implications for health improvement.

    By Avenell A, Broom J, Brown TJ, Poobalan A, Aucott L, Stearns SC, et al.

22. Autoantibody testing in children with newly diagnosed type 1 diabetes mellitus.

    By Dretzke J, Cummins C, Sandercock J, Fry-Smith A, Barrett T, Burls A.

23. Clinical effectiveness and cost-effectiveness of prehospital intravenous fluids in trauma patients.

    By Dretzke J, Sandercock J, Bayliss S, Burls A.

24. Newer hypnotic drugs for the short-term management of insomnia: a systematic review and economic evaluation.

    By Dündar Y, Boland A, Strobl J, Dodd S, Haycox A, Bagust A, et al.

25. Development and validation of methods for assessing the quality of diagnostic accuracy studies.

    By Whiting P, Rutjes AWS, Dinnes J, Reitsma JB, Bossuyt PMM, Kleijnen J.

26. EVALUATE hysterectomy trial: a multicentre randomised trial comparing abdominal, vaginal and laparoscopic methods of hysterectomy.

    By Garry R, Fountain J, Brown J, Manca A, Mason S, Sculpher M, et al.

27. Methods for expected value of information analysis in complex health economic models: developments on the health economics of interferon-β and glatiramer acetate for multiple sclerosis.

    By Tappenden P, Chilcott JB, Eggington S, Oakley J, McCabe C.

28. Effectiveness and cost-effectiveness of imatinib for first-line treatment of chronic myeloid leukaemia in chronic phase: a systematic review and economic analysis.

    By Dalziel K, Round A, Stein K, Garside R, Price A.

29. VenUS I: a randomised controlled trial of two types of bandage for treating venous leg ulcers.

    By Iglesias C, Nelson EA, Cullum NA, Torgerson DJ, on behalf of the VenUS Team.

30. Systematic review of the effectiveness and cost-effectiveness, and economic evaluation, of myocardial perfusion scintigraphy for the diagnosis and management of angina and myocardial infarction.

    By Mowatt G, Vale L, Brazzelli M, Hernandez R, Murray A, Scott N, et al.

31. A pilot study on the use of decision theory and value of information analysis as part of the NHS Health Technology Assessment programme.

    By Claxton K, Ginnelly L, Sculpher M, Philips Z, Palmer S.

32. The Social Support and Family Health Study: a randomised controlled trial and economic evaluation of two alternative forms of postnatal support for mothers living in disadvantaged inner-city areas.

    By Wiggins M, Oakley A, Roberts I, Turner H, Rajan L, Austerberry H, et al.

33. Psychosocial aspects of genetic screening of pregnant women and newborns: a systematic review.

    By Green JM, Hewison J, Bekker HL, Bryant, Cuckle HS.

34. Evaluation of abnormal uterine bleeding: comparison of three outpatient procedures within cohorts defined by age and menopausal status.

    By Critchley HOD, Warner P, Lee AJ, Brechin S, Guise J, Graham B.

35. Coronary artery stents: a rapid systematic review and economic evaluation.

    By Hill R, Bagust A, Bakhai A, Dickson R, Dündar Y, Haycox A, et al.

36. Review of guidelines for good practice in decision-analytic modelling in health technology assessment.

    By Philips Z, Ginnelly L, Sculpher M, Claxton K, Golder S, Riemsma R, et al.

37. Rituximab (MabThera^®) for aggressive non-Hodgkin’s lymphoma: systematic review and economic evaluation.

    By Knight C, Hind D, Brewer N, Abbott V.

38. Clinical effectiveness and cost-effectiveness of clopidogrel and modified-release dipyridamole in the secondary prevention of occlusive vascular events: a systematic review and economic evaluation.

    By Jones L, Griffin S, Palmer S, Main C, Orton V, Sculpher M, et al.

39. Pegylated interferon α-2a and -2b in combination with ribavirin in the treatment of chronic hepatitis C: a systematic review and economic evaluation.

    By Shepherd J, Brodin H, Cave C, Waugh N, Price A, Gabbay J.

40. Clopidogrel used in combination with aspirin compared with aspirin alone in the treatment of non-ST-segment- elevation acute coronary syndromes: a systematic review and economic evaluation.

    By Main C, Palmer S, Griffin S, Jones L, Orton V, Sculpher M, et al.

41. Provision, uptake and cost of cardiac rehabilitation programmes: improving services to under-represented groups.

    By Beswick AD, Rees K, Griebsch I, Taylor FC, Burke M, West RR, et al.

42. Involving South Asian patients in clinical trials.

    By Hussain-Gambles M, Leese B, Atkin K, Brown J, Mason S, Tovey P.

43. Clinical and cost-effectiveness of continuous subcutaneous insulin infusion for diabetes.

    By Colquitt JL, Green C, Sidhu MK, Hartwell D, Waugh N.

44. Identification and assessment of ongoing trials in health technology assessment reviews.

    By Song FJ, Fry-Smith A, Davenport C, Bayliss S, Adi Y, Wilson JS, et al.

45. Systematic review and economic evaluation of a long-acting insulin analogue, insulin glargine

    By Warren E, Weatherley-Jones E, Chilcott J, Beverley C.

46. Supplementation of a home-based exercise programme with a class-based programme for people with osteoarthritis of the knees: a randomised controlled trial and health economic analysis.

    By McCarthy CJ, Mills PM, Pullen R, Richardson G, Hawkins N, Roberts CR, et al.

47. Clinical and cost-effectiveness of once-daily versus more frequent use of same potency topical corticosteroids for atopic eczema: a systematic review and economic evaluation.

    By Green C, Colquitt JL, Kirby J, Davidson P, Payne E.

48. Acupuncture of chronic headache disorders in primary care: randomised controlled trial and economic analysis.

    By Vickers AJ, Rees RW, Zollman CE, McCarney R, Smith CM, Ellis N, et al.

49. Generalisability in economic evaluation studies in healthcare: a review and case studies.

    By Sculpher MJ, Pang FS, Manca A, Drummond MF, Golder S, Urdahl H, et al.

50. Virtual outreach: a randomised controlled trial and economic evaluation of joint teleconferenced medical consultations.

    By Wallace P, Barber J, Clayton W, Currell R, Fleming K, Garner P, et al.

1. Randomised controlled multiple treatment comparison to provide a cost-effectiveness rationale for the selection of antimicrobial therapy in acne.

   By Ozolins M, Eady EA, Avery A, Cunliffe WJ, O’Neill C, Simpson NB, et al.

2. Do the findings of case series studies vary significantly according to methodological characteristics?

   By Dalziel K, Round A, Stein K, Garside R, Castelnuovo E, Payne L.

3. Improving the referral process for familial breast cancer genetic counselling: findings of three randomised controlled trials of two interventions.

   By Wilson BJ, Torrance N, Mollison J, Wordsworth S, Gray JR, Haites NE, et al.

4. Randomised evaluation of alternative electrosurgical modalities to treat bladder outflow obstruction in men with benign prostatic hyperplasia.

   By Fowler C, McAllister W, Plail R, Karim O, Yang Q.

5. A pragmatic randomised controlled trial of the cost-effectiveness of palliative therapies for patients with inoperable oesophageal cancer.

   By Shenfine J, McNamee P, Steen N, Bond J, Griffin SM.

6. Impact of computer-aided detection prompts on the sensitivity and specificity of screening mammography.

   By Taylor P, Champness J, Given- Wilson R, Johnston K, Potts H.

7. Issues in data monitoring and interim analysis of trials.

   By Grant AM, Altman DG, Babiker AB, Campbell MK, Clemens FJ, Darbyshire JH, et al.

8. Lay public’s understanding of equipoise and randomisation in randomised controlled trials.

   By Robinson EJ, Kerr CEP, Stevens AJ, Lilford RJ, Braunholtz DA, Edwards SJ, et al.

9. Clinical and cost-effectiveness of electroconvulsive therapy for depressive illness, schizophrenia, catatonia and mania: systematic reviews and economic modelling studies.

   By Greenhalgh J, Knight C, Hind D, Beverley C, Walters S.

10. Measurement of health-related quality of life for people with dementia: development of a new instrument (DEMQOL) and an evaluation of current methodology.

    By Smith SC, Lamping DL, Banerjee S, Harwood R, Foley B, Smith P, et al.

11. Clinical effectiveness and cost-effectiveness of drotrecogin alfa (activated) (Xigris^®) for the treatment of severe sepsis in adults: a systematic review and economic evaluation.

    By Green C, Dinnes J, Takeda A, Shepherd J, Hartwell D, Cave C, et al.

12. A methodological review of how heterogeneity has been examined in systematic reviews of diagnostic test accuracy.

    By Dinnes J, Deeks J, Kirby J, Roderick P.

13. Cervical screening programmes: can automation help? Evidence from systematic reviews, an economic analysis and a simulation modelling exercise applied to the UK.

    By Willis BH, Barton P, Pearmain P, Bryan S, Hyde C.

14. Laparoscopic surgery for inguinal hernia repair: systematic review of effectiveness and economic evaluation.

    By McCormack K, Wake B, Perez J, Fraser C, Cook J, McIntosh E, et al.

15. Clinical effectiveness, tolerability and cost-effectiveness of newer drugs for epilepsy in adults: a systematic review and economic evaluation.

    By Wilby J, Kainth A, Hawkins N, Epstein D, McIntosh H, McDaid C, et al.

16. A randomised controlled trial to compare the cost-effectiveness of tricyclic antidepressants, selective serotonin reuptake inhibitors and lofepramine.

    By Peveler R, Kendrick T, Buxton M, Longworth L, Baldwin D, Moore M, et al.

17. Clinical effectiveness and cost-effectiveness of immediate angioplasty for acute myocardial infarction: systematic review and economic evaluation.

    By Hartwell D, Colquitt J, Loveman E, Clegg AJ, Brodin H, Waugh N, et al.

18. A randomised controlled comparison of alternative strategies in stroke care.

    By Kalra L, Evans A, Perez I, Knapp M, Swift C, Donaldson N.

19. The investigation and analysis of critical incidents and adverse events in healthcare.

    By Woloshynowych M, Rogers S, Taylor-Adams S, Vincent C.

20. Potential use of routine databases in health technology assessment.

    By Raftery J, Roderick P, Stevens A.

21. Clinical and cost-effectiveness of newer immunosuppressive regimens in renal transplantation: a systematic review and modelling study.

    By Woodroffe R, Yao GL, Meads C, Bayliss S, Ready A, Raftery J, et al.

22. A systematic review and economic evaluation of alendronate, etidronate, risedronate, raloxifene and teriparatide for the prevention and treatment of postmenopausal osteoporosis.

    By Stevenson M, Lloyd Jones M, De Nigris E, Brewer N, Davis S, Oakley J.

23. A systematic review to examine the impact of psycho-educational interventions on health outcomes and costs in adults and children with difficult asthma.

    By Smith JR, Mugford M, Holland R, Candy B, Noble MJ, Harrison BDW, et al.

24. An evaluation of the costs, effectiveness and quality of renal replacement therapy provision in renal satellite units in England and Wales.

    By Roderick P, Nicholson T, Armitage A, Mehta R, Mullee M, Gerard K, et al.

25. Imatinib for the treatment of patients with unresectable and/or metastatic gastrointestinal stromal tumours: systematic review and economic evaluation.

    By Wilson J, Connock M, Song F, Yao G, Fry-Smith A, Raftery J, et al.

26. Indirect comparisons of competing interventions.

    By Glenny AM, Altman DG, Song F, Sakarovitch C, Deeks JJ, D’Amico R, et al.

27. Cost-effectiveness of alternative strategies for the initial medical management of non-ST elevation acute coronary syndrome: systematic review and decision-analytical modelling.

    By Robinson M, Palmer S, Sculpher M, Philips Z, Ginnelly L, Bowens A, et al.

28. Outcomes of electrically stimulated gracilis neosphincter surgery.

    By Tillin T, Chambers M, Feldman R.

29. The effectiveness and cost-effectiveness of pimecrolimus and tacrolimus for atopic eczema: a systematic review and economic evaluation.

    By Garside R, Stein K, Castelnuovo E, Pitt M, Ashcroft D, Dimmock P, et al.

30. Systematic review on urine albumin testing for early detection of diabetic complications.

    By Newman DJ, Mattock MB, Dawnay ABS, Kerry S, McGuire A, Yaqoob M, et al.

31. Randomised controlled trial of the cost-effectiveness of water-based therapy for lower limb osteoarthritis.

    By Cochrane T, Davey RC, Matthes Edwards SM.

32. Longer term clinical and economic benefits of offering acupuncture care to patients with chronic low back pain.

    By Thomas KJ, MacPherson H, Ratcliffe J, Thorpe L, Brazier J, Campbell M, et al.

33. Cost-effectiveness and safety of epidural steroids in the management of sciatica.

    By Price C, Arden N, Coglan L, Rogers P.

34. The British Rheumatoid Outcome Study Group (BROSG) randomised controlled trial to compare the effectiveness and cost-effectiveness of aggressive versus symptomatic therapy in established rheumatoid arthritis.

    By Symmons D, Tricker K, Roberts C, Davies L, Dawes P, Scott DL.

35. Conceptual framework and systematic review of the effects of participants’ and professionals’ preferences in randomised controlled trials.

    By King M, Nazareth I, Lampe F, Bower P, Chandler M, Morou M, et al.

36. The clinical and cost-effectiveness of implantable cardioverter defibrillators: a systematic review.

    By Bryant J, Brodin H, Loveman E, Payne E, Clegg A.

37. A trial of problem-solving by community mental health nurses for anxiety, depression and life difficulties among general practice patients. The CPN-GP study.

    By Kendrick T, Simons L, Mynors-Wallis L, Gray A, Lathlean J, Pickering R, et al.

38. The causes and effects of socio-demographic exclusions from clinical trials.

    By Bartlett C, Doyal L, Ebrahim S, Davey P, Bachmann M, Egger M, et al.

39. Is hydrotherapy cost-effective? A randomised controlled trial of combined hydrotherapy programmes compared with physiotherapy land techniques in children with juvenile idiopathic arthritis.

    By Epps H, Ginnelly L, Utley M, Southwood T, Gallivan S, Sculpher M, et al.

40. A randomised controlled trial and cost-effectiveness study of systematic screening (targeted and total population screening) versus routine practice for the detection of atrial fibrillation in people aged 65 and over. The SAFE study.

    By Hobbs FDR, Fitzmaurice DA, Mant J, Murray E, Jowett S, Bryan S, et al.

41. Displaced intracapsular hip fractures in fit, older people: a randomised comparison of reduction and fixation, bipolar hemiarthroplasty and total hip arthroplasty.

    By Keating JF, Grant A, Masson M, Scott NW, Forbes JF.

42. Long-term outcome of cognitive behaviour therapy clinical trials in central Scotland.

    By Durham RC, Chambers JA, Power KG, Sharp DM, Macdonald RR, Major KA, et al.

43. The effectiveness and cost-effectiveness of dual-chamber pacemakers compared with single-chamber pacemakers for bradycardia due to atrioventricular block or sick sinus syndrome: systematic review and economic evaluation.

    By Castelnuovo E, Stein K, Pitt M, Garside R, Payne E.

44. Newborn screening for congenital heart defects: a systematic review and cost-effectiveness analysis.

    By Knowles R, Griebsch I, Dezateux C, Brown J, Bull C, Wren C.

45. The clinical and cost-effectiveness of left ventricular assist devices for end-stage heart failure: a systematic review and economic evaluation.

    By Clegg AJ, Scott DA, Loveman E, Colquitt J, Hutchinson J, Royle P, et al.

46. The effectiveness of the Heidelberg Retina Tomograph and laser diagnostic glaucoma scanning system (GDx) in detecting and monitoring glaucoma.

    By Kwartz AJ, Henson DB, Harper RA, Spencer AF, McLeod D.

47. Clinical and cost-effectiveness of autologous chondrocyte implantation for cartilage defects in knee joints: systematic review and economic evaluation.

    By Clar C, Cummins E, McIntyre L, Thomas S, Lamb J, Bain L, et al.

48. Systematic review of effectiveness of different treatments for childhood retinoblastoma.

    By McDaid C, Hartley S, Bagnall A-M, Ritchie G, Light K, Riemsma R.

49. Towards evidence-based guidelines for the prevention of venous thromboembolism: systematic reviews of mechanical methods, oral anticoagulation, dextran and regional anaesthesia as thromboprophylaxis.

    By Roderick P, Ferris G, Wilson K, Halls H, Jackson D, Collins R, et al.

50. The effectiveness and cost-effectiveness of parent training/education programmes for the treatment of conduct disorder, including oppositional defiant disorder, in children.

    By Dretzke J, Frew E, Davenport C, Barlow J, Stewart-Brown S, Sandercock J, et al.

1. The clinical and cost-effectiveness of donepezil, rivastigmine, galantamine and memantine for Alzheimer’s disease.

   By Loveman E, Green C, Kirby J, Takeda A, Picot J, Payne E, et al.

2. FOOD: a multicentre randomised trial evaluating feeding policies in patients admitted to hospital with a recent stroke.

   By Dennis M, Lewis S, Cranswick G, Forbes J.

3. The clinical effectiveness and cost-effectiveness of computed tomography screening for lung cancer: systematic reviews.

   By Black C, Bagust A, Boland A, Walker S, McLeod C, De Verteuil R, et al.

4. A systematic review of the effectiveness and cost-effectiveness of neuroimaging assessments used to visualise the seizure focus in people with refractory epilepsy being considered for surgery.

   By Whiting P, Gupta R, Burch J, Mujica Mota RE, Wright K, Marson A, et al.

5. Comparison of conference abstracts and presentations with full-text articles in the health technology assessments of rapidly evolving technologies.

   By Dundar Y, Dodd S, Dickson R, Walley T, Haycox A, Williamson PR.

6. Systematic review and evaluation of methods of assessing urinary incontinence.

   By Martin JL, Williams KS, Abrams KR, Turner DA, Sutton AJ, Chapple C, et al.

7. The clinical effectiveness and cost-effectiveness of newer drugs for children with epilepsy. A systematic review.

   By Connock M, Frew E, Evans B-W, Bryan S, Cummins C, Fry-Smith A, et al.

8. Surveillance of Barrett’s oesophagus: exploring the uncertainty through systematic review, expert workshop and economic modelling.

   By Garside R, Pitt M, Somerville M, Stein K, Price A, Gilbert N.

9. Topotecan, pegylated liposomal doxorubicin hydrochloride and paclitaxel for second-line or subsequent treatment of advanced ovarian cancer: a systematic review and economic evaluation.

   By Main C, Bojke L, Griffin S, Norman G, Barbieri M, Mather L, et al.

10. Evaluation of molecular techniques in prediction and diagnosis of cytomegalovirus disease in immunocompromised patients.

    By Szczepura A, Westmoreland D, Vinogradova Y, Fox J, Clark M.

11. Screening for thrombophilia in high-risk situations: systematic review and cost-effectiveness analysis. The Thrombosis: Risk and Economic Assessment of Thrombophilia Screening (TREATS) study.

    By Wu O, Robertson L, Twaddle S, Lowe GDO, Clark P, Greaves M, et al.

12. A series of systematic reviews to inform a decision analysis for sampling and treating infected diabetic foot ulcers.

    By Nelson EA, O’Meara S, Craig D, Iglesias C, Golder S, Dalton J, et al.

13. Randomised clinical trial, observational study and assessment of cost-effectiveness of the treatment of varicose veins (REACTIV trial).

    By Michaels JA, Campbell WB, Brazier JE, MacIntyre JB, Palfreyman SJ, Ratcliffe J, et al.

14. The cost-effectiveness of screening for oral cancer in primary care.

    By Speight PM, Palmer S, Moles DR, Downer MC, Smith DH, Henriksson M, et al.

15. Measurement of the clinical and cost-effectiveness of non-invasive diagnostic testing strategies for deep vein thrombosis.

    By Goodacre S, Sampson F, Stevenson M, Wailoo A, Sutton A, Thomas S, et al.

16. Systematic review of the effectiveness and cost-effectiveness of HealOzone^® for the treatment of occlusal pit/fissure caries and root caries.

    By Brazzelli M, McKenzie L, Fielding S, Fraser C, Clarkson J, Kilonzo M, et al.

17. Randomised controlled trials of conventional antipsychotic versus new atypical drugs, and new atypical drugs versus clozapine, in people with schizophrenia responding poorly to, or intolerant of, current drug treatment.

    By Lewis SW, Davies L, Jones PB, Barnes TRE, Murray RM, Kerwin R, et al.

18. Diagnostic tests and algorithms used in the investigation of haematuria: systematic reviews and economic evaluation.

    By Rodgers M, Nixon J, Hempel S, Aho T, Kelly J, Neal D, et al.

19. Cognitive behavioural therapy in addition to antispasmodic therapy for irritable bowel syndrome in primary care: randomised controlled trial.

    By Kennedy TM, Chalder T, McCrone P, Darnley S, Knapp M, Jones RH, et al.

20. A systematic review of the clinical effectiveness and cost-effectiveness of enzyme replacement therapies for Fabry’s disease and mucopolysaccharidosis type 1.

    By Connock M, Juarez-Garcia A, Frew E, Mans A, Dretzke J, Fry-Smith A, et al.

21. Health benefits of antiviral therapy for mild chronic hepatitis C: randomised controlled trial and economic evaluation.

    By Wright M, Grieve R, Roberts J, Main J, Thomas HC, on behalf of the UK Mild Hepatitis C Trial Investigators.

22. Pressure relieving support surfaces: a randomised evaluation.

    By Nixon J, Nelson EA, Cranny G, Iglesias CP, Hawkins K, Cullum NA, et al.

23. A systematic review and economic model of the effectiveness and cost-effectiveness of methylphenidate, dexamfetamine and atomoxetine for the treatment of attention deficit hyperactivity disorder in children and adolescents.

    By King S, Griffin S, Hodges Z, Weatherly H, Asseburg C, Richardson G, et al.

24. The clinical effectiveness and cost-effectiveness of enzyme replacement therapy for Gaucher’s disease: a systematic review.

    By Connock M, Burls A, Frew E, Fry-Smith A, Juarez-Garcia A, McCabe C, et al.

25. Effectiveness and cost-effectiveness of salicylic acid and cryotherapy for cutaneous warts. An economic decision model.

    By Thomas KS, Keogh-Brown MR, Chalmers JR, Fordham RJ, Holland RC, Armstrong SJ, et al.

26. A systematic literature review of the effectiveness of non-pharmacological interventions to prevent wandering in dementia and evaluation of the ethical implications and acceptability of their use.

    By Robinson L, Hutchings D, Corner L, Beyer F, Dickinson H, Vanoli A, et al.

27. A review of the evidence on the effects and costs of implantable cardioverter defibrillator therapy in different patient groups, and modelling of cost-effectiveness and cost–utility for these groups in a UK context.

    By Buxton M, Caine N, Chase D, Connelly D, Grace A, Jackson C, et al.

28. Adefovir dipivoxil and pegylated interferon alfa-2a for the treatment of chronic hepatitis B: a systematic review and economic evaluation.

    By Shepherd J, Jones J, Takeda A, Davidson P, Price A.

29. An evaluation of the clinical and cost-effectiveness of pulmonary artery catheters in patient management in intensive care: a systematic review and a randomised controlled trial.

    By Harvey S, Stevens K, Harrison D, Young D, Brampton W, McCabe C, et al.

30. Accurate, practical and cost-effective assessment of carotid stenosis in the UK.

    By Wardlaw JM, Chappell FM, Stevenson M, De Nigris E, Thomas S, Gillard J, et al.

31. Etanercept and infliximab for the treatment of psoriatic arthritis: a systematic review and economic evaluation.

    By Woolacott N, Bravo Vergel Y, Hawkins N, Kainth A, Khadjesari Z, Misso K, et al.

32. The cost-effectiveness of testing for hepatitis C in former injecting drug users.

    By Castelnuovo E, Thompson-Coon J, Pitt M, Cramp M, Siebert U, Price A, et al.

33. Computerised cognitive behaviour therapy for depression and anxiety update: a systematic review and economic evaluation.

    By Kaltenthaler E, Brazier J, De Nigris E, Tumur I, Ferriter M, Beverley C, et al.

34. Cost-effectiveness of using prognostic information to select women with breast cancer for adjuvant systemic therapy.

    By Williams C, Brunskill S, Altman D, Briggs A, Campbell H, Clarke M, et al.

35. Psychological therapies including dialectical behaviour therapy for borderline personality disorder: a systematic review and preliminary economic evaluation.

    By Brazier J, Tumur I, Holmes M, Ferriter M, Parry G, Dent-Brown K, et al.

36. Clinical effectiveness and cost-effectiveness of tests for the diagnosis and investigation of urinary tract infection in children: a systematic review and economic model.

    By Whiting P, Westwood M, Bojke L, Palmer S, Richardson G, Cooper J, et al.

37. Cognitive behavioural therapy in chronic fatigue syndrome: a randomised controlled trial of an outpatient group programme.

    By O’Dowd H, Gladwell P, Rogers CA, Hollinghurst S, Gregory A.

38. A comparison of the cost-effectiveness of five strategies for the prevention of nonsteroidal anti-inflammatory drug-induced gastrointestinal toxicity: a systematic review with economic modelling.

    By Brown TJ, Hooper L, Elliott RA, Payne K, Webb R, Roberts C, et al.

39. The effectiveness and cost-effectiveness of computed tomography screening for coronary artery disease: systematic review.

    By Waugh N, Black C, Walker S, McIntyre L, Cummins E, Hillis G.

40. What are the clinical outcome and cost-effectiveness of endoscopy undertaken by nurses when compared with doctors? A Multi-Institution Nurse Endoscopy Trial (MINuET).

    By Williams J, Russell I, Durai D, Cheung W-Y, Farrin A, Bloor K, et al.

41. The clinical and cost-effectiveness of oxaliplatin and capecitabine for the adjuvant treatment of colon cancer: systematic review and economic evaluation.

    By Pandor A, Eggington S, Paisley S, Tappenden P, Sutcliffe P.

42. A systematic review of the effectiveness of adalimumab, etanercept and infliximab for the treatment of rheumatoid arthritis in adults and an economic evaluation of their cost-effectiveness.

    By Chen Y-F, Jobanputra P, Barton P, Jowett S, Bryan S, Clark W, et al.

43. Telemedicine in dermatology: a randomised controlled trial.

    By Bowns IR, Collins K, Walters SJ, McDonagh AJG.

44. Cost-effectiveness of cell salvage and alternative methods of minimising perioperative allogeneic blood transfusion: a systematic review and economic model.

    By Davies L, Brown TJ, Haynes S, Payne K, Elliott RA, McCollum C.

45. Clinical effectiveness and cost-effectiveness of laparoscopic surgery for colorectal cancer: systematic reviews and economic evaluation.

    By Murray A, Lourenco T, de Verteuil R, Hernandez R, Fraser C, McKinley A, et al.

46. Etanercept and efalizumab for the treatment of psoriasis: a systematic review.

    By Woolacott N, Hawkins N, Mason A, Kainth A, Khadjesari Z, Bravo Vergel Y, et al.

47. Systematic reviews of clinical decision tools for acute abdominal pain.

    By Liu JLY, Wyatt JC, Deeks JJ, Clamp S, Keen J, Verde P, et al.

48. Evaluation of the ventricular assist device programme in the UK.

    By Sharples L, Buxton M, Caine N, Cafferty F, Demiris N, Dyer M, et al.

49. A systematic review and economic model of the clinical and cost-effectiveness of immunosuppressive therapy for renal transplantation in children.

    By Yao G, Albon E, Adi Y, Milford D, Bayliss S, Ready A, et al.

50. Amniocentesis results: investigation of anxiety. The ARIA trial.

    By Hewison J, Nixon J, Fountain J, Cocks K, Jones C, Mason G, et al.

1. Pemetrexed disodium for the treatment of malignant pleural mesothelioma: a systematic review and economic evaluation.

   By Dundar Y, Bagust A, Dickson R, Dodd S, Green J, Haycox A, et al.

2. A systematic review and economic model of the clinical effectiveness and cost-effectiveness of docetaxel in combination with prednisone or prednisolone for the treatment of hormone-refractory metastatic prostate cancer.

   By Collins R, Fenwick E, Trowman R, Perard R, Norman G, Light K, et al.

3. A systematic review of rapid diagnostic tests for the detection of tuberculosis infection.

   By Dinnes J, Deeks J, Kunst H, Gibson A, Cummins E, Waugh N, et al.

4. The clinical effectiveness and cost-effectiveness of strontium ranelate for the prevention of osteoporotic fragility fractures in postmenopausal women.

   By Stevenson M, Davis S, Lloyd-Jones M, Beverley C.

5. A systematic review of quantitative and qualitative research on the role and effectiveness of written information available to patients about individual medicines.

   By Raynor DK, Blenkinsopp A, Knapp P, Grime J, Nicolson DJ, Pollock K, et al.

6. Oral naltrexone as a treatment for relapse prevention in formerly opioid-dependent drug users: a systematic review and economic evaluation.

   By Adi Y, Juarez-Garcia A, Wang D, Jowett S, Frew E, Day E, et al.

7. Glucocorticoid-induced osteoporosis: a systematic review and cost–utility analysis.

   By Kanis JA, Stevenson M, McCloskey EV, Davis S, Lloyd-Jones M.

8. Epidemiological, social, diagnostic and economic evaluation of population screening for genital chlamydial infection.

   By Low N, McCarthy A, Macleod J, Salisbury C, Campbell R, Roberts TE, et al.

9. Methadone and buprenorphine for the management of opioid dependence: a systematic review and economic evaluation.

   By Connock M, Juarez-Garcia A, Jowett S, Frew E, Liu Z, Taylor RJ, et al.

10. Exercise Evaluation Randomised Trial (EXERT): a randomised trial comparing GP referral for leisure centre-based exercise, community-based walking and advice only.

    By Isaacs AJ, Critchley JA, See Tai S, Buckingham K, Westley D, Harridge SDR, et al.

11. Interferon alfa (pegylated and non-pegylated) and ribavirin for the treatment of mild chronic hepatitis C: a systematic review and economic evaluation.

    By Shepherd J, Jones J, Hartwell D, Davidson P, Price A, Waugh N.

12. Systematic review and economic evaluation of bevacizumab and cetuximab for the treatment of metastatic colorectal cancer.

    By Tappenden P, Jones R, Paisley S, Carroll C.

13. A systematic review and economic evaluation of epoetin alfa, epoetin beta and darbepoetin alfa in anaemia associated with cancer, especially that attributable to cancer treatment.

    By Wilson J, Yao GL, Raftery J, Bohlius J, Brunskill S, Sandercock J, et al.

14. A systematic review and economic evaluation of statins for the prevention of coronary events.

    By Ward S, Lloyd Jones M, Pandor A, Holmes M, Ara R, Ryan A, et al.

15. A systematic review of the effectiveness and cost-effectiveness of different models of community-based respite care for frail older people and their carers.

    By Mason A, Weatherly H, Spilsbury K, Arksey H, Golder S, Adamson J, et al.

16. Additional therapy for young children with spastic cerebral palsy: a randomised controlled trial.

    By Weindling AM, Cunningham CC, Glenn SM, Edwards RT, Reeves DJ.

17. Screening for type 2 diabetes: literature review and economic modelling.

    By Waugh N, Scotland G, McNamee P, Gillett M, Brennan A, Goyder E, et al.

18. The effectiveness and cost-effectiveness of cinacalcet for secondary hyperparathyroidism in end-stage renal disease patients on dialysis: a systematic review and economic evaluation.

    By Garside R, Pitt M, Anderson R, Mealing S, Roome C, Snaith A, et al.

19. The clinical effectiveness and cost-effectiveness of gemcitabine for metastatic breast cancer: a systematic review and economic evaluation.

    By Takeda AL, Jones J, Loveman E, Tan SC, Clegg AJ.

20. A systematic review of duplex ultrasound, magnetic resonance angiography and computed tomography angiography for the diagnosis and assessment of symptomatic, lower limb peripheral arterial disease.

    By Collins R, Cranny G, Burch J, Aguiar-Ibáñez R, Craig D, Wright K, et al.

21. The clinical effectiveness and cost-effectiveness of treatments for children with idiopathic steroid-resistant nephrotic syndrome: a systematic review.

    By Colquitt JL, Kirby J, Green C, Cooper K, Trompeter RS.

22. A systematic review of the routine monitoring of growth in children of primary school age to identify growth-related conditions.

    By Fayter D, Nixon J, Hartley S, Rithalia A, Butler G, Rudolf M, et al.

23. Systematic review of the effectiveness of preventing and treating Staphylococcus aureus carriage in reducing peritoneal catheter-related infections.

    By McCormack K, Rabindranath K, Kilonzo M, Vale L, Fraser C, McIntyre L, et al.

24. The clinical effectiveness and cost of repetitive transcranial magnetic stimulation versus electroconvulsive therapy in severe depression: a multicentre pragmatic randomised controlled trial and economic analysis.

    By McLoughlin DM, Mogg A, Eranti S, Pluck G, Purvis R, Edwards D, et al.

25. A randomised controlled trial and economic evaluation of direct versus indirect and individual versus group modes of speech and language therapy for children with primary language impairment.

    By Boyle J, McCartney E, Forbes J, O’Hare A.

26. Hormonal therapies for early breast cancer: systematic review and economic evaluation.

    By Hind D, Ward S, De Nigris E, Simpson E, Carroll C, Wyld L.

27. Cardioprotection against the toxic effects of anthracyclines given to children with cancer: a systematic review.

    By Bryant J, Picot J, Levitt G, Sullivan I, Baxter L, Clegg A.

28. Adalimumab, etanercept and infliximab for the treatment of ankylosing spondylitis: a systematic review and economic evaluation.

    By McLeod C, Bagust A, Boland A, Dagenais P, Dickson R, Dundar Y, et al.

29. Prenatal screening and treatment strategies to prevent group B streptococcal and other bacterial infections in early infancy: cost-effectiveness and expected value of information analyses.

    By Colbourn T, Asseburg C, Bojke L, Philips Z, Claxton K, Ades AE, et al.

30. Clinical effectiveness and cost-effectiveness of bone morphogenetic proteins in the non-healing of fractures and spinal fusion: a systematic review.

    By Garrison KR, Donell S, Ryder J, Shemilt I, Mugford M, Harvey I, et al.

31. A randomised controlled trial of postoperative radiotherapy following breast-conserving surgery in a minimum-risk older population. The PRIME trial.

    By Prescott RJ, Kunkler IH, Williams LJ, King CC, Jack W, van der Pol M, et al.

32. Current practice, accuracy, effectiveness and cost-effectiveness of the school entry hearing screen.

    By Bamford J, Fortnum H, Bristow K, Smith J, Vamvakas G, Davies L, et al.

33. The clinical effectiveness and cost-effectiveness of inhaled insulin in diabetes mellitus: a systematic review and economic evaluation.

    By Black C, Cummins E, Royle P, Philip S, Waugh N.

34. Surveillance of cirrhosis for hepatocellular carcinoma: systematic review and economic analysis.

    By Thompson Coon J, Rogers G, Hewson P, Wright D, Anderson R, Cramp M, et al.

35. The Birmingham Rehabilitation Uptake Maximisation Study (BRUM). Homebased compared with hospital-based cardiac rehabilitation in a multi-ethnic population: cost-effectiveness and patient adherence.

    By Jolly K, Taylor R, Lip GYH, Greenfield S, Raftery J, Mant J, et al.

36. A systematic review of the clinical, public health and cost-effectiveness of rapid diagnostic tests for the detection and identification of bacterial intestinal pathogens in faeces and food.

    By Abubakar I, Irvine L, Aldus CF, Wyatt GM, Fordham R, Schelenz S, et al.

37. A randomised controlled trial examining the longer-term outcomes of standard versus new antiepileptic drugs. The SANAD trial.

    By Marson AG, Appleton R, Baker GA, Chadwick DW, Doughty J, Eaton B, et al.

38. Clinical effectiveness and cost-effectiveness of different models of managing long-term oral anti-coagulation therapy: a systematic review and economic modelling.

    By Connock M, Stevens C, Fry-Smith A, Jowett S, Fitzmaurice D, Moore D, et al.

39. A systematic review and economic model of the clinical effectiveness and cost-effectiveness of interventions for preventing relapse in people with bipolar disorder.

    By Soares-Weiser K, Bravo Vergel Y, Beynon S, Dunn G, Barbieri M, Duffy S, et al.

40. Taxanes for the adjuvant treatment of early breast cancer: systematic review and economic evaluation.

    By Ward S, Simpson E, Davis S, Hind D, Rees A, Wilkinson A.

41. The clinical effectiveness and cost-effectiveness of screening for open angle glaucoma: a systematic review and economic evaluation.

    By Burr JM, Mowatt G, Hernández R, Siddiqui MAR, Cook J, Lourenco T, et al.

42. Acceptability, benefit and costs of early screening for hearing disability: a study of potential screening tests and models.

    By Davis A, Smith P, Ferguson M, Stephens D, Gianopoulos I.

43. Contamination in trials of educational interventions.

    By Keogh-Brown MR, Bachmann MO, Shepstone L, Hewitt C, Howe A, Ramsay CR, et al.

44. Overview of the clinical effectiveness of positron emission tomography imaging in selected cancers.

    By Facey K, Bradbury I, Laking G, Payne E.

45. The effectiveness and cost-effectiveness of carmustine implants and temozolomide for the treatment of newly diagnosed high-grade glioma: a systematic review and economic evaluation.

    By Garside R, Pitt M, Anderson R, Rogers G, Dyer M, Mealing S, et al.

46. Drug-eluting stents: a systematic review and economic evaluation.

    By Hill RA, Boland A, Dickson R, Dündar Y, Haycox A, McLeod C, et al.

47. The clinical effectiveness and cost-effectiveness of cardiac resynchronisation (biventricular pacing) for heart failure: systematic review and economic model.

    By Fox M, Mealing S, Anderson R, Dean J, Stein K, Price A, et al.

48. Recruitment to randomised trials: strategies for trial enrolment and participation study. The STEPS study.

    By Campbell MK, Snowdon C, Francis D, Elbourne D, McDonald AM, Knight R, et al.

49. Cost-effectiveness of functional cardiac testing in the diagnosis and management of coronary artery disease: a randomised controlled trial. The CECaT trial.

    By Sharples L, Hughes V, Crean A, Dyer M, Buxton M, Goldsmith K, et al.

50. Evaluation of diagnostic tests when there is no gold standard. A review of methods.

    By Rutjes AWS, Reitsma JB, Coomarasamy A, Khan KS, Bossuyt PMM.

51. Systematic reviews of the clinical effectiveness and cost-effectiveness of proton pump inhibitors in acute upper gastrointestinal bleeding.

    By Leontiadis GI, Sreedharan A, Dorward S, Barton P, Delaney B, Howden CW, et al.

52. A review and critique of modelling in prioritising and designing screening programmes.

    By Karnon J, Goyder E, Tappenden P, McPhie S, Towers I, Brazier J, et al.

53. An assessment of the impact of the NHS Health Technology Assessment Programme.

    By Hanney S, Buxton M, Green C, Coulson D, Raftery J.

1. A systematic review and economic model of switching from nonglycopeptide to glycopeptide antibiotic prophylaxis for surgery.

   By Cranny G, Elliott R, Weatherly H, Chambers D, Hawkins N, Myers L, et al.

2. ‘Cut down to quit’ with nicotine replacement therapies in smoking cessation: a systematic review of effectiveness and economic analysis.

   By Wang D, Connock M, Barton P, Fry-Smith A, Aveyard P, Moore D.

3. A systematic review of the effectiveness of strategies for reducing fracture risk in children with juvenile idiopathic arthritis with additional data on long-term risk of fracture and cost of disease management.

   By Thornton J, Ashcroft D, O’Neill T, Elliott R, Adams J, Roberts C, et al.

4. Does befriending by trained lay workers improve psychological well-being and quality of life for carers of people with dementia, and at what cost? A randomised controlled trial.

   By Charlesworth G, Shepstone L, Wilson E, Thalanany M, Mugford M, Poland F.

5. A multi-centre retrospective cohort study comparing the efficacy, safety and cost-effectiveness of hysterectomy and uterine artery embolisation for the treatment of symptomatic uterine fibroids. The HOPEFUL study.

   By Hirst A, Dutton S, Wu O, Briggs A, Edwards C, Waldenmaier L, et al.

6. Methods of prediction and prevention of pre-eclampsia: systematic reviews of accuracy and effectiveness literature with economic modelling.

   By Meads CA, Cnossen JS, Meher S, Juarez-Garcia A, ter Riet G, Duley L, et al.

7. The use of economic evaluations in NHS decision-making: a review and empirical investigation.

   By Williams I, McIver S, Moore D, Bryan S.

8. Stapled haemorrhoidectomy (haemorrhoidopexy) for the treatment of haemorrhoids: a systematic review and economic evaluation.

   By Burch J, Epstein D, Baba-Akbari A, Weatherly H, Fox D, Golder S, et al.

9. The clinical effectiveness of diabetes education models for Type 2 diabetes: a systematic review.

   By Loveman E, Frampton GK, Clegg AJ.

10. Payment to healthcare professionals for patient recruitment to trials: systematic review and qualitative study.

    By Raftery J, Bryant J, Powell J, Kerr C, Hawker S.

11. Cyclooxygenase-2 selective non-steroidal anti-inflammatory drugs (etodolac, meloxicam, celecoxib, rofecoxib, etoricoxib, valdecoxib and lumiracoxib) for osteoarthritis and rheumatoid arthritis: a systematic review and economic evaluation.

    By Chen Y-F, Jobanputra P, Barton P, Bryan S, Fry-Smith A, Harris G, et al.

12. The clinical effectiveness and cost-effectiveness of central venous catheters treated with anti-infective agents in preventing bloodstream infections: a systematic review and economic evaluation.

    By Hockenhull JC, Dwan K, Boland A, Smith G, Bagust A, Dundar Y, et al.

13. Stepped treatment of older adults on laxatives. The STOOL trial.

    By Mihaylov S, Stark C, McColl E, Steen N, Vanoli A, Rubin G, et al.

14. A randomised controlled trial of cognitive behaviour therapy in adolescents with major depression treated by selective serotonin reuptake inhibitors. The ADAPT trial.

    By Goodyer IM, Dubicka B, Wilkinson P, Kelvin R, Roberts C, Byford S, et al.

15. The use of irinotecan, oxaliplatin and raltitrexed for the treatment of advanced colorectal cancer: systematic review and economic evaluation.

    By Hind D, Tappenden P, Tumur I, Eggington E, Sutcliffe P, Ryan A.

16. Ranibizumab and pegaptanib for the treatment of age-related macular degeneration: a systematic review and economic evaluation.

    By Colquitt JL, Jones J, Tan SC, Takeda A, Clegg AJ, Price A.

17. Systematic review of the clinical effectiveness and cost-effectiveness of 64-slice or higher computed tomography angiography as an alternative to invasive coronary angiography in the investigation of coronary artery disease.

    By Mowatt G, Cummins E, Waugh N, Walker S, Cook J, Jia X, et al.

18. Structural neuroimaging in psychosis: a systematic review and economic evaluation.

    By Albon E, Tsourapas A, Frew E, Davenport C, Oyebode F, Bayliss S, et al.

19. Systematic review and economic analysis of the comparative effectiveness of different inhaled corticosteroids and their usage with long-acting beta₂ agonists for the treatment of chronic asthma in adults and children aged 12 years and over.

    By Shepherd J, Rogers G, Anderson R, Main C, Thompson-Coon J, Hartwell D, et al.

20. Systematic review and economic analysis of the comparative effectiveness of different inhaled corticosteroids and their usage with long-acting beta₂ agonists for the treatment of chronic asthma in children under the age of 12 years.

    By Main C, Shepherd J, Anderson R, Rogers G, Thompson-Coon J, Liu Z, et al.

21. Ezetimibe for the treatment of hypercholesterolaemia: a systematic review and economic evaluation.

    By Ara R, Tumur I, Pandor A, Duenas A, Williams R, Wilkinson A, et al.

22. Topical or oral ibuprofen for chronic knee pain in older people. The TOIB study.

    By Underwood M, Ashby D, Carnes D, Castelnuovo E, Cross P, Harding G, et al.

23. A prospective randomised comparison of minor surgery in primary and secondary care. The MiSTIC trial.

    By George S, Pockney P, Primrose J, Smith H, Little P, Kinley H, et al.

24. A review and critical appraisal of measures of therapist–patient interactions in mental health settings.

    By Cahill J, Barkham M, Hardy G, Gilbody S, Richards D, Bower P, et al.

25. The clinical effectiveness and cost-effectiveness of screening programmes for amblyopia and strabismus in children up to the age of 4–5 years: a systematic review and economic evaluation.

    By Carlton J, Karnon J, Czoski-Murray C, Smith KJ, Marr J.

26. A systematic review of the clinical effectiveness and cost-effectiveness and economic modelling of minimal incision total hip replacement approaches in the management of arthritic disease of the hip.

    By de Verteuil R, Imamura M, Zhu S, Glazener C, Fraser C, Munro N, et al.

27. A preliminary model-based assessment of the cost–utility of a screening programme for early age-related macular degeneration.

    By Karnon J, Czoski-Murray C, Smith K, Brand C, Chakravarthy U, Davis S, et al.

28. Intravenous magnesium sulphate and sotalol for prevention of atrial fibrillation after coronary artery bypass surgery: a systematic review and economic evaluation.

    By Shepherd J, Jones J, Frampton GK, Tanajewski L, Turner D, Price A.

29. Absorbent products for urinary/faecal incontinence: a comparative evaluation of key product categories.

    By Fader M, Cottenden A, Getliffe K, Gage H, Clarke-O’Neill S, Jamieson K, et al.

30. A systematic review of repetitive functional task practice with modelling of resource use, costs and effectiveness.

    By French B, Leathley M, Sutton C, McAdam J, Thomas L, Forster A, et al.

31. The effectiveness and cost-effectivness of minimal access surgery amongst people with gastro-oesophageal reflux disease – a UK collaborative study. The reflux trial.

    By Grant A, Wileman S, Ramsay C, Bojke L, Epstein D, Sculpher M, et al.

32. Time to full publication of studies of anti-cancer medicines for breast cancer and the potential for publication bias: a short systematic review.

    By Takeda A, Loveman E, Harris P, Hartwell D, Welch K.

33. Performance of screening tests for child physical abuse in accident and emergency departments.

    By Woodman J, Pitt M, Wentz R, Taylor B, Hodes D, Gilbert RE.

34. Curative catheter ablation in atrial fibrillation and typical atrial flutter: systematic review and economic evaluation.

    By Rodgers M, McKenna C, Palmer S, Chambers D, Van Hout S, Golder S, et al.

35. Systematic review and economic modelling of effectiveness and cost utility of surgical treatments for men with benign prostatic enlargement.

    By Lourenco T, Armstrong N, N’Dow J, Nabi G, Deverill M, Pickard R, et al.

36. Immunoprophylaxis against respiratory syncytial virus (RSV) with palivizumab in children: a systematic review and economic evaluation.

    By Wang D, Cummins C, Bayliss S, Sandercock J, Burls A.

1. Deferasirox for the treatment of iron overload associated with regular blood transfusions (transfusional haemosiderosis) in patients suffering with chronic anaemia: a systematic review and economic evaluation.

   By McLeod C, Fleeman N, Kirkham J, Bagust A, Boland A, Chu P, et al.

2. Thrombophilia testing in people with venous thromboembolism: systematic review and cost-effectiveness analysis.

   By Simpson EL, Stevenson MD, Rawdin A, Papaioannou D.

3. Surgical procedures and non-surgical devices for the management of non-apnoeic snoring: a systematic review of clinical effects and associated treatment costs.

   By Main C, Liu Z, Welch K, Weiner G, Quentin Jones S, Stein K.

4. Continuous positive airway pressure devices for the treatment of obstructive sleep apnoea–hypopnoea syndrome: a systematic review and economic analysis.

   By McDaid C, Griffin S, Weatherly H, Durée K, van der Burgt M, van Hout S, Akers J, et al.

5. Use of classical and novel biomarkers as prognostic risk factors for localised prostate cancer: a systematic review.

   By Sutcliffe P, Hummel S, Simpson E, Young T, Rees A, Wilkinson A, et al.

6. The harmful health effects of recreational ecstasy: a systematic review of observational evidence.

   By Rogers G, Elston J, Garside R, Roome C, Taylor R, Younger P, et al.

7. Systematic review of the clinical effectiveness and cost-effectiveness of oesophageal Doppler monitoring in critically ill and high-risk surgical patients.

   By Mowatt G, Houston G, Hernández R, de Verteuil R, Fraser C, Cuthbertson B, et al.

8. The use of surrogate outcomes in model-based cost-effectiveness analyses: a survey of UK Health Technology Assessment reports.

   By Taylor RS, Elston J.

9. Controlling Hypertension and Hypotension Immediately Post Stroke (CHHIPS) – a randomised controlled trial.

   By Potter J, Mistri A, Brodie F, Chernova J, Wilson E, Jagger C, et al.

10. Routine antenatal anti-D prophylaxis for RhD-negative women: a systematic review and economic evaluation.

    By Pilgrim H, Lloyd-Jones M, Rees A.

11. Amantadine, oseltamivir and zanamivir for the prophylaxis of influenza (including a review of existing guidance no. 67): a systematic review and economic evaluation.

    By Tappenden P, Jackson R, Cooper K, Rees A, Simpson E, Read R, et al.

12. Improving the evaluation of therapeutic interventions in multiple sclerosis: the role of new psychometric methods.

    By Hobart J, Cano S.

13. Treatment of severe ankle sprain: a pragmatic randomised controlled trial comparing the clinical effectiveness and cost-effectiveness of three types of mechanical ankle support with tubular bandage. The CAST trial.

    By Cooke MW, Marsh JL, Clark M, Nakash R, Jarvis RM, Hutton JL, et al. , on behalf of the CAST trial group.

14. Non-occupational postexposure prophylaxis for HIV: a systematic review.

    By Bryant J, Baxter L, Hird S.

15. Blood glucose self-monitoring in type 2 diabetes: a randomised controlled trial.

    By Farmer AJ, Wade AN, French DP, Simon J, Yudkin P, Gray A, et al.

16. How far does screening women for domestic (partner) violence in different health-care settings meet criteria for a screening programme? Systematic reviews of nine UK National Screening Committee criteria.

    By Feder G, Ramsay J, Dunne D, Rose M, Arsene C, Norman R, et al.

17. Spinal cord stimulation for chronic pain of neuropathic or ischaemic origin: systematic review and economic evaluation.

    By Simpson, EL, Duenas A, Holmes MW, Papaioannou D, Chilcott J.

18. The role of magnetic resonance imaging in the identification of suspected acoustic neuroma: a systematic review of clinical and cost-effectiveness and natural history.

    By Fortnum H, O’Neill C, Taylor R, Lenthall R, Nikolopoulos T, Lightfoot G, et al.

19. Dipsticks and diagnostic algorithms in urinary tract infection: development and validation, randomised trial, economic analysis, observational cohort and qualitative study.

    By Little P, Turner S, Rumsby K, Warner G, Moore M, Lowes JA, et al.

20. Systematic review of respite care in the frail elderly.

    By Shaw C, McNamara R, Abrams K, Cannings-John R, Hood K, Longo M, et al.

21. Neuroleptics in the treatment of aggressive challenging behaviour for people with intellectual disabilities: a randomised controlled trial (NACHBID).

    By Tyrer P, Oliver-Africano P, Romeo R, Knapp M, Dickens S, Bouras N, et al.

22. Randomised controlled trial to determine the clinical effectiveness and cost-effectiveness of selective serotonin reuptake inhibitors plus supportive care, versus supportive care alone, for mild to moderate depression with somatic symptoms in primary care: the THREAD (THREshold for AntiDepressant response) study.

    By Kendrick T, Chatwin J, Dowrick C, Tylee A, Morriss R, Peveler R, et al.

23. Diagnostic strategies using DNA testing for hereditary haemochromatosis in at-risk populations: a systematic review and economic evaluation.

    By Bryant J, Cooper K, Picot J, Clegg A, Roderick P, Rosenberg W, et al.

24. Enhanced external counterpulsation for the treatment of stable angina and heart failure: a systematic review and economic analysis.

    By McKenna C, McDaid C, Suekarran S, Hawkins N, Claxton K, Light K, et al.

25. Development of a decision support tool for primary care management of patients with abnormal liver function tests without clinically apparent liver disease: a record-linkage population cohort study and decision analysis (ALFIE).

    By Donnan PT, McLernon D, Dillon JF, Ryder S, Roderick P, Sullivan F, et al.

26. A systematic review of presumed consent systems for deceased organ donation.

    By Rithalia A, McDaid C, Suekarran S, Norman G, Myers L, Sowden A.

27. Paracetamol and ibuprofen for the treatment of fever in children: the PITCH randomised controlled trial.

    By Hay AD, Redmond NM, Costelloe C, Montgomery AA, Fletcher M, Hollinghurst S, et al.

28. A randomised controlled trial to compare minimally invasive glucose monitoring devices with conventional monitoring in the management of insulin-treated diabetes mellitus (MITRE).

    By Newman SP, Cooke D, Casbard A, Walker S, Meredith S, Nunn A, et al.

29. Sensitivity analysis in economic evaluation: an audit of NICE current practice and a review of its use and value in decision-making.

    By Andronis L, Barton P, Bryan S.

30. Trastuzumab for the treatment of primary breast cancer in HER2-positive women: a single technology appraisal.

    By Ward S, Pilgrim H, Hind D.

31. Docetaxel for the adjuvant treatment of early node-positive breast cancer: a single technology appraisal.

    By Chilcott J, Lloyd Jones M, Wilkinson A.

32. The use of paclitaxel in the management of early stage breast cancer.

    By Griffin S, Dunn G, Palmer S, Macfarlane K, Brent S, Dyker A, et al.

33. Rituximab for the first-line treatment of stage III/IV follicular non-Hodgkin’s lymphoma.

    By Dundar Y, Bagust A, Hounsome J, McLeod C, Boland A, Davis H, et al.

34. Bortezomib for the treatment of multiple myeloma patients.

    By Green C, Bryant J, Takeda A, Cooper K, Clegg A, Smith A, et al.

35. Fludarabine phosphate for the firstline treatment of chronic lymphocytic leukaemia.

    By Walker S, Palmer S, Erhorn S, Brent S, Dyker A, Ferrie L, et al.

36. Erlotinib for the treatment of relapsed non-small cell lung cancer.

    By McLeod C, Bagust A, Boland A, Hockenhull J, Dundar Y, Proudlove C, et al.

37. Cetuximab plus radiotherapy for the treatment of locally advanced squamous cell carcinoma of the head and neck.

    By Griffin S, Walker S, Sculpher M, White S, Erhorn S, Brent S, et al.

38. Infliximab for the treatment of adults with psoriasis.

    By Loveman E, Turner D, Hartwell D, Cooper K, Clegg A

39. Psychological interventions for postnatal depression: cluster randomised trial and economic evaluation. The PoNDER trial.

    By Morrell CJ, Warner R, Slade P, Dixon S, Walters S, Paley G, et al.

40. The effect of different treatment durations of clopidogrel in patients with non-ST-segment elevation acute coronary syndromes: a systematic review and value of information analysis.

    By Rogowski R, Burch J, Palmer S, Craigs C, Golder S, Woolacott N.

41. Systematic review and individual patient data meta-analysis of diagnosis of heart failure, with modelling of implications of different diagnostic strategies in primary care.

    By Mant J, Doust J, Roalfe A, Barton P, Cowie MR, Glasziou P, et al.

42. A multicentre randomised controlled trial of the use of continuous positive airway pressure and non-invasive positive pressure ventilation in the early treatment of patients presenting to the emergency department with severe acute cardiogenic pulmonary oedema: the 3CPO trial.

    By Gray AJ, Goodacre S, Newby DE, Masson MA, Sampson F, Dixon S, et al. , on behalf of the 3CPO study investigators.

43. Early high-dose lipid-lowering therapy to avoid cardiac events: a systematic review and economic evaluation.

    By Ara R, Pandor A, Stevens J, Rees A, Rafia R.

44. Adefovir dipivoxil and pegylated interferon alpha for the treatment of chronic hepatitis B: an updated systematic review and economic evaluation.

    By Jones J, Shepherd J, Baxter L, Gospodarevskaya E, Hartwell D, Harris P, et al.

45. Methods to identify postnatal depression in primary care: an integrated evidence synthesis and value of information analysis.

    By Hewitt CE, Gilbody SM, Brealey S, Paulden M, Palmer S, Mann R, et al.

46. A double-blind randomised placebo-controlled trial of topical intranasal corticosteroids in 4- to 11-year-old children with persistent bilateral otitis media with effusion in primary care.

    By Williamson I, Benge S, Barton S, Petrou S, Letley L, Fasey N, et al.

47. The effectiveness and cost-effectiveness of methods of storing donated kidneys from deceased donors: a systematic review and economic model.

    By Bond M, Pitt M, Akoh J, Moxham T, Hoyle M, Anderson R.

48. Rehabilitation of older patients: day hospital compared with rehabilitation at home. A randomised controlled trial.

    By Parker SG, Oliver P, Pennington M, Bond J, Jagger C, Enderby PM, et al.

49. Breastfeeding promotion for infants in neonatal units: a systematic review and economic analysis.

    By Renfrew MJ, Craig D, Dyson L, McCormick F, Rice S, King SE, et al.

50. The clinical effectiveness and cost-effectiveness of bariatric (weight loss) surgery for obesity: a systematic review and economic evaluation.

    By Picot J, Jones J, Colquitt JL, Gospodarevskaya E, Loveman E, Baxter L, et al.

51. Rapid testing for group B streptococcus during labour: a test accuracy study with evaluation of acceptability and cost-effectiveness.

    By Daniels J, Gray J, Pattison H, Roberts T, Edwards E, Milner P, et al.

52. Screening to prevent spontaneous preterm birth: systematic reviews of accuracy and effectiveness literature with economic modelling.

    By Honest H, Forbes CA, Durée KH, Norman G, Duffy SB, Tsourapas A, et al.

53. The effectiveness and cost-effectiveness of cochlear implants for severe to profound deafness in children and adults: a systematic review and economic model.

    By Bond M, Mealing S, Anderson R, Elston J, Weiner G, Taylor RS, et al.

54. Gemcitabine for the treatment of metastatic breast cancer.

    By Jones J, Takeda A, Tan SC, Cooper K, Loveman E, Clegg A.

55. Varenicline in the management of smoking cessation: a single technology appraisal.

    By Hind D, Tappenden P, Peters J, Kenjegalieva K.

56. Alteplase for the treatment of acute ischaemic stroke: a single technology appraisal.

    By Lloyd Jones M, Holmes M.

57. Rituximab for the treatment of rheumatoid arthritis.

    By Bagust A, Boland A, Hockenhull J, Fleeman N, Greenhalgh J, Dundar Y, et al.

58. Omalizumab for the treatment of severe persistent allergic asthma.

    By Jones J, Shepherd J, Hartwell D, Harris P, Cooper K, Takeda A, et al.

59. Rituximab for the treatment of relapsed or refractory stage III or IV follicular non-Hodgkin’s lymphoma.

    By Boland A, Bagust A, Hockenhull J, Davis H, Chu P, Dickson R.

60. Adalimumab for the treatment of psoriasis.

    By Turner D, Picot J, Cooper K, Loveman E.

61. Dabigatran etexilate for the prevention of venous thromboembolism in patients undergoing elective hip and knee surgery: a single technology appraisal.

    By Holmes M, C Carroll C, Papaioannou D.

62. Romiplostim for the treatment of chronic immune or idiopathic thrombocytopenic purpura: a single technology appraisal.

    By Mowatt G, Boachie C, Crowther M, Fraser C, Hernández R, Jia X, et al.

63. Sunitinib for the treatment of gastrointestinal stromal tumours: a critique of the submission from Pfizer.

    By Bond M, Hoyle M, Moxham T, Napier M, Anderson R.

64. Vitamin K to prevent fractures in older women: systematic review and economic evaluation.

    By Stevenson M, Lloyd-Jones M, Papaioannou D.

65. The effects of biofeedback for the treatment of essential hypertension: a systematic review.

    By Greenhalgh J, Dickson R, Dundar Y.

66. A randomised controlled trial of the use of aciclovir and/or prednisolone for the early treatment of Bell’s palsy: the BELLS study.

    By Sullivan FM, Swan IRC, Donnan PT, Morrison JM, Smith BH, McKinstry B, et al.

67. Lapatinib for the treatment of HER2-overexpressing breast cancer.

    By Jones J, Takeda A, Picot J, von Keyserlingk C, Clegg A.

68. Infliximab for the treatment of ulcerative colitis.

    By Hyde C, Bryan S, Juarez-Garcia A, Andronis L, Fry-Smith A.

69. Rimonabant for the treatment of overweight and obese people.

    By Burch J, McKenna C, Palmer S, Norman G, Glanville J, Sculpher M, et al.

70. Telbivudine for the treatment of chronic hepatitis B infection.

    By Hartwell D, Jones J, Harris P, Cooper K.

71. Entecavir for the treatment of chronic hepatitis B infection.

    By Shepherd J, Gospodarevskaya E, Frampton G, Cooper, K.

72. Febuxostat for the treatment of hyperuricaemia in people with gout: a single technology appraisal.

    By Stevenson M, Pandor A.

73. Rivaroxaban for the prevention of venous thromboembolism: a single technology appraisal.

    By Stevenson M, Scope A, Holmes M, Rees A, Kaltenthaler E.

74. Cetuximab for the treatment of recurrent and/or metastatic squamous cell carcinoma of the head and neck.

    By Greenhalgh J, Bagust A, Boland A, Fleeman N, McLeod C, Dundar Y, et al.

75. Mifamurtide for the treatment of osteosarcoma: a single technology appraisal.

    By Pandor A, Fitzgerald P, Stevenson M, Papaioannou D.

76. Ustekinumab for the treatment of moderate to severe psoriasis.

    By Gospodarevskaya E, Picot J, Cooper K, Loveman E, Takeda A.

77. Endovascular stents for abdominal aortic aneurysms: a systematic review and economic model.

    By Chambers D, Epstein D, Walker S, Fayter D, Paton F, Wright K, et al.

78. Clinical and cost-effectiveness of epoprostenol, iloprost, bosentan, sitaxentan and sildenafil for pulmonary arterial hypertension within their licensed indications: a systematic review and economic evaluation.

    By Chen Y-F, Jowett S, Barton P, Malottki K, Hyde C, Gibbs JSR, et al.

79. Cessation of attention deficit hyperactivity disorder drugs in the young (CADDY) – a pharmacoepidemiological and qualitative study.

    By Wong ICK, Asherson P, Bilbow A, Clifford S, Coghill D, R DeSoysa R, et al.

80. ARTISTIC: a randomised trial of human papillomavirus (HPV) testing in primary cervical screening.

    By Kitchener HC, Almonte M, Gilham C, Dowie R, Stoykova B, Sargent A, et al.

81. The clinical effectiveness of glucosamine and chondroitin supplements in slowing or arresting progression of osteoarthritis of the knee: a systematic review and economic evaluation.

    By Black C, Clar C, Henderson R, MacEachern C, McNamee P, Quayyum Z, et al.

82. Randomised preference trial of medical versus surgical termination of pregnancy less than 14 weeks’ gestation (TOPS).

    By Robson SC, Kelly T, Howel D, Deverill M, Hewison J, Lie MLS, et al.

83. Randomised controlled trial of the use of three dressing preparations in the management of chronic ulceration of the foot in diabetes.

    By Jeffcoate WJ, Price PE, Phillips CJ, Game FL, Mudge E, Davies S, et al.

84. VenUS II: a randomised controlled trial of larval therapy in the management of leg ulcers.

    By Dumville JC, Worthy G, Soares MO, Bland JM, Cullum N, Dowson C, et al.

85. A prospective randomised controlled trial and economic modelling of antimicrobial silver dressings versus non-adherent control dressings for venous leg ulcers: the VULCAN trial

    By Michaels JA, Campbell WB, King BM, MacIntyre J, Palfreyman SJ, Shackley P, et al.

86. Communication of carrier status information following universal newborn screening for sickle cell disorders and cystic fibrosis: qualitative study of experience and practice.

    By Kai J, Ulph F, Cullinan T, Qureshi N.

87. Antiviral drugs for the treatment of influenza: a systematic review and economic evaluation.

    By Burch J, Paulden M, Conti S, Stock C, Corbett M, Welton NJ, et al.

88. Development of a toolkit and glossary to aid in the adaptation of health technology assessment (HTA) reports for use in different contexts.

    By Chase D, Rosten C, Turner S, Hicks N, Milne R.

89. Colour vision testing for diabetic retinopathy: a systematic review of diagnostic accuracy and economic evaluation.

    By Rodgers M, Hodges R, Hawkins J, Hollingworth W, Duffy S, McKibbin M, et al.

90. Systematic review of the effectiveness and cost-effectiveness of weight management schemes for the under fives: a short report.

    By Bond M, Wyatt K, Lloyd J, Welch K, Taylor R.

91. Are adverse effects incorporated in economic models? An initial review of current practice.

    By Craig D, McDaid C, Fonseca T, Stock C, Duffy S, Woolacott N.

1. Multicentre randomised controlled trial examining the cost-effectiveness of contrast-enhanced high field magnetic resonance imaging in women with primary breast cancer scheduled for wide local excision (COMICE).

   By Turnbull LW, Brown SR, Olivier C, Harvey I, Brown J, Drew P, et al.

2. Bevacizumab, sorafenib tosylate, sunitinib and temsirolimus for renal cell carcinoma: a systematic review and economic evaluation.

   By Thompson Coon J, Hoyle M, Green C, Liu Z, Welch K, Moxham T, et al.

3. The clinical effectiveness and cost-effectiveness of testing for cytochrome P450 polymorphisms in patients with schizophrenia treated with antipsychotics: a systematic review and economic evaluation.

   By Fleeman N, McLeod C, Bagust A, Beale S, Boland A, Dundar Y, et al.

4. Systematic review of the clinical effectiveness and cost-effectiveness of photodynamic diagnosis and urine biomarkers (FISH, ImmunoCyt, NMP22) and cytology for the detection and follow-up of bladder cancer.

   By Mowatt G, Zhu S, Kilonzo M, Boachie C, Fraser C, Griffiths TRL, et al.

5. Effectiveness and cost-effectiveness of arthroscopic lavage in the treatment of osteoarthritis of the knee: a mixed methods study of the feasibility of conducting a surgical placebo-controlled trial (the KORAL study).

   By Campbell MK, Skea ZC, Sutherland AG, Cuthbertson BH, Entwistle VA, McDonald AM, et al.

6. A randomised 2 × 2 trial of community versus hospital pulmonary rehabilitation for chronic obstructive pulmonary disease followed by telephone or conventional follow-up.

   By Waterhouse JC, Walters SJ, Oluboyede Y, Lawson RA.

7. The effectiveness and cost-effectiveness of behavioural interventions for the prevention of sexually transmitted infections in young people aged 13–19: a systematic review and economic evaluation.

   By Shepherd J, Kavanagh J, Picot J, Cooper K, Harden A, Barnett-Page E, et al.

8. Dissemination and publication of research findings: an updated review of related biases.

   By Song F, Parekh S, Hooper L, Loke YK, Ryder J, Sutton AJ, et al.

9. The effectiveness and cost-effectiveness of biomarkers for the prioritisation of patients awaiting coronary revascularisation: a systematic review and decision model.

   By Hemingway H, Henriksson M, Chen R, Damant J, Fitzpatrick N, Abrams K, et al.

10. Comparison of case note review methods for evaluating quality and safety in health care.

    By Hutchinson A, Coster JE, Cooper KL, McIntosh A, Walters SJ, Bath PA, et al.

11. Clinical effectiveness and cost-effectiveness of continuous subcutaneous insulin infusion for diabetes: systematic review and economic evaluation.

    By Cummins E, Royle P, Snaith A, Greene A, Robertson L, McIntyre L, et al.

12. Self-monitoring of blood glucose in type 2 diabetes: systematic review.

    By Clar C, Barnard K, Cummins E, Royle P, Waugh N.

13. North of England and Scotland Study of Tonsillectomy and Adeno-tonsillectomy in Children (NESSTAC): a pragmatic randomised controlled trial with a parallel non-randomised preference study.

    By Lock C, Wilson J, Steen N, Eccles M, Mason H, Carrie S, et al.

14. Multicentre randomised controlled trial of the clinical and cost-effectiveness of a bypass-surgery-first versus a balloon-angioplasty-first revascularisation strategy for severe limb ischaemia due to infrainguinal disease. The Bypass versus Angioplasty in Severe Ischaemia of the Leg (BASIL) trial.

    By Bradbury AW, Adam DJ, Bell J, Forbes JF, Fowkes FGR, Gillespie I, et al.

15. A randomised controlled multicentre trial of treatments for adolescent anorexia nervosa including assessment of cost-effectiveness and patient acceptability – the TOuCAN trial.

    By Gowers SG, Clark AF, Roberts C, Byford S, Barrett B, Griffiths A, et al.

16. Randomised controlled trials for policy interventions: a review of reviews and meta-regression.

    By Oliver S, Bagnall AM, Thomas J, Shepherd J, Sowden A, White I, et al.

17. Paracetamol and selective and non-selective non-steroidal anti-inflammatory drugs (NSAIDs) for the reduction of morphine-related side effects after major surgery: a systematic review.

    By McDaid C, Maund E, Rice S, Wright K, Jenkins B, Woolacott N.

18. A systematic review of outcome measures used in forensic mental health research with consensus panel opinion.

    By Fitzpatrick R, Chambers J, Burns T, Doll H, Fazel S, Jenkinson C, et al.

19. The clinical effectiveness and cost-effectiveness of topotecan for small cell lung cancer: a systematic review and economic evaluation.

    By Loveman E, Jones J, Hartwell D, Bird A, Harris P, Welch K, et al.

20. Antenatal screening for haemoglobinopathies in primary care: a cohort study and cluster randomised trial to inform a simulation model. The Screening for Haemoglobinopathies in First Trimester (SHIFT) trial.

    By Dormandy E, Bryan S, Gulliford MC, Roberts T, Ades T, Calnan M, et al.

21. Early referral strategies for management of people with markers of renal disease: a systematic review of the evidence of clinical effectiveness, cost-effectiveness and economic analysis.

    By Black C, Sharma P, Scotland G, McCullough K, McGurn D, Robertson L, et al.

22. A randomised controlled trial of cognitive behaviour therapy and motivational interviewing for people with Type 1 diabetes mellitus with persistent sub-optimal glycaemic control: A Diabetes and Psychological Therapies (ADaPT) study.

    By Ismail K, Maissi E, Thomas S, Chalder T, Schmidt U, Bartlett J, et al.

23. A randomised controlled equivalence trial to determine the effectiveness and cost–utility of manual chest physiotherapy techniques in the management of exacerbations of chronic obstructive pulmonary disease (MATREX).

    By Cross J, Elender F, Barton G, Clark A, Shepstone L, Blyth A, et al.

24. A systematic review and economic evaluation of the clinical effectiveness and cost-effectiveness of aldosterone antagonists for postmyocardial infarction heart failure.

    By McKenna C, Burch J, Suekarran S, Walker S, Bakhai A, Witte K, et al.

25. Avoiding and identifying errors in health technology assessment models: qualitative study and methodological review.

    By Chilcott JB, Tappenden P, Rawdin A, Johnson M, Kaltenthaler E, Paisley S, et al.

26. BoTULS: a multicentre randomised controlled trial to evaluate the clinical effectiveness and cost-effectiveness of treating upper limb spasticity due to stroke with botulinum toxin type A.

    By Shaw L, Rodgers H, Price C, van Wijck F, Shackley P, Steen N, et al. , on behalf of the BoTULS investigators.

Health Technology Assessment programme

1. Director, NIHR HTA programme, Professor of Clinical Pharmacology, University of Liverpool

2. Director, Medical Care Research Unit, University of Sheffield