Use of generic and condition-specific measures of health related quality of life in NICE decision-making: systematic review, statistical modeling and survey

EQ-5D

The EQ-5D describes HRQL in terms of five dimensions: mobility, self-care, usual activities, pain/discomfort and anxiety/depression. 2 Each dimension is usually described in terms of three levels of severity, although a version with five levels has recently been published. 3 The health classification system for the three-level version describes 243 health states and a tariff of values for each health state is available for several countries, including the UK. The UK value set was obtained from valuations provided by 3395 members of the general population using the time trade-off (TTO) valuation method. 4,29

SF-6D

Derived from the SF-36 and Short Form questionnaire-12 dimensions (SF-12) health questionnaires, the SF-6D has six dimensions (physical functioning, role limitation, social functioning, bodily pain, mental health and vitality) and each dimension has four to six severity levels. 6,30 Any patient who completes the SF-36 or the SF-12 can be uniquely classified according to the SF-6D. The health classification system of SF-6D describes a total of 18,000 health states and a tariff of values for each health state is available for several countries, including the UK. The UK value set was obtained from valuations provided by 611 members of the general population using the standard gamble (SG) valuation method. 30

Health Utilities Index Mark 3

Health Utilities Index is a group of GPBMs for measuring comprehensive health status and HRQL, including Health Utilities Index Mark 1(HUI1), Health Utilities Index Mark 2 (HUI2) and HUI3. HUI3 has nine dimensions (vision, hearing, speech, ambulation/mobility, pain, dexterity, self-care, emotion and cognition) and each dimension has three to six levels. The health classification system of HUI3 describes almost a million unique health states and a tariff of values for each health state is available for Canada. The Canadian value set was obtained from valuations provided by 504 members of the general population using the visual analogue scale (VAS) and SG valuation methods. 31

Assessment of quality and relevance

For the review, of most importance was the relevance of the study in terms of the patient population and inclusion of evidence to establish the psychometric performance of the generic measures. Studies including a mixed population of patients (i.e. with various conditions) were only included if they reported health-related utility values or dimension responses for subgroups of patients with one of the four specific conditions being evaluated. Nevertheless, a judgment regarding the risk of bias for each study was determined by reviewing the methods of patient recruitment and noting any missing data reported (either study drop-outs or incomplete questionnaires). Studies were not required to be specifically designed to assess validity, responsiveness or reliability, provided that they reported data in sufficient detail to allow an assessment of these traits. The intention of the assessment of quality was not to exclude relevant studies, but to highlight any concerns about quality when findings were interpreted.

Assessment of reliability

The reliability of a measure is defined as its ability to reproduce results when measurements are repeated on an unchanged population,33 or the comparability of responses across different assessors (for example, patient and proxy report). Reliability can be measured by retesting and reporting either the correlation or difference between estimates. In some circumstances, no change in health status may be expected over time and, subsequently, the values obtained using the measures may be stable. These results were interpreted as evidence of the reliability and stability of instruments. Other assessments of reliability included assessments of inter-rater reliability based on a comparison of responses given by multiple people completing the questionnaire on the patients' behalf. When considering the results of inter-rater comparisons, it is important to note that all of the GPBMs have been designed for self-completion and to report self-assessed HRQL. Therefore, perfect agreement between the intended respondents and their proxies may not be expected. Finally studies reporting internal consistency were also included as assessed through multitrait analysis.

Assessment of construct validity

Validity is defined as how well an instrument measures what it was intended to measure. More specifically, for the GPBMs, whether the dimensions adequately cover the key determinants of health-related utility. Criterion validity is determined by comparing an instrument to an established gold standard; however, a gold standard with which to benchmark HRQL measures against does not exist. Therefore, it is necessary to assess the validity of measures of health-related utility using measures that have evidence of construct validity for that condition, which establishes if patterns in scores confirm constructs or hypotheses about expected patterns.

We assessed the construct validity of the GPBMs using the ‘known-group’ method. The known-group method compares the values obtained from the GPBMs between groups of patients who are expected to differ [qualitatively or statistically using t-test or analysis of variance (ANOVA)] in the construct measured by the indicator used to define the groups. The known groups in this context are often defined according to clinical severity using other measures. It should be noted that the usefulness of these comparisons can be limited by sample size, particularly as studies are usually not powered to detect differences according to preference-based measures. In addition, consideration must be given to the appropriateness of the clinical measure and the groups defined by it, and exogenous factors that may influence HRQL. For instance, groups defined solely by the presence of a biomarker may have no impact on HRQL. If patients have a number of comorbidities, then these may have a greater impact on HRQL than the condition of interest. Known groups can also be defined using a case–control analysis in which comparison is between patient and general public population, or defined on the basis of other aspects such as age, gender or countries. However, a more stringent test is to define known groups based on different levels of condition severity (for example, by using a clinical indicator).

We also examined convergent validity, which is a type of construct validity. Convergent validity is defined as the extent to which one measure correlates with another measure of the same or similar concept. In this review, we examined the extent to which the EQ-5D, SF-6D or HUI3 correlate with other measures of QoL or clinical severity. Correlation was defined as ‘low’ if correlation coefficient was < 0.3, ‘moderate’ if between 0.3 and 0.5 and ‘strong’ if > 0.5. Correlations need to be interpreted with caution as it is not always clear how strong the relationship between the generic and condition-specific indicators should be. Furthermore, we interpreted estimation of regression between GPBMs and other measures as another indication of a correlation, focusing on whether some measures were significant predictors of others.

Assessment of responsiveness

Responsiveness assesses the ability of an instrument to measure a change in health-related utility over time. As with construct validity, the measurement of responsiveness is difficult as there is no gold standard measure with which to compare. Nevertheless, we assessed the responsiveness of health-related utility measures by comparing change in health-related utility measured over a period of time in which health status is expected to change (e.g. before and after an intervention) with the change demonstrated by another measure of health. For inclusion in the assessment of responsiveness, the comparator measure must have demonstrated a change in health. We did not review data from studies outside of the review relating to responsiveness of the comparator measures. Good evidence of responsiveness is considered where the GPBM shows statistically significant change in health (e.g. t-test) shown by other measures or clinical indicators. Weaker evidence of responsiveness is considered where the same trend of change is shown but the change is not statistically significant. When responsiveness indices for estimates of health-related utility are reported [e.g. effect size (ES) or standard response mean], they were compared with other measures. ES is the mean change in score of a measure between two different time points divided by the SD of the score at baseline. Standardised response mean is the mean change score of a measure between two different time points divided by the SD of the change score. As for the tests of validity, it is important to consider whether the measures of health change that are being used to assess responsiveness are valid. In addition, it is important to consider whether other health changes not directly related to the condition could have impacted upon health-related utility (e.g. side effects of treatment).

Presentation and analysis

Data for each of the four conditions are presented separately. Information on the study design, participant characteristics and the measures included are reported. Within each of the broadly defined conditions, there is a range of underlying aetiologies with different symptoms. The results for visual disorders, skin diseases and cancers are therefore presented for subgroups defined according to type of condition. Subgroups are not presented for hearing impairments as the studies were mainly defined according to the presence or absence of hearing loss and/or extent of hearing loss. For each condition, a summary table is presented which reports an overview of the conclusions drawn from each paper for each of the types of assessment.

Search results: vision

Bibliographic searching was completed in August 2010 and total of 1025 potentially relevant papers were identified. Abstracts and titles for all papers were screened to identify papers meeting the inclusion criteria; 969 records were excluded and full papers were ordered for the remaining 56 records. After reviewing the full papers, 25 were excluded and a total of 31 papers were included in the review. A flow chart of the study selection process is shown in Figure 1 .

FIGURE 1.

Flow diagram showing selection of studies: vision.

Quality assessment: vision

A range of recruitment procedures were reported. Some were retrospective analyses of data sets with predetermined inclusion criteria,34–36 some were case–control analyses,37–39 and the majority were cross-sectional observational studies. 22,34,36,40–52 The only randomised controlled trial (RCT) had well-defined inclusion criteria. 53 Response rates for questionnaires ranged from 33% to 96%, with completion rates of longitudinal studies > 85% in all but one study50 (range 52–98%). No study was excluded after the assessment of quality.

Study design and patients' characteristics: vision

Summary characteristics of the 31 studies are presented in Table 2 . Thirty of the 31 studies were observational studies22,34,36–39,40–52,54–64 and the remaining study was a RCT. 53 The studies were conducted in different countries including the UK, the USA and Canada and some were multicountry studies. The studies identified included a wide range of visual disorders. Five studies were in patients with glaucoma,34,44–46,54 seven studies were in patients with age-related macular degeneration (AMD),22,43,47–49,55,56 eight studies included patients with cataracts,36–39,53,57–59 two studies were on patients with diabetic retinopathy,42,50 three were on patients with conjunctivitis51,60,61 and the remaining studies included people with various other visual conditions. 40,41,52,62,63,64

The inclusion criteria varied across the studies reviewed within each of the specific conditions. Some studies reported that patients were identified through case notes, but no more details are provided. It was noted whether AMD was bilateral or unilateral and wet or dry, whether cataracts were present in the first or second eye and whether glaucoma was primary or multiple. Sample sizes also varied across studies, ranging from 4962 to 961. 63 One study40 included children with a mean age of 6 years and used HUI3. The authors reported that the HUI system had been used in a previous study of young children with a range of impairments similar to those included in their study, although it should be noted that this did not refer specifically to the HUI3 at that time. All other studies included adult patients and the AMD studies included patients over 70 years.

Measures used in studies: vision

Table 3 summarises the measures that have been used in the 31 studies included in the review. For the three GPBMs of interest, the EQ-5D was reported in 27 studies22,37–39,41–63 and therefore was the most commonly utility measure, six studies reported the HUI322,34,36,40,42,64 and only one study reported the SF-6D. 22 Ten studies also reported direct valuations of patients' own health states using methods such as the TTO or VAS. 22,44,45,51,58–63 Twenty-three studies reported visual acuity (VA)22,34,37–39,41–50,52–55,58,61,63,64 to indicate visual severity. In addition, various patient-reported visual-specific QoL measures were used.

Reliability: vision

No tests of reliability were performed on the generic preference-based measures.

Known-group analysis and convergent validity: vision

Known-group analysis was performed in 24 studies:22,34,36–45,47–51,54,56,60–64 20 for EQ-5D,22,37–39,41–45,47–51,54,56,60–63 five for HUI3,34,36,40,42,64 but no studies for SF-6D. In six of the studies, groups were defined by VA, or by contrast sensitivity, and mean estimates of utility for each defined group were provided. 22,41–43,54,61 The remaining 25 studies had either a case–control design, had different conditions or did not define levels of severity.

Nine of the 31 studies reviewed provide evidence on correlation or regression between GPBMs with either each other or with visual measures. 22,37,38,44,46,50,52–54 Eight studies report evidence of convergent validity in EQ-5D compared with a visual measure,37,38,44,46,50,52–54 with Espallargues et al. 22 also reporting correlations across EQ-5D, SF-6D and HUI3. Details of the data are summarised in Appendix 3 and below by type of vision disorder.

Responsiveness

Only three studies reported responsiveness of the utility measures in visual disorders ( Appendix 4 ). 55,57,58 Kim et al. 55 reported a statistically significant improvement in both the Visual Function Questionnaire (4 dimension) (VF-4D) and the EQ-5D after photodynamic therapy in patients with AMD. Black et al. 57 reported a statistically significant improvement in both the Visual Function Questionnaire (14 item) (VF-14) and the EQ-5D postcataract surgery, although the latter was relatively small. Conner-Spady et al. 58 reported a statistically significant improvement in the Visual Function Assessment (VFA) and VA post cataract surgery, but the subsequent mean improvements in EQ-VAS and EQ-5D were small and not statistically significant. This may suggest that the EQ-5D is not responsive in this population; however, it should be recognised that the study was not initially powered to identify statistically significant changes and a mean improvement was identified. In addition, the VAS did not change from pre to post treatment; therefore, the treatment may not significantly impact on HRQL.

Summary of results for visual review

The 31 studies included in this review show a worsening of utility values as visual impairment increased in many though not all studies. The magnitude and statistical significance of the association varied between different GPBMs of HRQL. Table 4 shows an overview of performance of utility measures in visual impairment.

The largest amount of evidence was found for the EQ-5D compared with the other generic measures and the results were mixed. Nearly all studies showed significant differences between patients with the condition and a control group. Studies comparing EQ-5D scores across severity groups were more mixed, with the majority of studies showing little or no difference between groups defined by clinical measures of visual impairment. No studies allowed an assessment of reliability for any of the measures. There were just three studies on responsiveness. and all were in the form of before-and-after studies of an intervention. 55,57,58 These identified changes consistent with an effective intervention, but differences were statistically significant in only two of three studies. 55,57 The assessment of convergent validity was also concerning, with half of the studies not demonstrating a statistically significant correlation with clinical measures. While there was less evidence for the HUI3, all but one study42 demonstrated good validity; no studies assessed responsiveness. There was very limited evidence on the SF-6D in patients with visual impairment.

Search results: hearing impairment

Bibliographic searching was completed in July 2010. The search strategy identified 119 articles. After reviewing titles and abstracts, 70 papers were excluded. Forty-nine papers were reviewed in full, and a further 31 were excluded and 18 papers were included in the final review. A flow chart of the study selection process is shown in Figure 2 .

FIGURE 2.

Flow diagram showing selection of studies: hearing impairment.

Quality assessment: hearing impairment

A range of study designs was reported in the studies included in the review. Three studies were cross-sectional65–67 but the majority were prospective or retrospective before-and-after studies. 21,23,68–78 Studies had well-defined inclusion/exclusion criteria for recruitment. For longitudinal studies, no study had extremely high levels of missing data and completion rates for patients in studies ranged from 60%68 to 100%. 66 The completion rates for the instruments included were usually high, ranging from 71%67 to 97%. 23 The reporting in these papers was reasonably clear. After quality assessment, no studies were excluded from the review.

Study characteristics: hearing impairment

The main characteristics of the 18 papers included in this review are shown in Table 5 . The two papers by Joore et al. 71,72 and the two papers by Joore73,74 reported the results of one specific study and, similarly, the two papers by Vuorialho et al. 77,78 reported a single study. In total, 14 separate studies were included in the review. The studies were undertaken in a range of countries, including the UK, the Netherlands, the USA, Canada and Finland. Some studies recruited patients with specific hearing problems, e.g. large vestibular aqueduct syndrome,23 but most were for defined the sample using clinical indicators such as the better ear unaided pure-tone average (PTA). As shown in Table 5 , the level of hearing loss varied between studies.

The sample sizes of the studies reviewed ranged from 2068 to 3272. 65 Most studies had approximately 100 participants, but two studies only had approximately 20 participants. 68,79 Five studies included young children with hearing impairments (the mean age of the samples ranged from 7 to 9 years),66–68,76,80 and the remaining studies included adults, with most focusing on older adults over 60 years of age. The studies involving children used parents or caregivers as proxies to assess HRQL of children.

Measures: hearing impairment

Table 6 summarises the measures used in the 18 papers. 21,23,65–80 Eleven papers reported EQ-5D,21,23,70–74,76–79 10 reported HUI321,23,65–69,75,79,80 and one used the SF-6D21 (alongside EQ-5D and HUI3). Among those studies that used EQ-5D, most reported the EQ-5D index based on the tariff of UK population values. In two cases, it was unclear which tariff of population values had been used. 71,77 Three papers also reported responses on the EQ-5D dimensions alongside the utility values. 72–74 A total of 11 papers reported patients' rating of own health using VAS66,70–74,76–80 and two used TTO methods. 79,80 A total of seven studies employed self-reported hearing-specific HRQL measures66,69–71,74,77,78 and seven studies reported clinical indicators to indicate severity of hearing impairment,23,65,67–69,75,77 including PTA for the best or worst ear without hearing aid and speech identification tests.

Reliability: hearing impairment

The review found little evidence on the reliability assessments of EQ-5D, HUI3 and SF-6D in hearing impairment. No papers reported test–retest experiments. Although not specifically for test–retest reliability purposes, one study71 reported EQ-5D responses and VAS indices at baseline and asked respondents to recall them 3 months after a hearing aid fitting. The authors did not find any significant difference between the baseline assessment and the recalled assessment of baseline health for EQ-5D.

Known-group analysis and convergent validity

Out of the 18 papers included in the review, seven papers provided information to enable an assessment of the validity of EQ-5D, HUI3 or SF-6D,23,65–68,75,76 although most studies were not designed to examine the validity of these measures. 23,65–68,75,76 The results are summarised in Appendix 5 .

Responsiveness

Twelve papers21,23,66,69–72,74,77–80 involved a total of nine studies that provided adequate information to allow an assessment of responsiveness of EQ-5D, HUI3 and/or SF-6D (see Appendix 6 ).

Summary and conclusion

Overall, the HUI3 was the most commonly used measure in the studies. In all six cases,23,65–68,75 the HUI3 detected a difference between groups defined by their severity of hearing impairment and four23,68,78,79 out of five23,66,69,79,80 cases detected statistically significant changes as a result of intervention ( Table 7 ). Differences picked up by the HUI3 were driven by the hearing dimensions and, in some cases, the speech and emotion dimensions. On the other hand, the findings of the review suggested relatively poor responsiveness of EQ-5D in this condition as, in five23,70–72,74,77,78 out of six cases,23,70–72,74,77–79 EQ-5D failed to detect change. The studies that allowed an assessment of known groups using the EQ-5D suggested it had only weak ability to discriminate difference between severity groups. Only one study involved the SF-6D; thus, the information is too limited to conclude on its performance. 21 No studies allowed an assessment of reliability to be made.

Search results: skin conditions

The bibliographic search was completed in September 2010. The search of electronic databases identified 161 records and two additional records were identified from the EuroQol Group website database. After reviewing titles and abstracts, 122 records were excluded. Forty-one papers were reviewed in full: a further 25 papers were excluded and 16 papers were included in the final review ( Figure 3 ).

FIGURE 3.

Flow diagram showing selection of studies for skin review.

Quality assessment: skin conditions

Three types of study designs were observed in the review. Eleven studies were RCTs,81–91 four studies were cross-sectional92–95 and one was an uncontrolled before-and-after study. 96 The majority of studies provided clear inclusion and exclusion criteria, but two did not. 81,82 Six papers did not report completion rates82,83,87,88,92,96 and, among the 10 studies reporting this information, completion rates were reasonable or high (ranging from 70%84 to 97%). 94 The completion rates for specific measures (e.g. item non-response) were generally high (above 90%). 82,91,92,95 No study was excluded after the assessment of quality.

Study design and patients' characteristics: skin conditions

The main characteristics of the 16 papers included in this review are shown in Table 8 . 81–96 Studies were conducted in various European and American countries, with several multinational studies. All but four studies recruited patients with psoriasis or psoriatic arthritis;82–88,92–96 the remaining studies recruited patients with acne,81 eczema,90 hidradenitis suppurativa89 or venous leg ulcers. 91 All studies included adults (mean age approximately 43 years), and male respondents accounted for 24–71% of the samples. Sample sizes ranged from 3291 to 27,994,95 with most studies including between 100 and 200 participants.

Measures used in studies: skin diseases

Table 9 summarises the measures that have been used in the 16 studies included in the review. Of the three GPBMs of interest, only those studies reporting EQ-5D were identified and included in the review. No studies reported data from SF-6D or HUI3. Fourteen studies also reported patients' valuation of their own health states using VAS. 81,82,84–89,90–92,94–96 Clinical indices were reported in studies to indicate severity of skin problems, including the Psoriasis Area Severity Index (PASI) by eight studies,85–88,92,94–96 Nail Psoriasis Severity Index (NAPSI) by one study,96 and the Acne Grade by one study. 81 Various generic measures [e.g. SF-36, Health Assessment Questionnaire – Disability Index (HAQ-DI), Health Assessment Questionnaire (HAQ)], skin-specific HRQL measures [e.g. Dermatology Life Quality Index (DLQI)], or symptom-specific HRQL measures (e.g. HADS, the Depression Inventory) were included in the studies (see Table 9 ).

Reliability: skin conditions

No study reported data on reliability of the three GPBMs.

Known-group analysis and convergent validity: skin conditions

Thirteen studies of patients with skin conditions provided sufficient evidence to allow assessment of known-group analysis and convergent validity of EQ-5D81–85,88–93,95,96 including: 12 known-group analyses82–86,88–93,96 and seven convergent validity analyses. 83,87,89–92,95 A summary of the findings is presented below. See Appendix 7 for details.

Responsiveness: skin conditions

A total of 10 studies provided evidence to allow assessment of responsiveness of EQ-5D in skin diseases. 81,82,84–86,88,91,94–96 Among them, eight studies included people with psoriasis or psoriatic arthritis,82,84–86,88,94–96 one study included people with acne81 and one study focused on venous leg ulcers. 91 Ten studies examined changes of scores over time or after treatment,81,82,84–86,88,91,94–96 and two provided details of ES or standard response mean estimation. 81,91 One study checked the correlation between change scores of health measures with changes in clinical measures95 (see Appendix 8 ).

Summary and conclusion: skin conditions

The overall performance of EQ-5D among skin diseases is summarised in Table 10 . Overall, there was evidence to suggest that EQ-5D is appropriate in terms of construct and convergent validity, as well as responsiveness in some skin conditions. All six studies showed that EQ-5D was able to reflect differences between severity groups84,89–91,93,96 and only one was not significant. 91 EQ-5D was shown to be able to significantly differentiate between patient and general populations in four case–control studies85,86–88 (one study did not report statistical tests),82 as well as groups defined by other aspects rather than severity. Moderate to strong correlations were found between EQ-5D and other measures. Nine81,82,84–86,88,94–96 out of 10 studies81,82,84–86,88,91,94–96 demonstrated that EQ-5D was able to detect change appropriately over time. Among these, only one study did not demonstrate a statistically significant difference. 82 ‘Skin conditions’ were defined in very broad terms for the purpose of the review and incorporate a range of conditions, each of which can affect different aspects of patients' QoL. Most of the studies identified were conducted for patients with psoriasis or psoriatic arthritis. Evidence was limited or unavailable for other skin conditions; however, the limited data available were generally positive. No studies reported evidence for HUI3 and SF-6D and no studies allowed an assessment of reliability for any of the measures.

Search results: cancer

Bibliographic searching was completed in August 2010. A total of 5223 potentially relevant papers were identified. Overall, a total of 5000 papers were excluded following screening of title and abstract. Full papers were reviewed for the remaining 223 records which met the inclusion criteria. After reviewing the full papers, 125 were excluded and a total of 98 papers were included in the review. A flow chart of the study selection process is shown in Figure 4 .

FIGURE 4.

Flow diagram showing selection of studies: cancer.

The 98 papers were grouped according to 20 different types of cancers. These included 18 papers on non-specific cancers,97–114 11 each for colon cancer115–125 and cancer survivors,126–136 10 for breast cancer,137–146 eight for gastric cancer147–154 and seven for prostate cancer,155–161 and a small number of papers for brain,162,163 cervical,164–167 kidney,168–171 lung103,172,173 and other cancers101,124,174–188 ( Table 11 gives details). As different cancers affect HRQL in different ways, the following sections present data according to the different types of cancer.

Quality assessment: cancer

A range of study designs were observed in the review. Some were cross-sectional studies,13,97–99,102,103,106–108,114,115,121,126,128–130,133,137,138,147,148,151,152,156,161,163,164,173,183,184,189 others were before-and-after studies110,112,116,117,120,123,139,140,145,190 or cohort studies100,141,142,155,157,158,160,162,191 and many were RCTs. 118,119,122,125,132,136,143,144,146,149,150,153,154,159,161,165,166,168–171,176,177,180–182,188,192,193 Most RCTs had clear inclusion and exclusion criteria and appropriate and explicit methods of randomisation. In some studies, the inclusion criteria were not clearly reported, which occurred mainly for studies of non-specific cancers. 97–100

Response rates varied between studies. Completion rates for breast cancer studies ranged from 74%141 to 99%143 and for colon cancer ranged from 67%115 to 90%. 120 No study was excluded after the assessment of quality.

Study characteristics: cancer

General characteristics of the 98 studies are presented in Table 12 . These studies were divided into 20 subgroups according to different types of cancer. Thirty-three studies were cross-sectional analyses,97–99,102,103,106–108,114,115,121,126–130,133,137,138,147,148,151,152,156,161,163,164,173,181–184,189 24 were RCTs,118,119,122,125,143,144,146,149,150,153,154,159,165,166,168–171,176,177,180,188,192,193 24 were before-and-after or longitudinal studies99,101,105,109,111–113,116,117,120,123,125,140,145,154,162,167,172,174,178,179,185,187,190 and nine were cohort studies. 100,104,141,142,155,157,158,160,191 Most groups included a mixture of study designs, exceptions were kidney cancer168,169,170,171,193 and lymphoma177,188,192 which were all RCTs and both lung cancer studies103,173 had cross-sectional designs. The selected studies were conducted in different countries across Europe, Asia and North America and eight were multinational studies. 118,146,157,168–171,193 Various treatments were included in the studies including types of surgery,117,141 radiotherapy and chemotherapy,100,137,162,175 other medicines and supportive care interventions or referral. 118,143,165,166 Most studies included adults, but some were collected data from children using HUI including studies of brain cancer,163 Hodgkin's lymphoma,185,190 and a couple of the studies where recruitment was not limited to a specific type of cancer. 98,99,105,107

The inclusion criteria for recruiting patients varied across the studies reviewed and within each type of specific cancer. Some studies recruited patients according to specific stages of cancer patients, for example primary tumours,162 stage II and III breast cancer with poor prognosis,140 tumour stage I, II and III breast cancer144 and advanced colorectal cancer. 119 Some studies involved patients after screening, for example studies of screening for cervical cancer. 164–167 For these screening studies, some of the respondents would be asymptomatic and therefore the GPBMs and other measures may not be expected to reflect differences between patients with and without cancer. Sample size varied across studies, ranging from 18112 to 113,587. 99

Measures: cancer

Table 13 summarises the measures that have been used in the 98 studies included in the review. For the three GPBMs of interest, EQ-5D was the most commonly used and was reported by 71 studies. 97,98,100,101,103–107,110–123,128,129,137–140,145–154,143,144,156–160,164–173,175–177,179–184,186,188,192–194 Twenty-four studies reported HUI2/HUI399,108,109,126,127,130,131–136,141,142,155,161,162,163,174,178,185–187,190 and only three studies reported SF-6D. 98,147,156 Two studies101,143 used EQ-5D and HUI3 alongside other measures and another three studies98,147,156 use both EQ-5D and HUI3 alongside other cancer-specific measures. Fifty-eight studies also reported patients' ratings of their own health status using VAS97,98,100–104,106–109,111,112,114–118,120–123,129,137–140,142–145,148,149,151–155,159,164–166,168–172,175,177,179–181,183–185,190,193,194 and valuations of own health were reported in three studies using TTO101,138,174 and in one study using the SG method. 161 Five studies also reported generic measures SF-12 or SF-36. 120,141,164,173,184 A wide range of cancer-specific measures of health were used, including the most commonly used European Organization for Research and Treatment of Cancer Quality-of-life Questionnaire (EORTC QLQ) in 26 studies100,101,111,115–117,121,124,128,129,144,146,147,149–151,153,154,172,179,180,182,183,188,192,193 and the FACT in 13 studies. 102,103,105,106,114,118,120,123,129,143,171,176,190 A range of other measures were reported, including variations of the previously mentioned cancer-specific HRQL measures such as the EORTC QLQ-Core 38 (EORTC QLQ-C38), staging of cancer progression using various staging systems and other measures of symptoms or aspects of health such as the HADS (see Table 13 for details). Many studies used multiple measures and did not always give consistent results, which make conclusions regarding concordance with results from the GBPMs more difficult to interpret.

Reliability: cancer

Fourteen studies127,130,131,133,134,163,168,174,176–178,184,190,192 reported evidence to allow assessment of reliability of EQ-5D (five studies)168,176,177,184,192 and HUI3 (nine studies)127,130,131,133,134,163,174,178,190 in patients with cancer and results are summarised in Appendix 9 . Cella et al. 168 examined EQ-5D in patients with kidney/renal cancer in terms of stability across treatment groups and found that EQ-5D, FACT and VAS scores did not differ between the different country cohorts. This provided some evidence for the reliability of EQ-5D in multinational trials. Similarly, Hahn et al. ,176 van Agthoven et al. 177 and Witzens-Harig et al. 192 reported that no significant differences between the treatment groups were found for EQ-5D, as well as EORTC QLQ-C30, among patients with leukaemia and lymphoma. Two studies examined the internal consistency of EQ-5D and HUI3 for specific questions/dimensions and dimensions/overall scores within measures. 163,184 One study reported that internal consistency was high for EQ-5D (as was the SF-36)184 and another study reported consistency for most questions for HUI3. 163 Inter-rater reliability of HUI3 was reported in nine studies. 127,130,131,133,134,163,174,178,190 These studies reported completion of HUI3 by multiple respondents and all studies demonstrated high agreement between different raters' assessments of the dimensions of HUI3. Although the instruments are designed for self-completion by adults, the agreement between raters provides some limited evidence of reliability.

Known-group analysis and convergent validity: cancer

Overall, 77 studies97–109,114,115,117,118,120–123,126–138,141,143–153,156–159,161–173,175,176,178,179–185,187,188,190,193 out of 98 provided evidence to allow for known-group analysis and convergent validity. Known-group analysis was carried out in 54 studies,97–99,103–106,109,114,115,117–122,126,127,130–135,138,148–152,156–159,162–167,169,170,172,173,175,176,179–183,188,190,193 41 for EQ-5D,97,98,103–106,114,115,117–122,138,148–152,156–159,164–167,169,170,172,173,175,176,179–183,188,193 of which two also included the SF-6D98,156 and 13 included the HUI3. 99,109,126,127,130–135,162,163,190 In most studies, groups were defined by severity of cancer on the basis of a global heath scale,126,163,179 or disease status120,127,162,164,195 or by treatment. 18,121,128,148,157 Some studies had case–control design comparing between cancer patients and the general public. 97,117,129,149 Several studies defined groups on the basis of other characteristics such as age and smoking status175 and country. 130 The differences in the clinical definition of groups, conditions, characteristics of patients and study designs make it difficult to directly compare the utility values, or to conduct meta-analyses across studies.

Convergent validity testing was carried out in 30 studies, 20 for EQ-5D98,100–103,107,123,137,138,144–146,148,151,156,164,165,173,184,196 and 10 for HUI3101,108,136,141,143,155,162,178,186,187 and one for SF-6D. 156 In most cases, evidence on the correlation between generic measure of HRQL with either each other or with cancer-specific measures was reported. 138,141,143–145,162 Regressions between scores of different measures were reported by several studies. 102,122,146,147

Details of the assessments of construct validity of utility measure in different type of cancers are shown in Appendix 10 and below are briefly summarised by specific types of cancers. For some types of cancer, there were only limited studies (fewer than three) for assessment of validity. The findings of these are summarised under the heading of ‘other cancers’.

Responsiveness: cancer

A total of 39 out of 98 studies among cancer patients provided sufficient evidence to allow assessment of responsiveness for EQ-5D (31 studies),104,110–113,116,124,139,140,143,144,147,149,153,154,157–160,165,167–169,170,171,176,177,179,180,188,192 for HUI3 (six studies)101,141,142,174,178,186 and both EQ-5D and HUI3 (two studies). 143,160 Most studies reported mean change of scores over the study period. 116,122,123,143,149,153,154,167 Some studies compared scores or responses at baseline and follow-up. 104,110,119,139 Some studies also reported responsiveness indices including ES or standard response mean,141,169,171,185 or a correlation between changes of different measures. 143,144 Statistical tests such as the t-test, ANOVA and Mann–Whitney U-test were conducted by some, but not all, studies. The detailed results are summarised below according to type of cancer. As for validity, cancer types for which only three or fewer studies reporting responsiveness data were available are grouped as ‘other cancers’. See Appendix 11 for details.

Summary and conclusion: cancer

The overall performance of EQ-5D, HUI3 and SF-6D are summarised in Table 14 . Among the 98 studies included in this review, the EQ-5D97,98,100,101,103–107,110–123,128,129,137–140,143–154,156–160,164–173,175–177,179–184,186,188,192,194 was the most commonly used GPBM, whereas HUI399,108,109,126,127,130–136,141,142,155,161–163,174,178,185–187,190 was the second most widely used measure. Few studies reported evidence for SF-6D. 98,147,156