The CUPID trial: a randomised double blind placebo-controlled parallel group multi-centre trial of cannabinoids to slow progression in multiple sclerosis

Article history

This issue of Health Technology Assessment contains a project originally commissioned by the MRC but managed by the Efficacy and Mechanism Evaluation Programme. The EME programme was created as part of the National Institute for Health Research (NIHR) and the Medical Research Council (MRC) coordinated strategy for clinical trials. The EME programme is funded by the MRC and NIHR, with contributions from the CSO in Scotland and NISCHR in Wales and the HSC R&D, Public Health Agency in Northern Ireland. It is managed by the NIHR Evaluation, Trials and Studies Coordinating Centre (NETSCC) based at the University of Southampton.

The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The HTA editors and publisher have tried to ensure the accuracy of the authors’ report and would like to thank the reviewers for their constructive comments on the draft document. However, they do not accept liability for damages or losses arising from the material published in this report.

Declared competing interests of authors

John Zajicek reports grants and personal fees from the Medical Research Council, personal fees from Bayer Schering, personal fees from Institut für klinische Forschung, Berlin, grants from the Multiple Sclerosis Society and grants from the Multiple Sclerosis Trust outside the submitted work. David Miller reports grants from Multiple Sclerosis Society of Great Britain and Northern Ireland, grants from University College London/University College London Hospitals Biomedical Research Centre, during the conduct of the study; grants and other from Biogen Idec, grants and other from Novartis, grants and other from GlaxoSmithKline, grants from the National Institute for Health Research, grants from Genzyme, grants from the US National Multiple Sclerosis Society and the Multiple Sclerosis Society of Great Britain and Northern Ireland, other from Bayer Schering, other from Mitsubishi Pharma Ltd, other from Merck, other from Chugai and personal fees from McAlpines Multiple Sclerosis, 4th edition, outside the submitted work. David MacManus reports grants from Biogen Idec, grants from GlaxoSmithKline, grants from Apitope, grants from Novartis and grants from Richmond Pharma outside the submitted work.

Copyright statement

© Queen’s Printer and Controller of HMSO 2015. This work was produced by Ball et al. under the terms of a commissioning contract issued by the Secretary of State for Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK.

Structure of this report

The report begins with a summary of the background to MS and treatments for the disease, findings from our previous study testing symptomatic benefits from cannabinoid use in MS and a review of trials focusing on alteration of disease course in progressive MS. This first chapter concludes with a description of the research aims.

The report then describes the methods from the main study and MRI substudy, including study design, recruitment and randomisation, outcome assessment, sample size and pre-specified statistical analyses. This is followed by results from pre-specified statistical analyses of the primary and secondary outcomes in the main study and MRI substudy. Subsequent chapters cover further results from post-hoc exploratory analyses, Rasch measurement theory (RMT) analysis of rating scale data and economic evaluation.

The final chapters summarise the findings from the study, provide interpretation in the light of previous studies and discuss the strengths and limitations of the research, implications for health care and recommendations for future research.

Background and objectives

Research aims

• To assess the efficacy of treatment with Δ⁹-THC in slowing progressive MS over a follow-up period of 3 years.

• To assess the safety of Δ⁹-THC.

• To assess the use of patient-orientated outcomes [in particular the MS Impact Scale-29 version 2 (MSIS-29v2) and 20-point physical subscale (MSIS-29phys)], in order to improve the methodology by which clinical trials are conducted in progressive MS.

Study design

The CUPID study was a double-blind, randomised, placebo-controlled, multicentre, parallel-group trial incorporating a MRI substudy. The study was designed to assess the safety and effectiveness of Δ⁹-THC in slowing MS disease progression over a 3-year period.

Adults with progressive MS4 and an EDSS score of 4.0–6.5 (walking affected by disease but still able to walk at least 20 metres without rest, with aids if necessary) were randomised individually to receive either oral Δ⁹-THC or matching placebo, in a 2 : 1 ratio.

Setting

The study was conducted in 27 NHS study sites in England, Wales and Scotland, comprising 25 hospital neurology departments and two rehabilitation departments (see Appendix 1). Principal investigators (PIs) were consultant neurologists or consultants in rehabilitation medicine. The majority of study sites had expressed an interest in the study and provided feasibility data before the trial started, but two sites were recruited once the study was under way. Despite early identification of study sites, there was wide variability in the time taken to obtain NHS research and development (R&D) approval at each site, putting early pressure on the rate of participant recruitment.

Study approvals

Applications for approval to conduct the study were submitted to the South and West Devon Research Ethics Committee (REC) and the Medicine and Healthcare products Regulatory Agency (MHRA) in December 2005.

Research Ethics Committee approval was confirmed on 4 April 2006 following some minor clarifications to the protocol. The REC reference was 06/Q2103/1. A clinical trial authorisation was granted by the MHRA on 2 May 2006 following revision of the protocol to include an annual assessment of depression and to strengthen the contraceptive advice for participants and partners. The trial was assigned European Union Drug Regulating Authorities Clinical Trials (EudraCT) Number 2005–002728–33 and the International Standard Randomised Controlled Trial Number (ISRCTN) ISRCTN62942668. NHS R&D approvals were obtained from all participating sites (see Appendix 2).

Training

All research site staff including neurologists, MS specialist nurses, research nurses, physiotherapists, pharmacists and MRI staff were invited to attend one of a series of study-specific training days held across the UK before the start of the study. Training was provided on all aspects of the study protocol including eligibility, recruitment, consent, prescription and titration of study treatment, study visit and assessment schedule, study assessments, blinding, pharmacovigilance, data collection documentation, and laboratory and MRI procedures. Individual training was provided for sites joining the study at a later stage.

A short EDSS training film was produced specifically for the study and provided to sites in DVD format. All neurologists undertaking EDSS assessments for the study were required to document that they had viewed the EDSS training material prior to assessing study participants. A training DVD was also provided for those site staff conducting the MS Functional Composite (MSFC) assessments and standardised equipment for conducting the MSFC components [9-hole peg test (9-HPT), timed 25-foot walk (T25-FW) and Paced Auditory Serial Addition Test (PASAT)] was provided to all sites.

All study team members involved in the recruitment and consent process, data collection, prescription of trial treatment, adverse event (AE) reporting and participant assessment were required to undertake Good Clinical Practice (GCP) training and to keep this up to date throughout their involvement in the trial.

Participant eligibility

The study eligibility criteria were as broad and pragmatic as possible to maximise both recruitment and generalisability of results. As the aim of the study was to investigate the effectiveness of Δ⁹-THC in slowing MS progression, patients whose disease had been stable for 12 months or more were excluded from participation. In the absence of recent documented evidence of disease activity or status, the assessment of disease stability relied on clinical judgement in discussion with the patient and/or family member.

Factors potentially influencing baseline EDSS assessment (e.g. recent relapse or steroid therapy) were grounds for exclusion from the trial, but patients in this category could be rescreened for inclusion in the trial at a later date. It was also deemed necessary to exclude patients with baseline EDSS scores of > 6.5 [an EDSS score of 6.5 was defined as constant bilateral support (canes, crutches or braces) required to walk about 20 metres without resting] because of the relative difficulties of identifying further progression in this group.

Recruitment of participants

Patients were prospectively recruited to the study between May 2006 and July 2008. Potential participants were identified by a health-care professional at each study site, usually from an existing caseload of patients known to have a diagnosis of PPMS or SPMS. A small number of patients were referred to study site neurologists by colleagues at non-participating hospitals close to recruiting sites. A few patients self-referred as a result of publicity about the study in various local and national media or from information obtained via the internet.

Arrangements for inviting patients to participate in the study varied according to circumstances, local practice and available resources. As the study started before the inception of the Comprehensive Local Research Networks, it was adopted onto the UK Clinical Research Network Dementias and Neurodegenerative Diseases Research Network portfolio which allowed access to research support staff at a few study sites. Study participants were recruited either individually or in cohorts, depending on practical arrangements at each site. Some sites held dedicated research clinics and were thus able to follow-up several study participants in one session. In terms of grouping EDSS assessments and batching laboratory samples, this was generally an efficient model and was often well received by participants who had contact with one another as a result. Other sites were only able to recruit and follow-up participants on an individual basis. Monthly recruitment totals by site are shown in Appendix 3.

Initial identification of potential participants was predominantly by direct consultant referral to the study team or by screening of hospital records. Potentially eligible patients were then contacted individually – usually by a research nurse or MS specialist nurse on behalf of the PI – to discuss the study, ascertain interest and check major eligibility criteria. Group information sessions were held at some sites for interested patients and their families. Regardless of the mode of contact, all potentially eligible patients were provided with a participant information sheet (see Appendix 4) for the study. Apparently eligible patients who were interested in taking part in the study were subsequently invited to attend a screening visit (visit 1).

Each study site kept a record of the number of patients invited for screening and the number of ineligible, eligible, non-consenting, consenting, randomised and non-randomised patients was monitored by the co-ordinating team. Throughout the study, the number of participants who discontinued trial medication prematurely or were lost to follow-up was recorded, with reasons where known.

Study site personnel

Each study site was required to nominate a treating physician and a separate assessing physician for the study, with appropriate arrangements for cover in case of staff absence. Treating physicians were responsible for assessing patient eligibility, obtaining informed consent, prescription and titration of trial treatment, reviewing participant progress and monitoring and recording AEs and concomitant medications. Assessing physicians conducted the EDSS assessments. Other study assessments (i.e. the MSFC and RMI) could be conducted by the assessing physician or a non-clinician with appropriate training. To reduce bias from potential unblinding, clinicians were advised not to change their role during the study, particularly from treating physician to assessor; this was monitored by the study co-ordinating team.

Screening visit (consent and entry to trial)

Patients attending the screening visit were given the opportunity to discuss the study and have any questions answered. The main study inclusion and exclusion criteria were checked by the treating physician (including EDSS score, assessed by the assessing clinician). Patients who were eligible and willing to participate gave written consent for the trial, including consent for their name and address to be held by the study co-ordinating centre for central management of postal and web-based study questionnaires.

All patients were required to provide blood samples for pre-trial biochemical and haematological analysis and a urine sample for baseline cannabinoid testing. Demographic data, medical history, concomitant medications and vital signs were recorded at this visit. The RMI and a MSFC practice run were completed.

Inclusion in the study was confirmed within 2 weeks of the screening visit, following results of the laboratory tests. Participants’ general practitioners (GPs) were notified of patients’ involvement in the trial once enrolment had been confirmed.

Study diary

Each participant was given a study diary which provided space for the participant to record any side effects or other relevant information to aid recall at subsequent study visits. The diary also contained advice on when/how to take the trial medication, instructions on weekly dosage, research team contact details and space for study appointment details. The diary contents were not used as part of the study data set.

Randomisation and masking

Once eligibility had been confirmed following receipt of laboratory results, participants were assigned via a secure computer-generated randomisation system to receive oral Δ⁹-THC (dronabinol) or matched placebo capsules for 36 months in a 2 : 1 active-to-placebo ratio. The allocation ratio was chosen in order to increase the acceptability of the trial to patients, with the aim of maximising recruitment and retention rates. The randomisation allocation sequence was generated by an independent statistician using a stochastic minimisation algorithm and balanced according to EDSS score, study site and disease type (PPMS or SPMS) with a random component. During the study, the independent statistician made periodic checks of the allocation ratios according to the minimisation factors, to ensure that the randomisation schedule was performing as intended.

To ensure allocation concealment, treatment assignment was undertaken by the co-ordinating pharmacy based at the lead hospital site, independently of the research team. The randomisation list was stored securely within the co-ordinating pharmacy and written procedures were in place in the event of a request for emergency unblinding, either for clinical reasons or to facilitate monitoring of unexpected serious adverse reactions.

Participants and all other personnel directly involved in the study were blinded to treatment allocation. The discussion of symptoms and/or side effects between participants and assessing physicians was actively discouraged, although assessing physicians occasionally had to review study patients outside the context of the study, for routine clinical follow-up or during episodes of hospitalisation. On completion of the study, each participant and the treating and assessing physicians were asked which treatment they thought the participant had been allocated.

Trial interventions

Trial medication was supplied in bottles by Insys Therapeutics, Inc. (Chandler, AZ, USA), labelled in accordance with relevant clinical trial guidelines. Active treatment (oral Δ⁹-THC, dronabinol) consisted of hard gelatin capsules each containing 3.5 mg of Δ⁹-THC. Placebo treatment consisted of identically matched sesame oil capsules such that dronabinol and placebo capsules were indistinguishable.

Trial medication was originally supplied for storage at room temperature, with a 1-year expiry. Despite the logistics of scheduling resupply at regular intervals, there were clear advantages to both participants and study sites in providing trial medication with room temperature stability. However, economic pressures on the suppliers during the course of the trial led to the introduction of refrigerated medication in order to increase shelf life and reduce wastage.

Medication was distributed to sites on an individual participant basis by the co-ordinating pharmacy. Initial supplies were provided on a weight-related basis following randomisation. Once participants were established on a steady dose, subsequent resupplies were provided on a dose-related basis.

Outcome assessments

The primary clinician-based outcome was time to first EDSS score progression. This was defined as an increase of at least 1 point from a baseline EDSS score of 4.0–5.0, or at least 0.5 points from a baseline EDSS score of 5.5–6.5, confirmed at the next scheduled 6-monthly visit. The primary patient-based outcome measure was change in MSIS-29phys score.

Secondary outcomes included the number and nature of AEs, MS Walking Scale-12 (MSWS-12v2) score, MSFC score, RMI score, Short Form questionnaire-36 items version 2 (SF-36v2) score, European Quality of Life-5 Dimensions (EQ-5D) questionnaire score, MS Spasticity Scale-88 (MSSS-88) score and a category rating scale. In addition, for a subgroup of patients allocated to the MRI substudy, outcomes included brain atrophy, in terms of annual percentage brain volume change (PBVC) and occurrence of new or newly enlarging T2 and T1 lesions from annual cranial MRI.

The EDSS was assessed at each site by a neurologist, following study-specific training. The MSFC assessment was performed by a non-physician, as long as training had been given. The RMI was similarly assessed every 6 months.

Participant-completed questionnaires

Between the screening and baseline visits, participants were asked to complete a questionnaire booklet comprising the MSIS-29v2, MSWS-12v2, SF-36v2, EQ-5D, MSSS-88 and category rating scale. The questionnaire booklet was repeated in its entirety at 3, 12, 24 and 36 months and (excluding the MSSS-88 and category rating scale) at 6, 18 and 30 months.

The baseline questionnaire was sent to all participants by post from the co-ordinating centre with a pre-paid reply envelope. Thereafter, participants could choose to complete subsequent questionnaires by post or online via a secure web-based system. In the event of non-receipt of completed questionnaires, repeat copies were sent, with e-mail reminders to those participants opting for online completion.

Participants were also asked to complete the Beck Depression Inventory-II (BDI-II) at baseline and annually thereafter. As the BDI-II was included as a safety measure rather than an outcome assessment, total scores were reported directly to PIs as they were provided.

Baseline visit (provision of trial medication)

The baseline visit (visit 2) was held approximately 2–4 weeks after the screening visit, following confirmation of eligibility and randomisation. Vital signs and AEs were recorded at this visit and the baseline MSFC assessment was completed. A prescription for trial medication was provided, with a starting dose of one capsule (3.5 mg of Δ⁹-THC equivalent) twice daily for all participants.

Owing to the large interindividual dose variation with oral cannabinoids, there was a 4-week titration period in which participants in both study arms could increase their dose by one capsule twice daily at weekly intervals, to a maximum weight-related dose (Table 1). Dose progression depended on adverse effects. If unwanted side effects developed, participants were advised not to increase the dose. If these side effects were considered intolerable, the dose was reduced.

Subsequent participant follow-up

Participants were scheduled to attend follow-up visits with the treating physician at 2 and 4 weeks after the baseline visit, for AE monitoring and dose adjustment. Once participants had been settled on a suitable treatment dose, clinician-based outcome data were collected at assessment visits at 3 and 6 months, then 6-monthly up to 36 months. Participants who demonstrated new EDSS score progression according to the defined study end point at 36 months attended a further visit at 42 months to confirm EDSS score progression. Monitoring of AEs, recording of vital signs and collection of safety blood samples were undertaken at the clinic visits. Urine samples for cannabinoid analysis were requested from each participant approximately four times during the study according to a random schedule, with the aim of detecting illicit cannabis use in the placebo group. Study medication was gradually discontinued over a period of a few weeks from the final visit (36 or 42 months depending on EDSS status). Table 2 shows the CUPID study visit and questionnaire completion schedule.

Various strategies were implemented during the course of the study to improve follow-up and data collection. It was identified that the completion rate for online questionnaires was lower than for postal questionnaires, despite e-mail reminders. Recognising that study participants may access their e-mail sporadically and, therefore, may miss electronic reminders, the co-ordinating centre added postal reminders, leading to an improvement in online completion rates.

The number and length of PROs appeared burdensome for a number of participants, including some who discontinued study medication prematurely but agreed to remain in follow-up. At the discretion of the co-ordinating centre, the questionnaire battery was reduced for some individual participants in order to secure at least some outcome data which otherwise were at risk of being lost completely. In these circumstances, priority was given to capturing the patient-reported primary outcome data (MSIS-29v2) at the expense of other measures, if necessary.

For participants in whom visual or motor/sensory impairment made questionnaire completion difficult, telephone support was provided by the co-ordinating centre to supply responses. Some participants with accumulating disability found it increasingly troublesome or tiring to attend clinic visits for EDSS assessment. Participants in this category who had also stopped medication prematurely were particularly at risk of being lost to follow-up since they had no need to attend in person for repeat supply of medication. To improve data completeness, the option of validated telephone EDSS assessment for participants unable or unwilling to attend clinic visits was introduced from June 2010. All telephone EDSS assessments were conducted by a trained clinician from the Plymouth co-ordinating team, following formal consent from participants.

Safety monitoring

In addition to the annual assessment of depression (BDI-II), safety monitoring included standard clinical laboratory assessments (chemistry, haematology, liver function) at 3 and 6 months and 6-monthly thereafter. Blood samples were analysed by a central laboratory and results reported to the PI by the co-ordinating centre.

Assessment of AEs was undertaken by the investigator at each study visit, with particular emphasis on the titration phase. If participants could not attend either the 2- or 4-week visit, site teams were encouraged to review AEs by telephone in order to optimise the individual dose of study treatment for each participant. Signs and symptoms were graded as mild, moderate or severe. Seriousness and causality were assessed by the reporting PI. AEs satisfying the criteria for serious AEs (SAEs) were subject to expedited reporting to the co-ordinating centre where a second assessment of causality was made. If either causality assessment indicated a definite, probable or possible relationship to study medication, an assessment of expectedness was made with reference to the current Investigator’s Brochure.

Magnetic resonance imaging substudy

Thirteen study sites participated in a MRI substudy. MRI site selection was on the basis of capacity and cost, with all MRI sites being required to perform a pre-enrolment qualifying scan prior to the start of participant recruitment. Non-contrast brain MRI was performed at baseline and annually in 273 participants (183 allocated to active treatment, 91 allocated to placebo). Images were analysed for PBVC and new and enlarging lesions at the Queen Square Multiple Sclerosis Centre, NMR Research Unit Trials Office, University College London’s Institute of Neurology, London, UK.

Data management and monitoring

Paper-based, two-part, no-carbon-required case report forms (CRFs) were used for both treating physician- and assessor-acquired data. Original data were sent by post to the co-ordinating centre in Plymouth and entered onto a central database held on a secure server. A system of double data entry was used, enabling generation of a single data set following a process of data comparison. Bespoke database reports were created to track participant status throughout the trial and ensure that data were requested and received from sites in a timely manner. A robust process was in place for clarification of queries in the case of missing or ambiguous data. The quality of the trial data were monitored using a combination of centralised data checking and site visits at which participant CRFs were compared with clinical case notes. Participants returned their self-completion questionnaires in Freepost™ envelopes directly to the co-ordinating office for double data entry.

Trial oversight

A Trial Steering Committee (TSC) (see Appendix 5 for details), including an independent chairperson, a neurologist, a statistician and a lay representative, was responsible for trial oversight and met on an annual basis. An independent Data Monitoring Committee (IDMC) (see Appendix 5 for details) met annually to review unblinded safety and efficacy data. Interim analyses of primary outcomes were produced by an independent statistician on request from the IDMC, using the pre-defined Haybittle–Peto boundary stopping rule. Four interim analyses were undertaken after 298 and 493 participants had been recruited and, then, annually during the follow-up period. The IDMC recommended continuation of the trial following all interim analyses. Appendix 6 gives details of patient and public involvement in the CUPID study.

Sample size and power

Previous data suggested a likely progression rate of approximately 70% in the placebo group. 5 Based on this and an expected 5% annual loss to follow-up rate, recruiting 492 patients provided 90% power, at a two-sided 5% significance level, to detect a one-third reduction in hazard of progression [i.e. hazard ratio (HR) 0.67], corresponding to a relative reduction in risk of progression over 3 years of 21% (from 70% to 55% progression in the Δ⁹-THC group).

Initial data from 210 patients with MS admitted in acute relapse for intravenous steroid treatment baseline and post treatment showed the standard deviation (SD) of the scores from MSIS-29v2 at baseline and follow-up were 21.8 and 24.3, respectively (Professor Jeremy Hobart, Institute of Neurology, London, 1999–2002, unpublished data). The SD of the difference in scores was estimated to be 20.6. With this SD, a difference in means of seven points (one-third of a SD) could be detected in a sample of 492 on a two-sided test carried out at the 5% level, with power in excess of 90%.

For the MRI substudy, allowing for losses to follow-up at a rate of 5% per year, it was estimated that a total of 261 patients, allocated to active treatment and placebo in a 2 : 1 ratio, would give 90% power to detect 40% slowing in rate of atrophy, with scans performed pre treatment and then at years 1, 2 and 3.

End Point Committee

An End Point Committee (EPC) was convened to adjudicate on EDSS score progression in participants for whom missing EDSS scores prevented confirmation of this end point according to protocol. The committee, comprising the chief investigator and two other consultant neurologists (one independent), met once on 27 February 2012. The EPC review was blinded to treatment allocation and included consideration of additional clinical details obtained from participants’ medical records. The outcomes determined by this committee were used in sensitivity analyses of EDSS score progression. Terms of reference are outlined in Appendix 7.

Statistical methods

The statistical analysis plan (SAP) was finalised and agreed by the TSC before unblinding. Data analysis, using the statistical software R version 2.14.2 (The R Foundation for Statistical Computing, Vienna, Austria), was undertaken on an ITT basis. All tests were carried out at the 5% significance level, with no adjustments for multiple testing.

In models for each outcome, main effects of study site, baseline EDSS score, disease type, age at registration, sex and weight were considered, as well as treatment allocation.

Ethics and research governance approval

The study was approved by the South and West Devon REC and conducted in accordance with GCP guidelines. Eligible patients provided written informed consent before participation.

Trial registration

The trial was registered as Current Controlled Trials ISRCTN62942668.

Role of the funding source

The funders had no role in study design, data collection, analysis, interpretation or writing of the report.

Summary of changes to the study protocol

The protocol as approved by MHRA and REC at the start of the study underwent four amendments of significance to the conduct of the study. An additional exclusion criterion (recent use of any experimental therapies with potential disease-modifying actions) was added during the first few months of the study in order to reduce potential confounding of results. At the start of 2009, refrigerated storage conditions for the trial medication were introduced. In April 2009, an additional follow-up phase was added to the study – this used the fact that participants with new EDSS progression at 36 months were required to be reviewed at 42 months on treatment as an opportunity to follow up the remaining participants off treatment from months 36 and 42. The protocol was last amended in May 2010 to include the option of EDSS assessment by telephone as opposed to face to face.

Unblinding of randomised treatments

The treatment allocation was unblinded six times during the course of the trial at the request of the sponsor organisation to assist in the management of suspected unexpected serious adverse reactions (SUSARs). In addition, unblinding of four participants at four separate sites was carried out at the request of the local investigator, on clinical grounds.

Telephone-based assessment of Expanded Disability Status Scale score

Of 3812 assessments of EDSS score over the study period, 42 (1.1%) were by telephone, rather than face to face. These telephone assessments were carried out on a total of 39 patients (25 assigned to active; 14 assigned to placebo).

Figure 1 shows the trial profile and Table 3 shows discontinuations of trial medication and losses to follow-up. A total of 498 patients were randomly assigned to active treatment (n = 332) or matching placebo (n = 166). The data from three patients (two randomised to active, one to placebo) were removed from the trial because they withdrew their consent after randomisation. A further two patients (one randomised to active, one to placebo) were found to be ineligible after randomisation. Four hundred and ninety-three (329 active, 164 placebo) received their allocated intervention and were therefore included in an ITT analysis. Of the 493 randomised and treated participants, 415 (84%) completed follow-up, of whom 119 (29%) had prematurely discontinued trial medication (Figure 1).

FIGURE 1.

The Consolidated Standards of Reporting Trials (CONSORT) flow diagram. a, Nine patients initially did not fulfil entry criteria, but did so on subsequent rescreening. DNA, did not attend; PEP, primary end point of EDSS score progression.

Baseline comparability of randomised groups

Baseline patient and disease characteristics were similar in both treatment groups (Table 4). At baseline, 59% of participants were women, 61% had SPMS and 78% had an EDSS score of 6.0 or 6.5. There were no important differences in outcome measures assessed at baseline (Table 5).

Prescribed dose of trial medication

Prescribed daily doses of trial medication at each 6-monthly follow-up are summarised in Table 6 for those patients not discontinuing trial medication and for all patients. Among those patients not withdrawing from trial medication (n = 178 active, n = 118 placebo), median prescribed daily dose during final year of follow-up was four capsules (25th–75th percentiles 2–6 capsules) in the active group compared with six (25th–75th percentiles 4–8 capsules) in the placebo group. Final year medians among all patients were four capsules (25th–75th percentiles 2.0–5.5) for active and six capsules (25th–75th percentiles 4–8 capsules) for placebo. Percentiles of prescribed daily dose among non-withdrawals, by treatment group and weight group, are shown in Figure 2.

FIGURE 2.

Percentiles of prescribed daily dose of trial medication among non-withdrawals at each visit, by treatment group and weight group. (a) Active, weight < 60 kg; (b) active, weight 60–80 kg; (c) active, weight > 80 kg; (d) placebo, weight < 60 kg; (e) placebo, weight 60–80 kg; and (f) active, weight > 80 kg. Heavy solid line, median; narrow solid lines, 25th and 75th percentiles; dashed lines, 5th and 90th percentiles. The maximum weight-related daily dose is superimposed in green.

Random urine testing to determine any illicit cannabis use

Results from urinalyses throughout the study are given in Table 7. These results showed little illicit cannabis use in the placebo group and an increasing proportion of negative test results within the active group over time.

The main results are summarised in Table 8 and detailed below.

Pre-specified analyses of primary clinical outcomes

Primary analysis of time to first confirmed Expanded Disability Status Scale score progression

Primary analysis using a Cox regression model showed no evidence of an effect of age (p = 0.36), disease type (p = 0.12), sex (p = 0.56), weight (p = 0.11) or treatment (p = 0.57; see Table 8) on time to confirmed EDSS score progression. The HR for first EDSS score progression event in patients randomly assigned to dronabinol compared with those assigned to placebo was 0.92 [95% confidence interval (CI) 0.68 to 1.23; see Table 8]. At trial completion, Kaplan–Meier estimates of the probability of EDSS score progression were 0.55 (95% CI 0.46 to 0.63) in the dronabinol group compared with 0.60 (95% CI 0.44 to 0.71) in the placebo group (Figure 3).

FIGURE 3.

Kaplan–Meier estimates of the probability of progression on the EDSS in the two treatment groups, confirmed after 6 months within the study period. The numbers at risk (cumulative number of censored observations) are given. Those patients who were lost to follow-up during the trial are marked by +. Those who reached the end of the trial without progressing are marked by x.

We noted evidence of some study site effects and of an effect of baseline EDSS score on time to confirmed progression (Figure 4). Most notably, relative to a baseline EDSS score of 4.0, there was an increased hazard of disease progression among those with a baseline EDSS score of 5.5 (HR 3.17, 95% CI 1.45 to 6.93; p = 0.004) and a reduced hazard among those with a baseline EDSS score of 6.5 (HR 0.49, 95% CI 0.24 to 0.98; p = 0.04). However, the numbers of participants in the individual EDSS groups are small (Figure 5), as are the numbers in some study sites (see Table 16 and Appendix 3).

FIGURE 4.

Estimated HRs and 95% CIs for EDSS score progression from a Cox regression model. HRs for study sites are compared with site 01; HR for SPMS is compared with PPMS; active compared with placebo; each baseline EDSS is compared with a baseline EDSS score of 4.0; and men are compared with women. BL, baseline.

FIGURE 5.

Kaplan–Meier estimates of the probability of progression on the EDSS, by baseline EDSS score, confirmed after 6 months within the study period. The numbers at risk (cumulative number of censored observations) are given. Those patients who were lost to follow-up during the trial are marked by +. Those who reached the end of the trial without progressing are marked by x. BL, baseline.

The global PH test gave no evidence that the PH assumption was violated (χ² = 36, 36 degrees of freedom; p = 0.47).

Sensitivity analyses of time to first confirmed Expanded Disability Status Scale score progression

Results of sensitivity analysis showed that when losses to follow-up were treated as progression events rather than censored observations, the estimated HR (active : placebo) for EDSS score progression changed to 1.11 (95% CI 0.86 to 1.44; see Table 8), but the estimated effect of treatment remained non-significant (p = 0.41). This change in HR might be because the dronabinol group had a higher proportion of losses to follow-up for EDSS assessment [56 of 71 (79%)] than the placebo group [15 of 71 (21%)] and represents a worst-case scenario in terms of patient deterioration and hence the potential benefit of dronabinol.

The EPC reviewed data on 95 patients [71 active (74.7%); 24 placebo (25.3%)], for which there were ambiguities regarding EDSS scores. The EPC considered 22 (12 active; 10 placebo) of these patients to have progressed. These patients had no confirmed progression according to the data collected from the trial schedule. A further four patients (three active; one placebo) were considered to have progressed prior to the time of progression determined from the trial schedule. Clinical information on the remaining 69 patients reviewed by the EPC either confirmed non-progression or was insufficient to draw any further conclusions over those made on the primary data. As a result, data derived following EPC review consisted of a total of 240 first progression events compared with 218 in the primary data (with losses to follow-up considered as censored observations in both).

Conclusions from the main analyses of time to first EDSS score progression were robust to sensitivity analyses in terms of whether or not conclusions from the EPC were considered in defining EDSS progressions under both approaches to dealing with losses to follow-up, that is treated as censored observations or as progression events (see Table 8).

Furthermore, estimated HRs (active : placebo) for EDSS score progression remained similar after sequential removal of study sites with high loss to follow-up rates, under each of the two ways of treating losses to follow-up and each of the two data sets, that is according to trial schedule or following EPC review (Figure 6).

FIGURE 6.

Estimated HRs (active : placebo) for EDSS score progression, with 95% CIs and p-values from Cox regression models, sequentially removing study sites with high loss to follow-up for each of the two data sets and each of the two ways of dealing with losses to follow-up. (a) Data = according to trial schedule, losses to follow-up = censored observations; (b) data = according to trial schedule, losses to follow-up = progression events; (c) data = following EPC review, losses to follow-up = censored observations; and (d) data = following EPC review, losses to follow-up = progression events. The two rows of n (%) on horizontal axes show top row = number of sites removed (% losses to follow-up in remaining data); bottom row = site number removed (within-site % losses to follow-up). n, number of participants included in each fitted model.

Pre-specified subgroup analyses of time to first confirmed Expanded Disability Status Scale score progression

Pre-specified subgroup analyses of time to first EDSS score progression suggested a differential effect of treatment between participants with lower (4.0–5.5) and higher (6.0–6.5) baseline EDSS scores (Figure 7). There was little evidence of differential effects of treatment among subgroups defined in terms of sex, disease type, or age or weight at registration.

FIGURE 7.

Estimated HRs (active : placebo) and 95% CIs for EDSS score progression by subgroup.

Primary analysis of change in Multiple Sclerosis Impact Scale-29 20-point physical subscale

A multilevel model fitted to repeated measures of MSIS-29phys score showed no evidence of an effect of treatment [estimated between-group difference (active–placebo) −0.91 points, 95% CI −2.01 to 0.19 points; p = 0.11; see Table 8], or of disease type, sex, weight or study site (data not shown; p > 0.05 for all).

It was estimated that MSIS-29phys score reduced by a mean of 1.4 points (95% CI 0.3 to 2.5 points; p = 0.02) for every 10-year increase in age. In both treatment groups, mean MSIS-29phys score decreased from baseline to month 3, after which it tended to increase (Figure 8).

FIGURE 8.

Estimated mean MSIS-29phys score, with 95% CIs, at each visit, separated by treatment group. The numbers of patients with total scores calculated at each visit, in each treatment group, are given directly above the CI for placebo and directly below the CI for active. Green points, active treatment; black points, placebo.

With the exception of a small reduction in MSIS-29phys score in patients with a baseline EDSS score of 5.0 compared with those with a score of 4.0, MSIS-29phys score tended to increase with increasing baseline EDSS score (data not shown).

Results from the primary analysis of repeated measures of MSIS-29phys remained unchanged after removal of non-significant terms from the fitted model and under an alternative analysis based on comparison of treatment groups in terms of change from baseline to last valid observation [estimated between-group difference (active–placebo) –1.4 points, 95% CI –3.3 to 0.4 points; p = 0.13].

Pre-specified analyses of secondary outcomes

Results of multilevel models fitted to data on the secondary outcomes MSWS-12v2, MSFC, RMI, SF-36(PH) and MSSS-88 are summarised in Table 8 and detailed below.

Multiple Sclerosis Walking Scale-12

A multilevel model fitted to repeated measures of MSWS-12v2 score showed no evidence of an effect of treatment [estimated effect –0.19 (95% CI –0.97 to 0.60); p = 0.74; see Table 8], or of disease type, sex or weight (data not shown; p > 0.05 for all). There was some evidence of study site effects (data not shown) and of effects of baseline EDSS score. Compared with those with a baseline EDSS score of 4.0, MSWS-12v2 was estimated to be, on average, 5.7 (95% CI 2.3 to 9.0), 6.1 (95% CI 3.7 to 8.5) and 9.3 (95% CI 6.8 to 11.8) points higher in those with a baseline EDSS score 5.5, 6.0 and 6.5, respectively. In both treatment groups, mean MSWS-12v2 score decreased from baseline to month 3, after which it tended to increase (Figure 9).

FIGURE 9.

Estimated mean MSWS-12v2, with 95% CIs, at each visit, separated by treatment group. The numbers of patients with total scores calculated at each visit, in each treatment group, are given directly above the CI for placebo and directly below the CI for active. Green points, active treatment; black points, placebo.

Multiple Sclerosis Functional Composite

A multilevel model fitted to repeated measures of MSFC composite z-score showed no evidence of an effect of treatment; estimated between-group difference (active–placebo) –0.03 (95% CI –0.19 to 0.09; p = 0.72; see Table 8). Multilevel models fitted to the MSFC component-wise z-scores each showed no evidence of an effect of treatment. Estimated between-group differences (active–placebo) were: T25-FW –0.08 (95% CI –0.25 to 0.09; p = 0.37); 9-HPT 0.05 (95% CI –0.04 to 0.13; p = 0.28); and PASAT –0.01 (95% CI –0.10 to 0.09; p = 0.92). Across both treatment groups, mean T25-FW, 9-HPT and composite z-scores increased from baseline to week 1, after which they tended to decrease. After an initial increase at week 1, PASAT z-scores remained relatively constant over the 3-year study period (Figure 10).

FIGURE 10.

Estimated mean MSFC composite (a), T25-FW (b), 9-HPT (c) and PASAT (d) z-scores, with 95% CIs, at each visit, separated by treatment group. The numbers of patients with scores calculated at each visit, in each treatment group, are given directly above the CI for placebo and directly below the CI for active. Green points, active treatment; black points, placebo.

Rivermead Mobility Index

A multilevel model fitted to repeated measures of RMI showed no evidence of an effect of treatment [the estimated between-group difference (active–placebo) was 0.04 (95% CI –0.24 to 0.32; p = 0.76; see Table 8)] or of disease type, sex or weight (data not shown; p > 0.05 for all). There was some evidence of study site effects (data not shown) and of effects of baseline EDSS score. Compared with those with a baseline EDSS score of 4.0, RMI was estimated to be, on average, 1.57 points (95% CI 0.38 to 2.76 points), 2.44 points (95% CI 1.59 to 3.30 points) and 4.99 points (95% CI 4.10 to 5.88 points) lower in those with a baseline EDSS score of 5.5, 6.0 and 6.5, respectively. In both treatment groups, RMI decreased from baseline to 30 months, after which it remained fairly constant (Figure 11).

FIGURE 11.

Estimated mean RMI, with 95% CIs, at each visit, separated by treatment group. The numbers of patients with total scores calculated at each visit, in each treatment group, are given directly above the CI for placebo and directly below the CI for active. Green points, active treatment; black points, placebo.

Short Form questionnaire-36 items (physical health subscale)

A multilevel model fitted to repeated measures of SF-36(PH) showed no evidence of an effect of treatment [the estimated between-group difference (active–placebo) was –0.15 (95% CI –0.83 to 0.53; p = 0.67; see Table 8)], or of disease type, sex or weight (data not shown; p > 0.05 for all). In both treatment groups, mean SF-36(PH) score increased from baseline to month 3, after which it tended to decrease (Figure 12).

FIGURE 12.

Estimated mean SF-36(PH), with 95% CIs, at each visit, separated by treatment group. The numbers of patients with total scores calculated at each visit, in each treatment group, are given directly above the CI for placebo and directly below the CI for active. Green points, active treatment; black points, placebo.

With the exception of a small increase in SF-36(PH) score in patients with a baseline EDSS score of 5.0 compared with those with a score of 4.0, SF-36(PH) score tended to decrease with increasing baseline EDSS score (data not shown).

Multiple Sclerosis Spasticity Scale-88

Figure 13 shows estimated mean MSSS-88 scores, with 95% CIs, by visit and treatment group, for each of the eight subscales.

FIGURE 13.

Estimated mean MSSS-88 subscales 1–8 scores, with 95% CIs, at each visit, separated by treatment group. The numbers of patients with total scores calculated at each visit, in each treatment group, are given directly above the CI for placebo and directly below the CI for active. Subscales relate to how bothered the patient has been by their spasticity in the past 2 weeks, where 1 = muscle stiffness; 2 = pain and discomfort; 3 = muscle spasms; 4 = effect of spasticity on daily activities; 5 = effect of spasticity on ability to walk; 6 = effect of spasticity on body movements; 7 = effect of spasticity on feelings; and 8 = effect of spasticity on social functioning. Green points, active treatment; black points, placebo.

For each of the physical components of the MSSS-88, subscales 1 to 6 inclusive, after an initial decrease from baseline to month 3, mean scores tended to increase. Estimated means were consistent across treatment groups, as seen by the overlapping CIs. For the two psychological components, subscales 7 and 8, after an initial decrease from baseline to month 3, mean scores remained relatively constant over the study period. Estimated mean scores for these components tended to be higher in the active group than in placebo, but any differences failed to reach statistical significance.

Multilevel models fitted to three groups of the MSSS-88, where MSSS-88 (1) combines subscales 1–3; MSSS-88 (2) combines subscales 4–6 and MSSS-88 (3) combines subscales 7 and 8 (as described in Chapter 2), each showed no evidence of an effect of treatment. Estimated between-group difference (active–placebo) for MSSS-88 (1) was 0.26 (95% CI –1.99 to 2.52; p = 0.82; see Table 8); for MSSS-88 (2) was –0.02 (95% CI –2.35 to 2.32; p = 0.99; see Table 8); and for MSSS-88 (3) was 1.00 (95% CI –0.70 to 2.70; p = 0.25; see Table 8). In both treatment groups, mean MSSS-88 (1) and mean MSSS-88 (2) decreased from baseline to month 3, after which they tended to increase (Figure 14). After an initial decrease from baseline to month 3, mean MSSS-88 (3) remained relatively constant over the study period (see Figure 14).

FIGURE 14.

Estimated mean MSSS-88 (1) total score from subscales 1–3, concerning muscle stiffness/spasms, pain and discomfort; MSSS-88 (2) total score from subscales 4–6, concerning activity, walking and body movements; and MSSS-88 (3) total score from subscales 7 and 8, concerning feelings and social functioning, with 95% CIs, at each visit, separated by treatment group. The numbers of patients with total scores calculated at each visit, in each treatment group, are given directly above the CI for placebo and directly below the CI for active. Green points, active treatment; black points, placebo.

Analysis of premature discontinuations of trial medication and losses to follow-up

Of the 493 patients included in the ITT analysis, 119 (24.1%; 89 active, 30 placebo) prematurely discontinued trial medication but remained in follow-up. Seventy-eight patients (15.8%; 62 active, 16 placebo) were lost to follow-up, meaning that 296 (60%) patients completed the study on trial treatment.

There was evidence of an increased risk of discontinuation of trial medication in the active group compared with placebo (p < 0.001, log-rank test) (Figure 15).

FIGURE 15.

Kaplan–Meier estimates of the probability of discontinuation of trial medication in the two treatment groups. Those patients who were lost to follow-up during the trial, without previous discontinuation of trial medication, are marked by +. Those patients who reached the end of the trial without discontinuing trial medication are marked by x.

Reasons for discontinuation of trial medication or loss to follow-up were dominated by AEs, accounting for 65% of all early discontinuations (Table 14). Reasons for loss to follow-up are summarised in Table 15. The most common reasons were reported as ‘MS or other health issues’ and ‘other’, accounting for 50% (39 out of 78) of all losses to follow-up. ‘Travel or burden of the trial’ accounted for 22% (17 out of 78) of reasons for loss to follow-up and accounted for a larger proportion of losses in placebo patients [5 out of 16 (31%) compared with 12 out of 62 (19%) of the losses in the active group].

Rates of discontinuations from trial medication or loss to follow-up varied across study sites (Table 16).

Following a forward selection procedure, a Cox regression model fitted to data on time to discontinuation of trial medication or loss to follow-up showed evidence of effects of treatment allocation, sex and study site on the risk of withdrawal or loss to follow-up. The risk of withdrawal or loss to follow-up was estimated to be higher in men than in women [HR (men : women) 1.37, 95% CI 1.02 to 1.84] and higher in the active group than in the placebo group [HR (active : placebo) 1.97, 95% CI 1.41 to 2.76].

Pre-specified analyses of magnetic resonance imaging substudy

Two hundred and seventy-four patients from 13 study sites were allocated to the MRI substudy. Of these, one patient was excluded at baseline visit (as baseline scan was deemed problematic, due to patient tremor).

Baseline data on demographic and disease characteristics were similar across treatment groups (Table 17). Fifty-nine per cent of patients were women, 64% had SPMS and 76% had an EDSS score of 6.0 or 6.5.

Forty-seven of the 182 patients (25.8%) on active treatment and 17 of the 91 patients (18.7%) on placebo were lost to follow-up during the study period. Figure 16 shows the flow of patients over the 3-year follow-up period.

FIGURE 16.

Flow of participants through the MRI substudy.

Between-treatment-group comparisons of PBVC and numbers of new or enlarging T2 and new T1 lesions at each annual follow-up showed little evidence of an association between treatment allocation and these outcomes (Table 18).

A multilevel model fitted to cumulative PBVC showed no evidence of an effect of active treatment on brain atrophy compared with placebo over the course of the study [estimated between-group difference in PBVC (active–placebo) was −0.01%, 95% CI −0.26% to 0.24%; p = 0.94; see Table 8]. However, brain atrophy did change significantly over time (p < 0.0001); using a fitted model, cumulative PBVC was estimated to be a mean of −0.58% at year 1, −1.20% at year 2 and −2.02% at year 3 (Figure 17).

FIGURE 17.

Estimated mean cumulative PBVC (%), and 95% CI, by treatment group, measured at yearly MRI visits. Green points, active treatment; black points, placebo. n, number of patients with cumulative PBVC calculated at each visit, given directly above the CI for placebo and directly below the CI for active.

There was evidence of an effect of baseline normalised brain volume (NBV) on brain atrophy. Using a fitted model, it was estimated that, for a 100-unit reduction in baseline NBV, brain atrophy increased by a mean of 0.21% (95% CI 0.08% to 0.34%; p = 0.003).

Treatment did not significantly affect the occurrence of new or newly enlarging T2 lesions [estimated odds ratio (OR) (active : placebo) 0.89, 95% CI 0.50 to 1.58; p = 0.70; see Table 8] or new T1 lesions [estimated OR (active : placebo) 1.05, 95% CI 0.59 to 1.88; p = 0.87; see Table 8].

Introduction

Post-hoc exploratory analyses covered three main areas, as described below.

Firstly, the suggestion of a treatment effect from pre-specified subgroup analysis of time to first confirmed EDSS score progression led to a post-hoc analysis of time to EDSS score progression among patients in this EDSS group. Further analysis examined the effect of treatment allocation in the two participant groups with a baseline EDSS score of 4.0–5.5 and 6.0–6.5, on change in MSIS-29phys and PBVC.

Secondly, an on-treatment data set included follow-up data on all patients up to the time of withdrawal from trial treatment, loss to follow-up or end of study – whichever came first. Further analyses of time to first confirmed EDSS score progression, change in MSIS-29phys and PBVC were carried out using these data sets.

Thirdly, in contrast to pre-specified analyses, presented in the previous chapter, which were based on time to first progression event (or last follow-up) with any subsequent changes in EDSS score not considered, post-hoc exploratory analyses considered EDSS progression as an unconfirmed, recurrent event, using the following criteria:

The first progression event is when EDSS score increases by at least 1 point from a baseline score of 4.0, 4.5 or 5.0, or by at least 0.5 points from a baseline score of 5.5 or higher. Subsequent progression events are when the EDSS score increases by these amounts, from the score at the previous progression. Progressions do not need to be confirmed at the next scheduled 6-monthly visit.

Methods

Results

Results from post-hoc exploratory subgroup and on-treatment analyses are summarised in Table 19 and detailed below.

Subgroup analyses

Figure 18 shows Kaplan–Meier estimates of the probability of EDSS score progression in the subgroup of patients with a baseline EDSS score of 4.0–5.5, separated by treatment group. There was some evidence of a potentially beneficial effect of active treatment, compared with placebo in this low-score EDSS subgroup (p = 0.01, log-rank test).

FIGURE 18.

Kaplan–Meier estimates of the probability of progression on the EDSS in the two treatment groups, confirmed after 6 months within the study period, among the subgroup of patients with a baseline EDSS score of 4.0–5.5. The numbers at risk (cumulative number of censored observations) are given. Those patients who were lost to follow-up during the trial are marked by +. Those who reached the end of the trial without progressing are marked by x.

There was some evidence of an effect of treatment on change in MSIS-29phys score in the group of patients with a baseline EDSS score of 4.0–5.5. On average, MSIS-29phys scores in the active treatment group were estimated to be 3.26 points lower (95% CI for reduction 1.63 to 4.89 points; p = 0.0001; see Table 19) than in the placebo group. There was no significant effect of treatment on change in MSIS-29phys score in the group of participants with a baseline EDSS score of 6.0–6.5 [estimated between-group difference (active–placebo) –0.21, 95% CI –1.37 to 0.96; p = 0.73; see Table 19].

There was no significant effect of treatment on brain atrophy in the two participant groups defined by baseline EDSS score; estimated between-group differences in PBVC (active–placebo) were –0.06% (95% CI –0.42% to 0.29%; p = 0.73; see Table 19) and 0.01% (95% CI –0.26% to 0.28%; p = 0.95; see Table 19) for baseline EDSS scores of 4.0–5.5 and 6.0–6.5, respectively.

On-treatment analyses

Analysis of time to first confirmed EDSS score progression was carried out using the on-treatment data set described above. Considering all withdrawals from trial treatment as losses to follow-up at the time of withdrawal, the on-treatment data set included 177 first progression events (114 in the active group, 63 in the placebo group), compared with 218 (145 active, 73 placebo) in the ITT data set used in the primary analysis. A Cox regression model provided no evidence of an effect of treatment on probability of progression [HR (active : placebo) 0.96, 95% CI 0.69 to 1.34; p = 0.83; see Table 19]. This estimated treatment effect was similar when study sites with low throughput (< 20 patients) were combined in a single effect in the fitted model [HR (active : placebo) 0.95, 95% CI 0.69 to 1.31; p = 0.76; see Table 19]. The global PH test gave no evidence that the PH assumption was violated under either fitted model (χ² = 30.2, 36 degrees of freedom, p = 0.74 for a Cox model including individual study site effects; and χ² = 13.3, 19 degrees of freedom, p = 0.82 for a Cox model including a single effect for sites with low throughput).

At trial completion, Kaplan–Meier estimates of the probability of EDSS score progression were 0.54 (95% CI 0.43 to 0.62) in the dronabinol group, compared with 0.58 (95% CI 0.41 to 0.71) in the placebo group (Figure 19).

FIGURE 19.

Kaplan–Meier estimates of the probability of progression on the EDSS in the two treatment groups, confirmed after 6 months within the study period, based on an on-treatment data set, in which withdrawals from trial treatment are considered as losses to follow-up at the time of withdrawal. The numbers at risk (cumulative number of censored observations) are given. Those patients who were lost to follow-up during the trial are marked by +. Those who reached the end of the trial without progressing are marked by x.

A multilevel model fitted to repeated measures of MSIS-29phys score in the on-treatment data set showed no evidence of an effect of treatment. The estimated between-group difference in MSIS-29phys (active–placebo) score was –0.77 (95% CI –1.92 to 0.38; p = 0.19; see Table 19).

Analysis of brain atrophy in the on-treatment data set on the MRI substudy was based on a total of 418 observations among 202 patients and included 200 measures of PBVC at year 1, 175 measures of cumulative PBVC at year 2 and 43 at year 3. There was no evidence of an effect of treatment on brain atrophy; the estimated between-group difference in PBVC (active–placebo) was 0.03% (95% CI –0.24% to 0.31%; p = 0.80; see Table 19). Using a fitted model, cumulative PBVC was estimated to be a mean of −0.60% at year 1, −1.17% at year 2 and −2.01% at year 3 (Figure 20).

FIGURE 20.

Estimated mean cumulative PBVC (%), and 95% CI, by treatment group, measured at yearly MRI visits, in the on-treatment data. Green points, active treatment; black points, placebo. n, number of patients with cumulative PBVC calculated at each visit, given directly above the CI for placebo and directly below the CI for active.

There was evidence of an effect of baseline NBV on brain atrophy. Using a fitted model, it was estimated that, for a 100-unit reduction in baseline NBV, atrophy increased by a mean of 0.20% (95% CI 0.06% to 0.35%; p = 0.008).

Expanded Disability Status Scale score transitions and recurrent progression events

Based on the definition of unconfirmed, recurrent progression events introduced earlier in this chapter, there were a total of 380 (245 among 182 patients on active treatment, 135 among 97 patients on placebo) progression events observed over the course of the trial. Frequencies and relative frequencies of patients having different numbers of events during follow-up are given in Table 20.

TABLE 20 - Frequencies and relative frequencies (%) of patients seen to have specified numbers of unconfirmed, recurrent progression events, by treatment group, baseline EDSS score and overall

Percentages may not sum to 100 due to rounding.

Patient characteristics	Number of progression events
	0	1	2	3 or 4
Treatment group
Active (n = 329)	147 (44.7)	128 (38.9)	45 (13.7)	9 (2.7)
Placebo (n = 164)	67 (40.9)	68 (41.5)	21 (12.8)	8 (4.9)
Baseline EDSS score
4.0–5.5 (n = 110)	33 (30.0)	58 (52.7)	17 (15.5)	2 (1.8)
6.0 (n = 254)	111 (43.7)	107 (42.1)	25 (9.8)	11 (4.3)
6.5 (n = 129)	70 (54.3)	31 (24.0)	24 (18.6)	4 (3.1)
Total (N = 493)	214 (43.4)	196 (39.8)	66 (13.4)	17 (3.4)

The probability of unconfirmed EDSS score progression appeared to depend on the starting EDSS score (Figures 21 and 22). For example, the probability of progression from a score of 5.5 tended to be higher than probabilities of progression from other starting scores. It is notable that this is the starting score for which the definition of progression changes from a 1-point increase to a 0.5-point increase. As starting EDSS score increased from 5.5, the probability of progression tended to decrease.

FIGURE 21.

Estimated probabilities of EDSS score progression (unconfirmed), with 95% CI, by starting EDSS score, in each 6-monthly interval from baseline. (a) 0–6 months (0–182 days from baseline); (b) 6–12 months (183–365 days from baseline); (c) 12–18 months (366–548 days from baseline); (d) 18–24 months (549–731 days from baseline); (e) 24–30 months (732–914 days from baseline); and (f) 30–36/42 months (915 or more days from baseline). n, number of patients with given starting EDSS score who have an EDSS score recorded at the end of the 6-monthly interval.

FIGURE 22.

Estimated probabilities of EDSS score progression (unconfirmed), with 95% CI, by starting EDSS score, in all 6-monthly intervals from baseline. n, number of patients with given starting EDSS score who have an EDSS score recorded at the end of the 6-monthly interval.

Overall, at a starting EDSS score of < 5.5, the probability of progression was slightly lower than for a starting EDSS score of 5.5 and slightly higher than for a starting EDSS score > 5.5 (see Figure 22). However, the number of observations at this lower end of the EDSS is relatively small, as reflected in the wide CIs.

Starting EDSS scores at baseline and at each 6-monthly follow-up were grouped into L, M and H, as described earlier in this chapter. Transition matrices were found for moving between states (Table 21). When considering progression from these three states, with the exception of the time period from baseline to 6 months, the probability of progression decreased with increasing starting EDSS score, for each 6-monthly period and overall (Figure 23).

TABLE 21 - Counts and proportions of transitions between EDSS score groups, in each 6-monthly follow-up period

H, EDSS score of 6.5 or higher; L, EDSS score of 5.5 or lower; M, EDSS score of 6.0.

Time interval (number of observations, % of total)	Counts				Row proportions
	From	To			From	To
		L	M	H		L	M	H
0–6 months (0–182 days from baseline) (n = 457, 92.7%)	L	79	20	4	L	0.7670	0.1942	0.0388
	M	16	149	69	M	0.0684	0.6368	0.2949
	H	2	19	99	H	0.0167	0.1583	0.8250
6–12 months (183–365 days from baseline) (n = 355, 72.0%)	L	60	23	3	L	0.6977	0.2674	0.0349
	M	6	129	29	M	0.0366	0.7866	0.1768
	H	0	25	80	H	0.0000	0.2381	0.7619
12–18 months (366–548 days from baseline) (n = 300, 60.9%)	L	47	9	0	L	0.8393	0.1607	0.0000
	M	7	126	26	M	0.0440	0.7925	0.1635
	H	0	13	72	H	0.0000	0.1529	0.8471
18–24 months (549–731 days from baseline) (n = 265, 53.8%)	L	38	11	1	L	0.7600	0.2200	0.0200
	M	5	112	21	M	0.0362	0.8116	0.1522
	H	0	12	65	H	0.0000	0.1558	0.8442
24–30 months (732–914 days from baseline) (n = 230, 46.7%)	L	34	6	0	L	0.8500	0.1500	0.0000
	M	9	92	23	M	0.0726	0.7419	0.1855
	H	0	11	55	H	0.0000	0.1667	0.8333
30–36/42 months (915 or more days from baseline) (n = 207, 42.0%)	L	35	9	0	L	0.7955	0.2045	0.0000
	M	0	85	19	M	0.0000	0.8173	0.1827
	H	0	8	51	H	0.0000	0.1356	0.8644

FIGURE 23.

Estimated probabilities of unconfirmed EDSS score progression, with 95% CI, in each 6-monthly interval from baseline and overall. 0–6 months (0–182 days from baseline); 6–12 months (183–365 days from baseline); 12–18 months (366–548 days from baseline); 18–24 months (549–731 days from baseline); 24–30 months (732–914 days from baseline); 30–36/42 months (915 or more days from baseline). In each case, the three estimates and intervals correspond to different starting EDSS score groups: left = L (starting EDSS score of 5.5 or less); centre = M (starting EDSS score of 6.0); right = H (starting EDSS score of 6.5 or higher).

Conclusions

Post-hoc exploratory analyses showed some evidence of a potentially beneficial effect of active treatment in terms of time to first confirmed EDSS score progression and change in MSIS-29phys score among those patients with a baseline EDSS score of 4.0–5.5. However, this subgroup of participants consists of just 110 individuals and so findings should be interpreted with caution. This beneficial effect was not seen in the MRI outcome, PBVC.

Results from analysis of time to first EDSS score progression, change in MSIS-29phys score and PBVC based on an on-treatment data set showed no evidence of a treatment effect on these outcomes and supported the conclusions from the primary analyses.

Detailed inspection of transition between EDSS scores highlighted the relatively low probability of progression on the EDSS when starting from the more disabled end of this scale. It also suggested an increased probability of progression from an EDSS score of 5.5; the lowest score at which a 0.5-point increase is deemed a progression. There was also an indication of an increased probability of progression from a baseline EDSS score of 5.5 in the primary analysis of time to first confirmed EDSS score progression (see Figures 4 and 5) but, once again, these findings must be interpreted with caution because of the small numbers of patients in the individual groups defined by baseline EDSS score.

Introduction

This chapter concerns the application in the CUPID study of RMT analyses. RMT is a modern psychometric method for constructing and evaluating rating scales. It has a number of advantages for clinical trials over traditional methods of analysing rating scale data. These advantages include the ability to derive interval-level measurement estimates from necessarily ordinal rating scale scores and the ability to examine change legitimately at the individual person level. In addition, RMT enables a very sophisticated evaluation of rating scale performance. This chapter, which has five sections, capitalises on those advantages of RMT.

Section 1 gives a brief introduction to RMT.

Section 2 reports the RMT-based evaluation of the performance as measurement instruments of MS-specific PROs used in the CUPID study: MSIS-29v2, MSWS-12v2 and MSSS-88 scores. Specifically, for each subscale within each instrument (e.g. MSIS-29v2 has two subscales; MSSS-88 has eight subscales; MSWS-12v2 has one scale), three performance-related issues were examined: scale-to-sample targeting (relative range of person and item locations, shape of person distribution); aspects of scale’s item performance [response category working, mapped continuum mapped by items, item fit statistics, item bias, differential item functioning (DIF)]; and aspects of derived person measurements [person separation index (PSI), person fit residuals, extreme scores]. It was concluded that all 11 scales/subscales performed well. This enabled interval-level measurement estimates for individual patients, with individual person standard error (SE) estimates, to be derived and taken forward to sections 3 and 4. Naturally, there were some issues of scale performance that could have been better.

Section 3 reports and compares the changes associated with active and placebo for those people in each treatment group, who remained on trial medication and had paired measurements at the two appropriate time points. These analyses used the interval-level measurement estimates and individual person SE estimates derived from section 2. Specifically, two potential benefits of dronabinol were examined: a symptomatic benefit between baseline and visit 5, and a disease-modifying benefit between baseline and the end of the study. For each benefit analysis, relative changes in the active and placebo groups at the group and individual person level were examined. At the group level, for each scale/subscale, the statistical significance of change scores [analysis of variance (ANOVA) paired sample t-tests] and the clinical significance of change scores [Cohen’s effect size; standardised response means (SRMs)] was examined in paired samples. At the individual person level, the proportions of people in each treatment group who achieved five levels of change (significantly better, non-significantly better, no change, non-significantly worse, significantly worse) were examined. In essence, analyses showed no significant differences between people treated with dronabinol or placebo. A notable finding was limited progression in the placebo group implying less progression over the study period than might have been expected in a cohort of people with progressive MS.

Section 4 reports post-hoc exploratory analyses. These analyses capitalise on the finding of a suggestion of a treatment effect in people with less EDSS-measured disability at baseline. Here the analyses of section 3 were repeated in two subgroups of people: those with a baseline EDSS score of 4.0–5.5; and those with a baseline EDSS score of 6.0–6.5. These post-hoc exploratory analyses alluded to the possibility of a clinically significant treatment effect in people with lower disability at baseline. It was also notable that placebo-treated people with higher levels of disability at baseline had surprisingly little progression during the CUPID study. This implies a limited pathophysiological substrate for examining the hypothesis that any treatment may have a disease-modifying effect.

Section 5 reflects on the findings and the lessons learned from the RMT analyses of the CUPID data.

Section 1: rating scales, rating scale data analysis and the added value of Rasch measurement theory

This section aims to give a very brief account of how rating scales work as measurement instruments, the scientific methods that underpin them and the case for choosing RMT as the most appropriate method for analysing rating scales data in the CUPID study. Fuller and heavily referenced accounts are given elsewhere. 6,15–17

Rating scales attempt to measure variables that cannot be easily quantified using other methods. For example, the MSWS-12v2 attempts to measure the walking ability of people with MS. Rating scales achieve measurement through a set of questions, each of which has two or more response options. The response options are allotted sequential integer scores. People answer the questions, choosing the most relevant response option. Measurements are derived from these data. This method of measurement stems from a body of research beginning in the early 1900s in education and psychology and with it developed the methods of testing the quality of the measurement process known as reliability and validity testing. The methods for developing rating scales and examining their reliability and validity have become known as psychometric methods.

Any rating scale can be considered a hypothesis of how a variable might be measured. A number of reasons underpin this statement. First, the aspects of people who rating scales are seeking to measure are complex socially constructed variables; here, aspects of the impact of MS. As such, their measurement cannot be easy. Second, there is uncertainty concerning the definitions of these variables. This hampers the construction of rating scales and opens the door for a range of potential measurement methods. Third, socially constructed variables are measured through their manifestations. For example, the walking ability of an individual is estimated from their performance on a finite set of tasks. It follows then, that the extent to which performance on a set of tasks can be combined is an empirical question. Finally, there is uncertainty of the extent to which the numbers generated by any rating scales satisfy criteria as reliable and valid measurements. For these reasons, any scale is a hypothesis of how a complex clinical variable might be measured and as such this hypothesis requires careful testing. For example, the MSWS-12v2 should be viewed as a hypothesis of how walking ability might be measured that requires careful testing. RMT provides a criterion for part of that hypothesis test.

Most rating scales assign successive integer scores to two or more ordered-item response categories that imply increasing problems. For example, in the MSIS-29v2 the four response options are: not at all = 1; slightly = 2; moderately = 3; and extremely = 4. Then, scores for groups of items that form subscales (e.g. the 20 physical impact items of the MSIS-29v2) are generally summed to produce a subscale score, which is used as a numerical index for a person on the physical scale and likewise for the other scales.

The general goal of psychometric evaluations of rating scales is to test whether or not this process of item scoring and summation satisfies the criteria for deriving measurements. More specifically, the goal is to establish whether or not it is legitimate to sum the integer-scored responses of a group of items and to determine the extent to which these summed scores are free from random error (whether or not they are reliable) in accounting for differences among people and measuring the attributes they purport to measure (whether or not they are valid). There are three main paradigms for developing, analysing and modifying rating scales: classical test theory (CTT), item response theory and RMT.

An evaluation of rating scales is well suited to the RMT paradigm because the Rasch model, a mathematical equation (model), provides a hypothesis test. This is because it articulates, a priori, the requirements of rating scale data for rating scales to satisfy criteria as measurement instruments. The model was derived from theory and is independent of any data set. Therefore, discrepancies detected by the analysis, that is between the hypothesis (scale data) and the hypothesis test (Rasch model requirements), indicate anomalies in the hypothesis (scale) as a measurement instrument. In this way, a RMT analysis provides diagnostic information-informing measurement instrument development by exposing anomalies to be understood and improved empirically.

Rasch measurement theory analyses use a mathematical model to test scale performance and generate measurements of people. The Rasch model, a probabilistic measurement model, was developed to express the requirement of invariant comparisons. By this we mean that the performance of the measurement method (here, a rating scale’s items) should, within reason, be independent of the people it is tested on and the measurements of people should be independent of the measurement method (here, a set of scale items) used to measure them. The Rasch model does not arise from the need to model (explain or summarise) any particular data set, and is compatible with the requirements of measurement methods used in physics, called fundamental or additive conjoint measurement.

One important property of the Rasch model is that for items with ordered response categories, as they are in the MS-specific rating scales using in the CUPID study, the successive categories should be scored with successive integers. This is a consequence of the requirement of invariance and not an assumption of the Rasch model. The use of both successive integer scores for scoring successive categories within items, and then the total sum score across items to characterise a person, is also a feature of the theory that underpins traditional psychometric methods (called traditional true score theory). However, in traditional psychometric methods (CTT), these are essentially starting points of the theory and do not arise from a priori requirements as they do in RMT.

A Rasch analysis of data, therefore, examines the extent to which the observed data accord with the requirements of the model. Therefore, at a general level it assesses the degree to which the responses of the persons to the items can be summed to provide a single score for each person which summarises each person’s location (measurement) on a scale. In other words, it tests whether or not the scale is able to place people in order on a scale, such that those who are ‘worse’ (in this case, suffer more from the effects of fatigue) produce higher scores than those who are ‘better’ (suffer less fatigue). This test of accord between the observed data and the expectations of the model is generally referred to as a ‘test of fit’. Tests of fit can be constructed to assess various specific hypotheses, such as that of multidimensionality and of unexpected dependence between pairs of items within subscales.

The procedure for testing the fit of the responses to the Rasch model involves first estimating the parameters of the model, which include a location parameter for each person and location parameters of the thresholds of the items which define the categories of a rating scale. A threshold is the transition point between adjacent item response categories. Specifically, it is the point on the scale at which the probability of scoring in adjacent categories is 50%. For example, each MSIS-29v2 item has four response categories: 1, 2, 3 and 4. Therefore, each item has three thresholds, 1–2, 2–3 and 3–4, which mark the three points on the scale at which the probability is 50% of scoring 1 and 2, 2 and 3, and 3 and 4, respectively. These parameter estimates are effectively the relevant summaries of the data. Given these estimates and the model, evidence is found of the extent to which the actual responses of people to the scale’s items can be recovered. This evidence is examined to assess the fit of the responses of each item to the Rasch model.

There is no necessary and sufficient test of fit between the data and the model. Therefore, multiple pieces of evidence of fit are required. Each focuses on different but related aspects as to where the responses might diverge from the model’s expectations. Typically, when using RUMM2030 (Rasch Unidimensional Measurement Models, Perth, WA, Australia), three statistical tests are used: chi-squared, fit residual and the residual correlation.

The chi-squared test of fit operates at the level of class intervals formed on the basis of the total scores of persons and then provides an estimate of the magnitude of departure of the mean score for an item in each class interval from the mean value expected according to the model. This is the most general test of fit and provides a graphical counterpart [item characteristic curves (ICCs)], which is also examined.

The fit residual test of fit operates at the level of the response of each person to each item and provides evidence that an item discriminates either more or less than is expected. Although there are multiple reasons why an item might discriminate more or less than expected, a smaller discrimination than expected might imply that the item is assessing a somewhat different construct from that assessed by the majority of items, whereas one greater than expected might imply that it is part of a subset of items that are overly dependent and in some sense are redundant.

The residual correlations test of fit provides evidence regarding which subsets of items might either be assessing some aspect that is common but different from the majority of the items, or which items might be redundant in their assessment relative to the rest. Which of these hypotheses is believed to be the case depends on a qualitative understanding of the construct, the items and the response formats.

Evidence not directly involving statistical tests of fit is also relevant. First, an important aspect of validity is examining the evidence that the scoring of successive categories by successive integers is justified. If it is not justified, it implies that the endorsement of a higher score does not imply more of the construct being assessed (impact of MS) than a lower score. In principle, it reflects some operational or conceptual problem with the ordering of the categories, such that respondents might consistently interpret the response categories in a different way than intended. Second, for the same reasons that we examine the empirical ordering of the item categories, we also consider whether or not the item locations represent a conceptually meaningful order, such that they constitute a measurement continuum. In other words, responses to items in a scale should follow a consistent order that represents the level of ‘severity’ of the construct. Finally, reliability was quantified using the PSI, which is analogous to Cronbach’s alpha. This index can be interpreted in terms of the spread of the persons generated by the items, and is used in two ways. First, a low PSI means that the power of the test of fit to detect misfit between the responses and the model is weak. Second, if an item shows marginal fit and removing the item reduces the PSI, it suggests that the item is adding random error and may not be consistent with the majority of the items.

The information provided by a RMT analysis is both sophisticated and extensive. Information from multiple tests is integrated. These are considered simultaneously and interactively, rather than individually and sequentially. Test result interpretation requires professional judgement, rather than adherence to rigid criteria, because the information needs to be contextualised and most statistical tests are sample size dependent. Additionally, as the analyses compare observed rating scale data against a stringent mathematical model, anomalies are expected. To facilitate interpretation, analyses are grouped under three broad, clinically relevant, simple (but not simplistic) questions: (1) is the scale to sample targeting adequate for making judgements about the performance of the scale and the measurement of people?; (2) has a measurement ruler been constructed successfully (scale/item analysis)?; and (3) how have the people been measured by the ruler (person measurement)? Data analyses reported here used RUMM2030.

Section 2: within-study measurement performance of the Multiple Sclerosis Impact Scale-29, Multiple Sclerosis Walking Scale-12 and Multiple Sclerosis Spasticity Scale-88 using Rasch measurement theory

Results

Table 22 summarises the results from the RMT analysis of the MSIS-29v2, MSWS-12v2 and MSSS-88. These three COAs contain 11 different measurement scales/subscales that differ in their content and number of items (from 8 to 20).

TABLE 22 - Rasch measurement theory evaluation of MS-specific PRO measures

ADL, activities of daily living; L, left; max., maximum; Mt, movement; R, right.

Scale	MSIS-29v2		MSWS-12v2	MSSS-88
				Symptoms (q1–12; 13–21; 22–35)			Physical (q36–46; 47–56; 57–67)			Psychological (q68–80; 81–88)
Subscale	Physical	Psychological	Walking	Stiffness	Pain	Spasms	ADL	Walking	Body Mt	Feelings	Social function
Items	20	9	12	12	9	14	11	10	11	13	8
Samples
Entered into project	3686	3667	3319	2343	2338	2341	2335	2223	2331	2328	2325
Invalid	0	0	0	0	0	0	0	0	0	0	0
Extremes	16	83	429	94	155	391	165	269	160	204	255
Item analysis	3670	3584	2850	2249	2183	1950	2170	1954	2171	2124	2070
Scale sample targeting
Item threshold range	–2.9/+2.6	–2.5/+2.1	–3.2/+4.1	–3.9/+4.2	–3.5/+2.6	–3.0/+2.3	–4.0/+4.0	–3.4/+2.7	–4.2/+3.3	–4.4/+3.0	–3.1/+2.7
Person measure range	–3.0/+5.2	–4.4/+4.2	–5.4/+5.9	–6.0/+5.8	–5.1/+4.6	–5.0/+4.7	–5.8/+5.8	–5.3/+4.8	–5.8/+5.3	–5.7/+5.0	–4.7/+4.3
Distribution shape	No skew	L skew+	R skew+++	No skew	L skew++	L skew	L skew++	R skew++	L skew+	R skew+	L skew++
Scale performance
Reversed thresholds	0	0	1	0	0	0	0	0	0	0	0
Continuum	–2.9/+2.6	–2.5/+2.1	–3.2/+4.1	–3.9/+4.2	–3.5/+2.6	–3.0/+2.3	–4.0/+4.0	–3.4/+2.7	–4.2/+3.3	–4.4/+3.0	–3.1/+2.7
Item fit residuals (total n)	–9.7/+ 19.2	–12.7/+17.5	–10.3/+8.5	–8.1/+ 4.5	–5.4/+5.1	–6.7/+5.9	–7.2/+9.7	–10.9/+12.5	–6.9/+6.8	–9.7/+8.8	–6.5/+3.3
Sign chi-squared (n = 500)	Item 20	Items 2, 5	Item 3	0	0	0	Item 2	Items 4, 10	0	0	0
Item bias (residual r > 0.30)	5 (max. + 0.41)	0	1 (–0.37)	5 (max. +0.57)	4 (max. –0.35)	3 (max. +0.47)	1 (max. –0.31)	9 (max. –0.35)	4 (max. +0.61)	4 (max. +0.45)	3 (max. –0.31)
DIF (treatment)	1 (item 15)	2 (items 26, 27)	0	0	0	0	0	0	1 (item 6)	0	0
Person measurement
PSI	0.92	0.85	0.88	0.93	0.88	0.88	0.93	0.89	0.93	0.92	0.86
Person fit range	–5.7/+5.1	–5.9/+2.9	–4.3/+3.9	–5.2/+4.0	–5.1/+3.2	–6.3/+3.6	–4.5/+3.7	–5.3/+3.8	–4.2/+3.6	–6.1/+4.0	–4.6/+3.6
< –2.5	214	236	46	137	165	131	203	148	216	130	247
> +2.5	92	21	6	32	11	39	25	23	32	38	17
Extremes	16	83	469	94	155	391	165	269	160	204	255
Person measures	–3.0/+5.2	–4.4/+4.2	–5.4/+5.9	–6.0/+5.8	–5.1/+ 4.6	–5.0/+4.7	–5.8/+5.8	–5.3/+4.8	–5.8/+5.3	–5.7/+5.0	–4.7/+4.3

All available scale completions were included in the analysis. The total number of measurements entered into the RMT analyses varied across the scales (2223–3686). These represent the person’s x time points and are influenced by the number of people dropping out over time, the frequency with which the scale is administered and whether or not scale completion was appropriate. For example, people who were unable to walk did not complete the walking scales.

The number of extremes varied across scales from a low of 16 (4.3%, MSIS-29phys) to a high of 429 (12.9%, MSWS-12v2). However, the highest percentage of extremes was 16.7% (391/2341, MSSS-88 spasms subscale). Extreme scores are similar to floor and ceiling effects. An extreme is person scale completion in which either all items have the minimum possible score or all items have the maximum possible score. As such, extremes are dependent on the number of items answered. However, extremes differ from floor and ceiling effects which are the percentage of person scale completions in which the maximum possible and minimum possible scale scores are achieved. Extremes are important because people at the extremes may have true changes in the target variables underestimated or not detected by the scale.

Scale-to-sample targeting

Scale-to-sample targeting was generally adequate. For all scales the spread of person measures exceeded the spread of item thresholds. This is a common finding because thresholds are the points on the continuum at which a person is likely to have a 50% chance of responding in adjacent response categories. There are two implications. Firstly, the CUPID sample was adequate for examining the performance of all 11 scales/subscales of the three MS-specific COAs. Secondly, this raises the possibility that all 11 scales may have the potential to underestimate the true change in target variables.

The shapes of the person measurement distributions of the individual scales/subscale were examined. Although all person measure distributions were generally Gaussian in nature, most distributions (9 out of 11) were skewed to some extent. The two non-skewed distributions were for the MSIS-29phys (Figure 24) and MSSS-88 muscle stiffness subscales. Six distributions were left skewed (towards less problems), three mildly (MSIS-29v2 psychological; MSSS-88 spasms and body movement), three moderately [MSSS-88 pain/discomfort, activities of daily living (ADL) and social function]. These scales have the potential to underestimate some improvements if they occur.

FIGURE 24.

Person-item threshold distribution for MSIS-29phys (grouping set to interval length of 0.20 making 60 groups).

Three person measure distributions were right skewed (towards more problems): one mildly (MSSS-88 feelings), one moderately (MSSS-88 walking) and one notably (MSWS-12v2; Figure 25). Two distributions were not skewed (MSIS-29phys and MSSS-88 stiffness). These scales have the potential to underestimate some worsening if they occur.

FIGURE 25.

Person-item threshold distribution for MSWS-12v2 (grouping set to interval length of 0.20 making 60 groups).

Scale performance (item analysis)

Response categories: to what extent did the item response categories work as intended?

Only one of the 11 scales/subscales (the MSWS-12v2) had any reversed thresholds and only one of the 12 items was affected (running). A look at the response category endorsement frequencies indicate a bimodal distribution with very few people responding to the middle category (sometimes limited). This may imply that people with MS can either run or they cannot and that further gradations may be empirically supported. This required further evaluation.

Mapped continuum: to what extent did the items map out a continuum on which people might be measured?

Item threshold location estimates for all 11 scales/subscales spread over good ranges (minimum 4.6 logits to maximum 8.1 logits) indicating that each item set (i.e. scale/subscale) mapped out a continuum on which people might be measured.

Item fit: to what extent do the items of a scale work together?

Item fit statistics indicated misfit for all scales. However, misfit was expected as the CUPID sample was large (range for item analysis, from 1950 to 3670) and fit statistics are sample size dependent. Therefore, we examined the implications of smaller samples. At adjusted sample sizes of n = 500 there was very little item misfit. Also, the ICCs showed adequate coherence between observed scores and predicted values. These findings implied that each of the 11 scales/subscales was made up of a statistically cohesive item set.

Item scoring bias: did responses to one item bias responses to others?

Across the 11 scales/subscales examined, very few correlations among residuals exceeded the recommended range of ± 0.30. This implied little item scoring bias.

Item stability: is item performance stable across important groups?

Item stability was examined across randomisation groups. There was good item performance stability (10 class intervals) and very little evidence of statistically significant instability despite the large sample sizes. Visual examination of the DIF plots implied that the statistically significant DIF that was detected was unlikely to be clinically meaningful.

Person measurement (person analysis)

The results of both the targeting analyses and the scale performance analyses implied that an examination of person measures was necessary.

Person separation: to what extent were people separated by the scales?

Person separation index for the 11 scales/subscales ranged from 0.85 to 0.93. This indicated that all 11 scales/subscales separated the people well within the sample in terms of the target constructs being measured. This provides a good basis for measurement in clinical trials.

Person fit statistics: how valid are person measurements?

For each scale, a number of individual person measurements had associated fit residuals outside the rule of thumb range of –2.5 to +2.5. This number was typically < 10% except for the MSSS-88 feelings subscale (12.7% of the samples). This indicated that for these scale completions the patterns of responses across the items was out of keeping with expectation. Fit residuals cannot be computed for people at the extremes.

For all 11 scales/subscales there were typically many more out of range fit residuals that were < –2.5 than were > +2.5. Person fit residuals < –2.5 imply response patterns that are more consistent than expected and tend to occur for people who have given the same response to all, or most, of the items in the set. In contrast, person fit residuals exceeding +2.5 can indicate response patterns that are clinically inconsistent.

Summary

Rasch measurement theory analyses of CUPID data for MSIS-29v2, MSWS-12v2 and MSSS-88 implied that performance was generally good. This means that it is legitimate to use the interval estimates for people and their SEs in subsequent analyses. The targeting plots raise some questions about whether or not some of the scales might underestimate changes and differences occurring in the study.

Section 3: evaluation of treatment effect per protocol

Results

Symptomatic effect (baseline to visit 5)

Tables 23–33 show the group and individual person change for each of the 11 scales/subscales. Changes in scores are computed as baseline minus visit 5 so that an improvement is a positive change and a worsening is a negative change. Results are very consistent across scales.

TABLE 23 - Multiple Sclerosis Impact Scale-29, 20-point physical subscale: symptomatic effect

Percentages may not sum to 100 due to rounding.

Analyses	Treatment group
	Placebo (n = 149)	Active (n = 266)
Descriptive statistics
Pre treatment (baseline), mean (SD)	0.543 (1.155)	0.539 (1.131)
On treatment (visit 5), mean (SD)	0.451 (1.110)	0.427 (1.180)
Change (baseline to visit 5), mean (SD)	0.0916 (0.899)	0.112 (0.881)
Magnitude of change – group level
One-way ANOVA, F-statistic (p-value)	0.052 (0.820)
Paired samples t-test, t-statistic (p-value)	1.243 (0.216)	2.078 (0.039)
Effect sizes
Cohen (mean change/SD pre treatment)	0.079	0.099
SRM (mean change/SD change)	0.102	0.127
Magnitude of change – individual person level
Better – significantly	16.8% (n = 25)	17.7% (n = 47)
Better – not significantly	27.5% (n = 41)	32.3% (n = 86)
No change	10.7% (n = 16)	7.5% (n = 20)
Worse – not significantly	36.2% (n = 54)	33.1% (n = 88)
Worse – significantly	8.7% (n = 13)	9.4% (n = 25)

TABLE 24 - Multiple Sclerosis Impact Scale-29, psychological impact subscale: symptomatic effect

Percentages may not sum to 100 due to rounding.

Analyses	Treatment group
	Placebo (n = 147)	Active (n = 265)
Descriptive statistics
Pre treatment (baseline), mean (SD)	–0.442 (1.458)	–0.410 (1.400)
On treatment (visit 5), mean (SD)	–0.461 (1.477)	–0.567 (1.400)
Change (baseline to visit 5), mean (SD)	0.0193 (1.1830)	0.157 (1.155)
Magnitude of change – group level
One-way ANOVA, F-statistic (p-value)	1.319 (0.251)
Paired samples t-test, t-statistic (p-value)	0.198 (0.843)	2.211 (0.028)
Effect sizes
Cohen (mean change/SD pre treatment)	0.013	0.112
SRM (mean change/SD change)	0.016	0.136
Magnitude of change – individual person level
Better – significantly	9.5% (n = 14)	9.8% (n = 26)
Better – not significantly	32.7% (n = 48)	44.5% (n = 118)
No change	10.9% (n = 16)	6.4% (n = 17)
Worse – not significantly	38.8% (n = 57)	31.7% (n = 84)
Worse – significantly	8.2% (n = 12)	7.5% (n = 20)

TABLE 25 - Multiple Sclerosis Walking Scale-12: symptomatic effect

Percentages may not sum to 100 due to rounding.

Analyses	Treatment group
	Placebo (n = 146)	Active (n = 257)
Descriptive statistics
Pre treatment (baseline), mean (SD)	2.573 (1.792)	2.626 (1.786)
On treatment (visit 5), mean (SD)	2.455 (1.901)	2.351 (2.078)
Change (baseline to visit 5), mean (SD)	0.118 (1.459)	0.275 (1.690)
Magnitude of change – group level
One-way ANOVA, F-statistic (p-value)	0.883 (0.348)
Paired samples t-test, t-statistic (p-value)	0.981 (0.328)	2.611 (0.010)
Effect sizes
Cohen (mean change/SD pre treatment)	0.066	0.154
SRM (mean change/SD change)	0.081	0.163
Magnitude of change – individual person level
Better – significantly	13.0% (n = 19)	21.8% (n = 56)
Better – not significantly	39.7% (n = 58)	29.2% (n = 75)
No change	6.8% (n = 10)	13.6% (n = 35)
Worse – not significantly	30.8% (n = 45)	25.7% (n = 66)
Worse – significantly	9.6% (n = 14)	9.7% (n = 25)

TABLE 26 - Multiple Sclerosis Spasticity Scale-88, muscle stiffness subscale, q01–12: symptomatic effect

Percentages may not sum to 100 due to rounding.

Analyses	Treatment group
	Placebo (n = 148)	Active (n = 262)
Descriptive statistics
Pre treatment (baseline), mean (SD)	–0.024 (2.115)	–0.125 (2.088)
On treatment (visit 5), mean (SD)	–0.384 (2.108)	–0.447 (2.220)
Change (baseline to visit 5), mean (SD)	0.360 (1.811)	0.321 (1.785)
Magnitude of change – group level
One-way ANOVA, F-statistic (p-value)	0.044 (0.835)
Paired samples t-test, t-statistic (p-value)	2.417 (0.170)	2.913 (0.004)
Effect sizes
Cohen (mean change/SD pre treatment)	0.170	0.154
SRM (mean change/SD change)	0.199	0.180
Magnitude of change – individual person level
Better – significantly	22.3% (n = 33)	20.2% (n = 53)
Better – not significantly	27.0% (n = 40)	34.0% (n = 89)
No change	6.8% (n = 10)	5.3% (n = 14)
Worse – not significantly	31.8% (n = 47)	26.0% (n = 68)
Worse – significantly	12.2% (n = 18)	14.5% (n = 38)

TABLE 27 - Multiple Sclerosis Spasticity Scale-88, pain/discomfort subscale, q13–21: symptomatic effect

Percentages may not sum to 100 due to rounding.

Analyses	Treatment group
	Placebo (n=148)	Active (n=264)
Descriptive statistics
Pre treatment (baseline), mean (SD)	–0.745 (1.934)	–0.906 (1.983)
On treatment (visit 5), mean (SD)	–1.137 (1.768)	–1.175 (1.899)
Change (baseline to visit 5), mean (SD)	0.393 (1.528)	0.269 (1.528)
Magnitude of change – group level
One-way ANOVA, F-statistic (p-value)	0.625 (0.430)
Paired samples t-test, t-statistic (p-value)	3.126 (0.002)	2.855 (0.005)
Effect sizes
Cohen (mean change/SD pre treatment)	0.203	0.135
SRM (mean change/SD change)	0.257	0.176
Magnitude of change – individual person level
Better – significantly	18.2% (n = 27)	15.5% (n = 41)
Better – not significantly	29.7% (n = 44)	36.7% (n = 97)
No change	10.1% (n = 15)	11.4% (n = 30)
Worse – not significantly	32.4% (n = 48)	26.9% (n = 71)
Worse – significantly	9.5% (n = 14)	9.5% (n = 25)

TABLE 28 - Multiple Sclerosis Spasticity Scale-88, muscle spasms subscale, q22–35: symptomatic effect

Percentages may not sum to 100 due to rounding.

Analyses	Treatment group
	Placebo (n = 148)	Active (n = 262)
Descriptive statistics
Pre treatment (baseline), mean (SD)	–2.053 (2.064)	–2.081 (1.905)
On treatment (visit 5), mean (SD)	–2.419 (1.803)	–2.331 (1.865)
Change (baseline to visit 5), mean (SD)	0.366 (1.713)	0.250 (1.388)
Magnitude of change – group level
One-way ANOVA, F-statistic (p-value)	0.555 (0.457)
Paired samples t-test, t-statistic (p-value)	2.599 (0.010)	2.916 (0.004)
Effect sizes
Cohen (mean change/SD pre treatment)	0.177	0.131
SRM (mean change/SD change)	0.214	0.180
Magnitude of change – individual person level
Better – significantly	15.5% (n = 23)	17.2% (n = 45)
Better – not significantly	30.4% (n = 45)	30.5% (n = 80)
No change	20.3% (n = 30)	16.8% (n = 44)
Worse – not significantly	24.3% (n = 36)	25.6% (n = 67)
Worse – significantly	9.5% (n = 14)	9.9% (n = 26)

TABLE 29 - Multiple Sclerosis Spasticity Scale-88, ADL subscale, q36–46: symptomatic effect

Percentages may not sum to 100 due to rounding.

Analyses	Treatment group
	Placebo (n = 148)	Active (n = 262)
Descriptive statistics
Pre treatment (baseline), mean (SD)	–1.111 (2.142)	–1.275 (2.108)
On treatment (visit 5), mean (SD)	–1.185 (2.227)	–1.474 (2.113)
Change (baseline to visit 5), mean (SD)	0.0739 (1.4350)	0.199 (1.624)
Magnitude of change – group level
One-way ANOVA, F-statistic (p-value)	0.614 (0.434)
Paired samples t-test, t-statistic (p-value)	0.626 (0.532)	1.988 (0.048)
Effect sizes
Cohen (mean change/SD pre treatment)	0.034	0.095
SRM (mean change/SD change)	0.051	0.123
Magnitude of change – individual person level
Better – significantly	14.9% (n = 22)	15.6% (n = 41)
Better – not significantly	33.8% (n = 50)	36.3% (n = 95)
No change	8.1% (n = 12)	10.7% (n = 28)
Worse – not significantly	32.4% (n = 48)	24.0% (n = 63)
Worse – significantly	10.8% (n = 16)	13.4% (n = 35)

TABLE 30 - Multiple Sclerosis Spasticity Scale-88, walking subscale, q47–56: symptomatic effect

Percentages may not sum to 100 due to rounding.

Analyses	Treatment group
	Placebo (n = 145)	Active (n = 255)
Descriptive statistics
Pre treatment (baseline), mean (SD)	1.385 (2.234)	1.196 (2.048)
On treatment (visit 5), mean (SD)	1.011 (2.098)	0.910 (2.304)
Change (baseline to visit 5), mean (SD)	0.374 (1.833)	0.286 (1.905)
Magnitude of change – group level
One-way ANOVA, F-statistic (p-value)	0.200 (0.655)
Paired samples t-test, t-statistic (p-value)	2.454 (0.015)	2.399 (0.017)
Effect sizes
Cohen (mean change/SD pre treatment)	0.167	0.140
SRM (mean change/SD change)	0.204	0.150
Magnitude of change – individual person level
Better – significantly	19.3% (n = 28)	19.2% (n = 49)
Better – not significantly	28.3% (n = 41)	30.2% (n = 77)
No change	20.0% (n = 29)	14.1% (n = 36)
Worse – not significantly	21.4% (n = 31)	27.1% (n = 69)
Worse – significantly	11.0% (n = 16)	9.4% (n = 24)

TABLE 31 - Multiple Sclerosis Spasticity Scale-88, body movements subscale, q57–67: symptomatic effect

Analyses	Treatment group
	Placebo (n = 147)	Active (n = 263)
Descriptive statistics
Pre treatment (baseline), mean (SD)	–0.143 (2.436)	–0.376 (2.262)
On treatment (visit 5), mean (SD)	–0.609 (2.260)	–0.629 (2.446)
Change (baseline to visit 5), mean (SD)	0.466 (1.885)	0.253 (2.045)
Magnitude of change – group level
One-way ANOVA, F-statistic (p-value)	1.079 (0.299)
Paired samples t-test, t-statistic (p-value)	2.995 (0.003)	2.006 (0.046)
Effect sizes
Cohen (mean change/SD pre treatment)	0.191	0.112
SRM (mean change/SD change)	0.247	0.124
Magnitude of change – individual person level
Better – significantly	25.2% (n = 37)	22.4% (n = 59)
Better – not significantly	25.2% (n = 37)	23.6% (n = 62)
No change	8.8% (n = 13)	9.5% (n = 25)
Worse – not significantly	28.6% (n = 42)	30.4% (n = 80)
Worse – significantly	12.2% (n = 18)	14.1% (n = 37)

TABLE 32 - Multiple Sclerosis Spasticity Scale-88, feelings subscale, q68–80: symptomatic effect

Analyses	Treatment group
	Placebo (n = 147)	Active (n = 261)
Descriptive statistics
Pre treatment (baseline), mean (SD)	–0.997 (2.200)	–1.023 (2.154)
On treatment (visit 5), mean (SD)	–1.270 (2.262)	–1.257 (2.343)
Change (baseline to visit 5), mean (SD)	0.273 (1.614)	0.234 (1.856)
Magnitude of change – group level
One-way ANOVA, F-statistic (p-value)	0.045 (0.832)
Paired samples t-test, t-statistic (p-value)	2.050 (0.042)	2.037 (0.043)
Effect sizes
Cohen (mean change/SD pre treatment)	0.124	0.109
SRM (mean change/SD change)	0.169	0.126
Magnitude of change – individual person level
Better – significantly	20.4% (n = 30)	19.9% (n = 52)
Better – not significantly	32.0% (n = 47)	30.3% (n = 79)
No change	14.3% (n = 21)	9.2% (n = 24)
Worse – not significantly	22.4% (n = 33)	23.4% (n = 61)
Worse – significantly	10.9% (n = 16)	17.2% (n = 45)

TABLE 33 - Multiple Sclerosis Spasticity Scale-88, social function subscale, q81–88: symptomatic effect

Percentages may not sum to 100 due to rounding.

Analyses	Treatment group
	Placebo (n = 146)	Active (n = 260)
Descriptive statistics
Pre treatment (baseline), mean (SD)	–0.792 (1.988)	–1.004 (1.865)
On treatment (visit 5), mean (SD)	–1.112 (1.993)	–1.061 (2.012)
Change (baseline to visit 5), mean (SD)	0.320 (1.700)	0.0568 (1.6470)
Magnitude of change – group level
One-way ANOVA, F-statistic (p-value)	2.328 (0.128)
Paired samples t-test, t-statistic (p-value)	2.272 (0.025)	0.556 (0.579)
Effect sizes
Cohen (mean change/SD pre treatment)	0.161	0.030
SRM (mean change/SD change)	0.188	0.034
Magnitude of change – individual person level
Better – significantly	19.9% (n = 29)	11.5% (n = 30)
Better – not significantly	26.7% (n = 39)	36.5% (n = 95)
No change	15.8% (n = 23)	11.9% (n = 31)
Worse – not significantly	28.8% (n = 42)	28.5% (n = 74)
Worse – significantly	8.9% (n = 13)	11.5% (n = 30)

Group-level changes

For all 11 scales, both active and placebo groups had positive mean change scores indicating an average improvement in all 11 target variables. However, the mean change scores were small and none of the changes for each group was statistically significant. There were no significant differences between the change scores for active and placebo.

The corresponding effect sizes were small, typically but not always < 0.20 (none exceeding 0.25) and generally similar for both active and placebo.

There were no clear trends at group level to imply that dronabinol, or placebo, may be superior.

There was no treatment effect on aspects of psychosocial function (MSIS-29v2 psychological impact subscale; MSSS-88 feelings subscale; and MSSS-88 psychosocial function subscale).

Individual person level change

The tables show that the proportions of people undergoing different degrees of change were relatively similar between active and placebo. None of the chi-squared values were significant.

Disease-modifying effect

Tables 34–44 show the group and individual person change for each of the 11 scales/subscales between baseline and the end of the CUPID study. Change scores are computed as baseline minus end so that an improvement is a positive change and a worsening is a negative change. As CUPID was a study of people with progressive MS we would expect there to be deterioration over time, on average, particularly in terms of motor-related functions and symptoms. Psychological functions can be affected by other factors, so it is hard to anticipate what would happen to these variables over time. However, if dronabinol has a mood enhancing effect, or simply makes people ‘feel better’, we might expect to see this reflected in the results.

TABLE 34 - Multiple Sclerosis Impact Scale-29, physical impact subscale: disease-modifying effect

Analyses	Treatment group
	Placebo (n = 112)	Active (n = 173)
Descriptive statistics
Pre treatment (baseline), mean (SD)	0.475 (1.202)	0.472 (1.143)
On treatment (end), mean (SD)	0.861 (1.509)	0.642 (1.189)
Change (baseline to end), mean (SD)	–0.3864 (1.3080)	–0.170 (1.070)
Magnitude of change – group level
One-way ANOVA, F-statistic (p-value)	2.324 (0.129)
Paired samples t-test, t-statistic (p-value)	–3.127 (0.002)	–2.091 (0.038)
Effect sizes
Cohen (mean change/SD pre treatment)	–0.321	–0.149
SRM (mean change/SD change)	–0.295	–0.159
Magnitude of change – individual person level
Better – significantly	10.7% (n = 12)	16.2% (n = 28)
Better – not significantly	23.2% (n = 26)	23.1% (n = 40)
No change	2.7% (n = 3)	3.5% (n = 6)
Worse – not significantly	33.9% (n = 38)	33.5% (n = 58)
Worse – significantly	29.5% (n = 33)	23.7% (n = 41)

TABLE 35 - Multiple Sclerosis Impact Scale-29, psychological impact subscale: disease-modifying effect

Percentages may not sum to 100 due to rounding.

Analyses	Treatment group
	Placebo (n = 109)	Active (n = 171)
Descriptive statistics
Pre treatment (baseline), mean (SD)	–0.539 (1.481)	–0.520 (1.397)
On treatment (end), mean (SD)	–0.488 (1.542)	–0.574 (1.383)
Change (baseline to end), mean (SD)	–0.050 (1.385)	0.055 (1.375)
Magnitude of change – group level
One-way ANOVA, F-statistic (p-value)	0.388 (0.534)
Paired samples t-test, t-statistic (p-value)	–0.379 (0.705)	0.522 (0.602)
Effect sizes
Cohen (mean change/SD pre treatment)	–0.034	0.039
SRM (mean change/SD change)	–0.036	0.040
Magnitude of change – individual person level
Better – significantly	9.2% (n = 10)	12.9% (n = 22)
Better – not significantly	33.9% (n = 37)	34.5% (n = 59)
No change	11.0% (n = 12)	8.8% (n = 15)
Worse – not significantly	35.8% (n = 39)	31.6% (n = 54)
Worse – significantly	10.1% (n = 11)	12.3% (n = 21)

TABLE 36 - Multiple Sclerosis Walking Scale-12: disease-modifying effect

Percentages may not sum to 100 due to rounding.

Analyses	Treatment group
	Placebo (n = 92)	Active (n = 147)
Descriptive statistics
Pre treatment (baseline), mean (SD)	2.391 (1.768)	2.452 (1.709)
On treatment (end), mean (SD)	2.891 (2.083)	2.831 (1.931)
Change (baseline to end), mean (SD)	–0.500 (2.052)	–0.378 (1.792)
Magnitude of change – group level
One-way ANOVA, F-statistic (p-value)	0.235 (0.629)
Paired samples t-test, t-statistic (p-value)	–2.338 (0.022)	–2.559 (0.012)
Effect sizes
Cohen (mean change/SD pre treatment)	–0.283	–0.221
SRM (mean change/SD change)	–0.244	–0.211
Magnitude of change – individual person level
Better – significantly	12.0% (n = 11)	10.9% (n = 16)
Better – not significantly	25.0% (n = 23)	27.9% (n = 41)
No change	8.7% (n = 8)	8.2% (n = 12)
Worse – not significantly	30.4% (n = 28)	35.4% (n = 52)
Worse – significantly	23.9% (n = 22)	17.7% (n = 26)

TABLE 37 - Multiple Sclerosis Spasticity Scale-88, muscle stiffness subscale, q01–12: disease-modifying effect

Percentages may not sum to 100 due to rounding.

Analyses	Treatment group
	Placebo (n = 104)	Active (n = 158)
Descriptive statistics
Pre treatment (baseline), mean (SD)	–0.174 (2.025)	–0.276 (2.120)
On treatment (end), mean (SD)	0.213 (2.179)	–0.154 (2.244)
Change (baseline to end), mean (SD)	–0.387 (2.254)	–0.122 (2.070)
Magnitude of change – group level
One-way ANOVA, F-statistic (p-value)	0.956 (0.329)
Paired samples t-test, t-statistic (p-value)	–1.751 (0.083)	–0.742 (0.459)
Effect sizes
Cohen (mean change/SD pre treatment)	–0.191	–0.058
SRM (mean change/SD change)	–0.172	–0.059
Magnitude of change – individual person level
Better – significantly	21.2% (n = 22)	20.9% (n = 33)
Better – not significantly	13.5% (n = 14)	22.8% (n = 36)
No change	2.9% (n = 3)	6.3% (n = 10)
Worse – not significantly	30.8% (n = 32)	32.3% (n = 51)
Worse – significantly	31.7% (n = 33)	17.7% (n = 28)

TABLE 38 - Multiple Sclerosis Spasticity Scale-88, pain/discomfort subscale, q13–21: disease-modifying effect

Percentages may not sum to 100 due to rounding.

Analyses	Treatment group
	Placebo (n = 104)	Active (n = 156)
Descriptive statistics
Pre treatment (baseline), mean (SD)	–0.745 (1.791)	–1.041 (1.992)
On treatment (end), mean (SD)	–0.805 (2.039)	–1.082 (1.925)
Change (baseline to end), mean (SD)	0.061 (1.907)	0.0418 (1.730)
Magnitude of change – group level
One-way ANOVA, F-statistic (p-value)	0.007 (0.935)
Paired samples t-test, t-statistic (p-value)	0.324 (0.747)	0.302 (0.763)
Effect sizes
Cohen (mean change/SD pre treatment)	0.034	0.021
SRM (mean change/SD change)	0.032	0.024
Magnitude of change – individual person level
Better – significantly	16.3% (n = 17)	14.7% (n = 23)
Better – not significantly	30.8% (n = 32)	26.3% (n = 41)
No change	6.7% (n = 7)	13.5% (n = 21)
Worse – not significantly	26.9% (n = 28)	31.4% (n = 49)
Worse – significantly	19.2% (n = 20)	14.1% (n = 22)

TABLE 39 - Multiple Sclerosis Spasticity Scale-88, muscle spasms subscale, q22–35: disease-modifying effect

Percentages may not sum to 100 due to rounding.

Analyses	Treatment group
	Placebo (n = 103)	Active (n = 156)
Descriptive statistics
Pre treatment (baseline), mean (SD)	–2.129 (2.086)	–2.108 (1.916)
On treatment (end), mean (SD)	–2.099 (1.894)	–2.105 (1.855)
Change (baseline to end), mean (SD)	–0.030 (2.071)	–0.004 (1.657)
Magnitude of change – group level
One-way ANOVA, F-statistic (p-value)	0.013 (0.911)
Paired samples t-test, t-statistic (p-value)	–0.146 (0.884)	–0.028 (0.978)
Effect sizes
Cohen (mean change/SD pre treatment)	–0.014	–0.002
SRM (mean change/SD change)	–0.014	–0.002
Magnitude of change – individual person level
Better – significantly	17.5% (n = 18)	20.5% (n = 32)
Better – not significantly	21.4% (n = 22)	22.4% (n = 35)
No change	11.7% (n = 12)	9.6% (n = 15)
Worse – not significantly	28.2% (n = 29)	33.3% (n = 52)
Worse – significantly	21.4% (n = 22)	14.1% (n = 22)

TABLE 40 - Multiple Sclerosis Spasticity Scale-88, ADL subscale, q36–46: disease-modifying effect

Analyses	Treatment group
	Placebo (n = 103)	Active (n = 154)
Descriptive statistics
Pre treatment (baseline), mean (SD)	–0.976 (2.091)	–1.446 (2.046)
On treatment (end), mean (SD)	–0.417 (2.865)	–1.213 (2.443)
Change (baseline to end), mean (SD)	–0.559 (2.228)	–0.233 (2.220)
Magnitude of change – group level
One-way ANOVA, F-statistic (p-value)	1.329 (0.250)
Paired samples t-test, t-statistic (p-value)	–2.547 (0.012)	–1.302 (0.195)
Effect sizes
Cohen (mean change/SD pre treatment)	–0.267	–0.114
SRM (mean change/SD change)	–0.251	–0.105
Magnitude of change – individual person level
Better – significantly	15.5% (n = 16)	18.2% (n = 28)
Better – not significantly	15.5% (n = 16)	25.3% (n = 39)
No change	5.8% (n = 6)	3.9% (n = 6)
Worse – not significantly	35.0% (n = 36)	26.6% (n = 41)
Worse – significantly	28.2% (n = 29)	26.0% (n = 40)

TABLE 41 - Multiple Sclerosis Spasticity Scale-88, walking subscale, q47–56: disease-modifying effect

Percentages may not sum to 100 due to rounding.

Analyses	Treatment group
	Placebo (n = 98)	Active (n = 145)
Descriptive statistics
Pre treatment (baseline), mean (SD)	1.352 (2.281)	0.790 (1.998)
On treatment (end), mean (SD)	1.335 (2.191)	1.001 (2.064)
Change (baseline to end), mean (SD)	0.017 (1.915)	–0.211 (2.220)
Magnitude of change – group level
One-way ANOVA, F-statistic (p-value)	0.685 (0.409)
Paired samples t-test, t-statistic (p-value)	0.085 (0.932)	–1.145 (0.254)
Effect sizes
Cohen (mean change/SD pre treatment)	0.007	–0.106
SRM (mean change/SD change)	0.009	–0.095
Magnitude of change – individual person level
Better – significantly	13.3% (n = 13)	19.3% (n = 28)
Better – not significantly	34.7% (n = 34)	21.4% (n = 31)
No change	9.2% (n = 9)	7.6% (n = 11)
Worse – not significantly	23.5% (n = 23)	31.7% (n = 46)
Worse – significantly	19.4% (n = 19)	20.0% (n = 29)

TABLE 42 - Multiple Sclerosis Spasticity Scale-88, body movements subscale, q57–67: disease-modifying effect

Percentages may not sum to 100 due to rounding.

Analyses	Treatment group
	Placebo (n = 100)	Active (n = 157)
Descriptive statistics
Pre treatment (baseline), mean (SD)	–0.058 (2.422)	–0.750 (2.190)
On treatment (end), mean (SD)	0.063 (2.299)	–0.507 (2.378)
Change (baseline to end), mean (SD)	–0.121 (2.181)	–0.242 (2.097)
Magnitude of change – group level
One-way ANOVA, F-statistic (p-value)	0.197 (0.657)
Paired samples t-test, t-statistic (p-value)	–0.557 (0.579)	–1.449 (0.149)
Effect sizes
Cohen (mean change/SD pre treatment)	–0.050	–0.111
SRM (mean change/SD change)	–0.056	–0.116
Magnitude of change – individual person level
Better – significantly	19.0% (n = 19)	17.8% (n = 28)
Better – not significantly	26.0% (n = 26)	25.5% (n = 40)
No change	5.0% (n = 5)	7.6% (n = 12)
Worse – not significantly	25.0% (n = 25)	28.0% (n = 44)
Worse – significantly	25.0% (n = 25)	21.0% (n = 33)

TABLE 43 - Multiple Sclerosis Spasticity Scale-88, feelings subscale, q68–80: disease-modifying effect

Analyses	Treatment group
	Placebo (n = 102)	Active (n = 156)
Descriptive statistics
Pre treatment (baseline), mean (SD)	–0.858 (2.106)	–1.182 (2.141)
On treatment (end), mean (SD)	–1.341 (2.215)	–1.349 (2.164)
Change (baseline to end), mean (SD)	0.482 (2.002)	0.167 (2.181)
Magnitude of change – group level
One-way ANOVA, F-statistic (p-value)	1.372 (0.243)
Paired samples t-test, t-statistic (p-value)	2.434 (0.017)	0.959 (0.339)
Effect sizes
Cohen (mean change/SD pre treatment)	0.229	0.078
SRM (mean change/SD change)	0.241	0.077
Magnitude of change – individual person level
Better – significantly	25.5% (n = 26)	25.6% (n = 40)
Better – not significantly	37.3% (n = 38)	26.3% (n = 41)
No change	3.9% (n = 4)	7.1% (n = 11)
Worse – not significantly	18.6% (n = 19)	19.2% (n = 30)
Worse – significantly	14.7% (n = 15)	21.8% (n = 34)

TABLE 44 - Multiple Sclerosis Spasticity Scale-88, social function subscale, q81–88: disease-modifying effect

Percentages may not sum to 100 due to rounding.

Analyses	Treatment group
	Placebo (n = 104)	Active (n = 155)
Descriptive statistics
Pre treatment (baseline), mean (SD)	–0.783 (1.963)	–1.286 (1.816)
On treatment (end), mean (SD)	–1.228 (2.123)	–1.271 (1.995)
Change (baseline to end), mean (SD)	0.445 (2.066)	–0.014 (1.953)
Magnitude of change – group level
One-way ANOVA, F-statistic (p-value)	3.287 (0.071)
Paired samples t-test, t-statistic (p-value)	2.197 (0.030)	–0.091 (0.928)
Effect sizes
Cohen (mean change/SD pre treatment)	0.227	–0.008
SRM (mean change/SD change)	0.215	–0.007
Magnitude of change – individual person level
Better – significantly	24.0% (n = 25)	18.1% (n = 28)
Better – not significantly	26.0% (n = 27)	29.7% (n = 46)
No change	15.4% (n = 16)	6.5% (n = 10)
Worse – not significantly	20.2% (n = 21)	26.5% (n = 41)
Worse – significantly	14.4% (n = 15)	19.4% (n = 30)

Group-level changes

For eight of the 11 scales/subscales there is no real difference between active- and placebo-treated people.

For three of 11 scales/subscales (MSIS-29phys; MSSS-88 muscle stiffness and ADL subscales), there was a suggestion that people treated with dronabinol have deteriorated less than people treated with placebo. These changes within the placebo group and within the active group, and between placebo and active, were not significant statistically.

Effect size calculations implied that the deterioration in placebo-treated people exceeded the threshold for small clinical worsening (< –0.20) for two subscales (MSIS-29phys and MSSS-88 ADL subscales) and is borderline small for the third (MSSS-88 muscle stiffness). The corresponding effect size calculations for dronabinol-treated people are approximately half the magnitude. This hinted at the possibility of a disease-modifying treatment effect.

There was no treatment effect on aspects of psychosocial function (MSIS-29v2 psychological impact subscale; MSSS-88 feelings subscale; or MSSS-88 psychosocial function subscale).

There are two notable findings. First, the mean changes over a considerable period of time are very small. This implies only clinically small progression during the study in a large cohort of people with a diagnosis of progressive MS. Second was the high dropout rate. The CUPID study recruited 493 people, randomising 329 to active and 164 to placebo. By the end of the study the maximum number of people completing a PRO was 173 in the active group, 53% of the original dronabinol-treated cohort and 112 in the placebo group, 68% of the original placebo-treated cohort.

Individual person-level change

The proportion of people in each of the five change groups is very similar for placebo and active although, as expected, this changes across scales. The similarity between placebo and active is notable even for the three scales/subscales where the group analyses hint at a disease-modifying treatment effect.

Perhaps the most striking findings from the individual person-level analysis are the proportions of people who appear to have improved by the end of the study. In the placebo group, this proportion ranged from 31% (MSSS-88 ADL subscale) to 63% (MSSS-88 feelings subscale). The proportion of people categorised as having a significant improvement ranged from 9% (MSIS-29v2 psychological impact scale) to 26% (MSSS-88 feeling subscale), with a mean of 17%. This warrants further examination.

For illustration, Figures 26 and 27 show the SRMs for the two MSIS-29v2 subscales and the individual person change for the MSIS-29phys.

FIGURE 26.

Standardised response means for the two MSIS-29v2 subscales, by treatment group. (a) MSIS-29v2 physical impact subscale; and (b) MSIS-29v2 psychological subscale.

FIGURE 27.

Individual-person change for the MSIS-29phys, by treatment group.

Section 4: exploratory evaluation of treatment effect by baseline disability level

Analyses presented earlier in this report hinted towards a treatment effect in people with lower EDSS scores at baseline and limited progression in participants with high EDSS scores. For this reason we explored the changes in the MS-specific PROs of people who were less disabled (baseline EDSS score of 4.0–5.5) and those who were more disabled (baseline EDSS score of 6.0–6.5). These analyses were undertaken in the full knowledge of being exploratory post-hoc analyses in non-randomised groups. As such, any results should be interpreted with caution.

Results

Symptomatic effect

Table 45 shows the results of analyses to determine evidence of a symptomatic effect. Sample sizes of the comparison groups varied notably, with the more disabled subgroup being far larger that the less disabled subgroup.

TABLE 45 - Examination of symptomatic treatment effect in baseline EDSS score-defined subgroups

ES, effect size.

Scale	Subscale	Treatment	Baseline EDSS subgroup
			EDSS score of 4.0–5.5				EDSS score of 6.0–6.5
			n	t-statistic (p-value)	Cohen’s ES	SRM	n	t-statistic (p-value)	Cohen’s ES	SRM
MSIS-29v2	Physical	Active	68	1.887 (0.063)	+0.181	+0.229	222	1.215 (0.226)	+0.067	+0.082
		Placebo	33	1.650 (0.190)	+0.245	+0.287	121	0.663 (0.509)	+0.046	+0.060
	Psychological	Active	68	2.365 (0.021)	+0.246	+0.287	221	0.738 (0.462)	+0.041	+0.050
		Placebo	33	1.597 (0.120)	+0.291	+0.278	118	–0.388 (0.700)	–0.028	–0.036
MSWS-12v2		Active	68	3.331 (0.001)	+0.357	+0.404	212	1.249 (0.213)	+0.088	+0.086
		Placebo	33	2.894 (0.007)	+0.310	+0.504	11	0.0375 (0.708)	+0.031	+0.035
MSSS-88	Stiffness	Active	67	1.159 (0.251)	+0.114	+0.142	217	2.309 (0.022)	+0.147	+0.157
		Placebo	33	1.930 (0.062)	+0.239	+0.336	120	1.656 (0.100)	+0.135	+0.151
	Pain and discomfort	Active	54	0.955 (0.343)	+0.085	+0.116	219	2.326 (0.021)	+0.129	+0.157
		Placebo	33	2.345 (0.025)	+0.321	+0.408	120	2.504 (0.014)	+0.176	+0.229
	Spasms	Active	67	1.602 (0.292)	+0.097	+0.130	218	2.621 (0.009)	+0.135	+0.178
		Placebo	33	0.213 (0.833)	+0.034	+0.037	120	2.937 (0.004)	+0.212	+0.268
	ADL	Active	67	0.057 (0.955)	+0.004	+0.007	218	1.543 (0.124)	+0.091	+0.105
		Placebo	33	0.641 (0.526)	+0.066	+0.112	120	0.084 (0.933)	+0.005	+0.008
	Walking	Active	67	0.6330 (0.0529)	+0.056	+0.077	211	1.814 (0.071)	+0.126	+0.125
		Placebo	33	2.391 (0.023)	+0.347	+0.416	117	1.178 (0.241)	+0.087	+0.109
	Body movements	Active	67	0.394 (0.695)	+0.035	+0.048	219	1.603 (0.110)	+0.110	+0.108
		Placebo	33	2.044 (0.049)	+0.177	+0.356	119	2.422 (0.017)	+0.186	+0.222
	Feelings	Active	67	1.757 (0.083)	+0.166	+0.215	217	1.219 (0.224)	+0.071	+0.083
		Placebo	32	1.604 (0.119)	+0.172	+0.284	120	1.097 (0.275)	+0.078	+0.100
	Social function	Active	67	0.392 (0.696)	+0.045	+0.048	216	0.510 (0.611)	+0.030	+0.035
		Placebo	32	1.694 (0.100)	+0.250	+0.300	119	1.751 (0.083)	+0.139	+0.161

Results for all 11 scales/subscales implied an improvement at visit 5 relative to visit 1. No t-statistics were significant at the p < 0.001 level (no Bonferroni correction). Effect sizes did not suggest any patterns in the data in terms of the comparison between active treatment in the less and more disabled subgroups. Some scales had larger effect sizes in the less disabled subgroup, some had larger effect sizes in the more disabled subgroup, and some were equal.

The comparisons of active with placebo were similar for most scales. Five MSSS-88 subscales (stiffness, pain, walking, body movements and social function) recorded notably larger benefits in the placebo group than in the active group among the less disabled cohort.

The two effect sizes (Cohen’s effect size and SRM) produced different results, as we have noted in our previous work.

Disease-modifying effect

Table 46 shows the results of analyses to determine evidence of a disease-modifying effect. Again, sample sizes of the comparison groups varied notably, with the more disabled subgroup being far larger that the less disabled subgroup. The sample size for the less disabled people on placebo was particularly small (between 23 and 29).

TABLE 46 - Examination of disease-modifying treatment effect in baseline EDSS-defined subgroups

ES, effect size.

Scale	Subscale	Treatment	Baseline EDSS subgroup
			EDSS score of 4.0–5.5				EDSS score of 6.0–6.5
			n	t-statistic (p-value)	Cohen’s ES	SRM	n	t-statistic (p-value)	Cohen’s ES	SRM
MSIS-29v2	Physical	Active	61	–1.191 (0.238)	–0.142	–0.153	191	–2.940 (0.004)	–0.202	–0.213
		Placebo	29	–2.326 (0.027)	–0.431	–0.432	111	–2.770 (0.007)	–0.297	–0.263
	Psychological	Active	60	0.856 (0.395)	+0.095	+0.111	189	0.390 (0.697)	+0.028	+0.029
		Placebo	29	0.665 (0.512)	+0.187	+0.123	108	–0.187 (0.852)	–0.016	–0.018
MSWS-12v2	Walking	Active	54	–1.622 (0.111)	–0.212	–0.221	147	–2.870 (0.005)	–0.274	–0.237
		Placebo	23	–2.978 (0.007)	–0.880	–0.621	88	–1.683 (0.096)	–0.210	–0.179
MSSS-88	Stiffness	Active	54	–1.884 (0.065)	–0.260	–0.256	170	–1.160 (0.248)	–0.086	–0.089
		Placebo	24	–0.677 (0.505)	–0.123	–0.138	102	–1.640 (0.140)	–0.191	–0.162
	Pain and discomfort	Active	54	–0.927 (0.358)	–0.108	–0.126	169	0.780 (0.437)	+0.051	+0.060
		Placebo	24	–0.097 (0.923)	–0.019	–0.020	102	0.401 (0.690)	+0.043	+0.040
	Spasms	Active	53	–1.111(0.272)	–0.147	–0.153	171	0.446 (0.656)	+0.031	+0.034
		Placebo	24	–0.786 (0.440)	–0.139	–0.160	101	0.118 (0.906)	+0.012	+0.012
	ADL	Active	53	–2.613 (0.012)	–0.372	–0.359	169	–2.500 (0.013)	–0.238	–0.192
		Placebo	24	–1.376 (0.182)	–0.254	–0.281	101	–2.981 (0.004)	–0.344	–0.297
	Walking	Active	53	–1.917 (0.061)	–0.244	–0.263	148	–1.035 (0.302)	–0.093	–0.085
		Placebo	24	0.002 (0.998)	0.0004	0.0004	93	0.370 (0.712)	+0.033	+0.038
	Body movements	Active	54	–2.565 (0.013)	–0.281	–0.349	171	–1.924 (0.056)	–0.147	–0.152
		Placebo	24	–1.752 (0.093)	–0.285	–0.358	98	–0.335 (0.738)	–0.032	–0.034
	Feelings	Active	54	1.100 (0.277)	+0.132	+0.150	170	1.077 (0.283)	+0.084	+0.083
		Placebo	23	–0.200 (0.843)	–0.052	–0.042	100	2.612 (0.010)	+0.253	+0.261
	Social function	Active	54	0.593 (0.555)	+0.084	+0.081	169	–0.292 (0.770)	–0.022	–0.022
		Placebo	23	–0.348 (0.731)	–0.089	–0.073	102	2.473 (0.015)	+0.258	+0.245

In the less disabled subsample, change scores were mostly negative, implying a worsening in these outcomes between the beginning and end of the study. Effect sizes for placebo and active were similar for eight scales. For two scales/subscales (MSWS-12v2; MSIS-29phys) there was notably less worsening in the active than the placebo group. In contrast, there was greater worsening in the active group for the MSSS-88 walking subscale.

In the more disabled group, effect sizes for both active and placebo were generally similar across the scales. It is notable that the group-based deterioration in physical function in the placebo group of the more disabled group was surprisingly small over the 3 years of the study, with the largest change being considered small by Cohen’s criteria.

Figures 28 and 29 show the SRM plots for the MSIS-29phys and MSWS-12v2, in the two subgroups defined by baseline EDSS score.

FIGURE 28.

Standardised response means for the MSIS-29phys, by treatment group and baseline EDSS score subgroup. (a) Baseline EDSS score of 4.0–5.5; and (b) baseline EDSS score of 6.0–6.5. Effect size difference (SRM) = 0.28; effect size difference (Cohen’s) = 0.29.

FIGURE 29.

Standardised response means for the MSWS-12v2, by treatment group and baseline EDSS score subgroup. (a) Baseline EDSS score of 4.0–5.5; and (b) baseline EDSS score of 6.0–6.5. Effect size difference (SRM) = 0.40; effect size difference (Cohen’s) = 0.67.

Finally, individual person changes were examined (computed as described previously), for the MSIS-29v2 and MSWS-12v2. These results are shown in Figures 30 and 31. For the MSIS-29phys, the plot shows less difference between the two disability groups than is implied by the effect sizes. For the MSWS-12v2, the differences between the two disability groups are notable.

FIGURE 30.

Individual-person change for the MSIS-29v2phys, in each treatment group, by disability subgroup. (a) Baseline EDSS score of 4.0–5.5; and (b) baseline EDSS score of 6.0–6.5.

FIGURE 31.

Individual-person change for the MSWS-12v2, in each treatment group, by disability subgroup. (a) Baseline EDSS score of 4.0–5.5; and (b) baseline EDSS score of 6.0–6.5.

Section 5: reflections and lessons learned from Rasch measurement theory analysis of Cannabinoid Use in Progressive Inflammatory brain Disease data

This chapter has concerned the application of the modern, sophisticated psychometric method RMT to data generated by MS-specific PRO measures used in a large multicentre Phase III pivotal clinical trial. Historically, despite its availability since the 1960s, RMT has been used frugally before. There may be two main reasons for this: the inaccessibility of RMT to clinicians, and the lack of emphasis on COAs as primary and secondary end points. The first reason is changing as RMT is becoming more widely known, understood and used. The second reason is changing, as there is increasing recognition of the importance of measuring patient-focused outcomes. In particular, the US Food and Drug Administration (FDA) has emphasised the importance of patient-focused COA and the use of PROs. Indeed, recently, two important figures related to COAs in general – the road map to patient-focused outcome assessment (Figure 32) and the wheel and spokes diagram for the qualification of COAs (Figure 33) – have appeared on their website.

FIGURE 32.

Roadmap to patient-focused outcome measurement in clinical trials. US FDA. COA Roadmap Diagram. URL: www.fda.gov/downloads/drugs/developmentapprovalprocess/drugdevelopmenttoolsqualificationprogram/ucm370174.pdf (accessed 29 April 2014).

FIGURE 33.

Qualification of COAs. US FDA. Qualification of Clinical Outcome Assessments (COAs). URL: www.fda.gov/downloads/Drugs/DevelopmentApprovalProcess/DrugDevelopmentToolsQualificationProgram/UCM370175.pdf (accessed 29 April 2014).

The evaluation of scale performance was informative. Analyses show the scales performed adequately, although the suboptimal targeting for some scales/subscales means there is room for improvement. It is difficult to quantify the implication of limitations in scale performance. However, our findings mean that we can be relatively confident in the interpretation of subsequent analyses.

The main finding of the study was that Δ⁹-THC does not appear to have either a symptomatic or a disease-modifying effect as measured using these MS-specific PROs. There were some suggestions of improvement but no consistent findings. It seems that the chance of detecting a disease-modifying effect, if present, was slim as the progression in the sample, on average, was statistically and clinically small. Also, a reasonably large proportion of people appeared to improve their function over time, which is surprising for people diagnosed as having a chronic progressive disabling incurable neurological disease.

These findings raised questions of the completeness of our understanding of progressive MS and the need for more basic research to understand exactly what is progressing in progressive MS.

Introduction

The aim of the economic evaluation was to compare the costs and consequences of cannabinoids (Δ⁹-THC) with those of usual care in progressive MS in the UK outpatient setting. The primary analytical perspective was the NHS and Personal and Social Services (PSS), with the patient perspective considered in secondary analyses. The primary health economic outcome was the quality-adjusted life-year (QALY) estimated using the EQ-5D18 and the primary economic end point was the 36-month follow-up. Costs and QALYs were discounted after the first year at the UK treasury rate of 3.5%.

Methods