The feasibility of determining the effectiveness and cost-effectiveness of medication organisation devices compared with usual care for older people in a community setting: systematic review, stakeholder focus groups and feasibility randomised controlled trial

Adherence measurement

Studies of MODs have frequently focused on a specific disease, with a single therapeutic outcome or detection of particular chemicals in body fluids being used as a measure of adherence. Thus, little guidance is available to guide targeting of the wider population of patients for MODs in routine practice. Historically, direct measures of adherence, such as observation and detection of chemicals in bodily fluids, have been considered the gold standard. Observation clearly has significant cost implications for large-scale studies and is subject to the Hawthorne effect. Detection of chemicals in body fluids has the merit of being objective; however, it can be invasive and costly, and there remains the potential for patients to alter their medication-taking behaviour in the days prior to sample provision. Some of the disadvantages associated with observation as an adherence measure are exemplified by a trial which randomised patients to receive potassium supplementation or placebo tablets. 23 Measurement of urine potassium levels identified a reduction over time, which was most likely attributable to reduced patient compliance with 24-hour urine sample collection as the trial progressed; hence measured potassium levels were artificially low. The taking of blood samples would overcome the issues of patient compliance with inconvenient 24-hour urine samples; however, the acceptability to patients of frequent blood samples is even lower and has been demonstrated to adversely affect trial recruitment, with 52% of patients not consenting to trial participation, citing fear of phlebotomy. 24 An additional problem associated with such direct measures is intra- and interpatient variability in drug handling. This can be overcome to a certain extent by estimating individual variation in drug handling via repeated samples over a short period of time; however, this type of invasive assessment has low patient acceptability. Alternatively, Bayesian methodology can be used; however, this is complex and again provides only an estimate of variability.

The dosage unit count (DUC) is generally accepted as the pragmatic approach to adherence assessment. It is based on the assumption that, if the medication is not in the container, it has been taken by the patient. This is problematic when attempting to identify intentional non-adherence because patients may deliberately remove and discard tablets in order to disguise non-adherence. However, the assumption is valid if patients are predominantly unintentionally non-adherent. Previous research has demonstrated that use of the DUC method in the older patient population is feasible and acceptable. 15

More recently, electronic adherence monitoring (EAM) systems have enabled objective measurement of medication adherence that is less susceptible to the Hawthorne effect by virtue of being less intrusive and less conspicuous to the patient than direct adherence measures or DUC. The most widely used EAM system is the Medication Event Monitoring System™ (MWV Healthcare, Richmond, VA, USA). The Medication Event Monitoring System is a bottle thats cap contains a microprocessor that records the date and time of each bottle opening event; it has been widely used in clinical trials to assess medication adherence. 12,23,25,26 Trials have, therefore, generally approached objective adherence monitoring by decanting medication from usual packaging to the monitoring bottles. This has the limitation of not assessing adherence in a naturalistic setting, as usual dispensing is now routinely in manufacturer-issued blister packaging. A number of EAM systems are under development, and a 2-month pilot study (n = 52) of one EAM system to assess feasibility and acceptability for usual-care blister packaging reported promising results. Adherence data were obtained from 94.3% of participants and 67.4% of participants reported that they would consider using the EAM system for a long-term study. 27 Therefore, it was proposed that bespoke versions of an EAM system be evaluated to determine whether or not an EAM system can enable medication-taking events from MODs and ‘usual-care’ blister packs to be objectively and accurately recorded and compared.

Cost-effectiveness

In addition to assessing whether or not MOD provision affects adherence to medication regimes, it will be important to assess whether or not it confers health or economic benefits. Measures to detect changes in the utilisation of health or social services will be put in place and tested to determine utility for a definitive study.

Patient autonomy

In addition to the impact of MODs on adherence, it is important to establish patient acceptability. No studies have reported the impact of MODs on patient autonomy or ability to manage one’s own medication; however, there is anecdotal evidence of reduced autonomy, as patients are unable to differentiate one medication from another when medicines are packed in MODs and, therefore, cannot choose to omit a single type of medication if they so desire (e.g. to delay taking a diuretic when embarking on a long journey), sometimes resulting in the omission of all doses. 1 Conversely, patients may report that they feel enabled by feeling confident about managing their medication. A number of studies have explored patient autonomy with respect to medication taking in the context of describing the extent to which patients feel involved in the decision-making process. 28–30 However, exploration of whether or not patients feel as though they have some control over the medication-taking process is limited. Development of a tool to assess the impact of MODs on patients’ confidence in their ability to manage their own medication, on their satisfaction with their packaging and the service they receive was included. Carers and relatives are closely involved in the daily lives of older people with multiple comorbidities and, therefore, it is also important to assess the impact of providing MODs to older people on their carers or close relatives. An additional tool to determine carer perceptions of the impact of medicine packaging on patients’ confidence and their ability to manage their medicines and effects on the resource provided by carers and relatives was also included.

Patient medication administration errors

Studies evaluating patient medication administration errors have cited access to extra medication as a source of errors and, therefore, reducing the amount of medication to which a patient has access may also be a further source of error reduction. 31 For the duration of the study it was important for participants to have access only to the new stock of drugs supplied to them. Therefore, development of an appropriate process to ensure that errors in compliance could not be attributed to extra medication was included.

General dispensing and administration errors

The most substantial review to date of errors associated with MOD use was conducted in Australia; the Australian Incident Monitoring Study31 reported that 0.43% (52 out of 12,000) of medication-related errors were associated with MODs. In 26 cases, there was a problem with filling the MOD, such as wrong dose, dose omission or wrong medication. In 21 of these cases, nursing staff were responsible for the error, with the remainder being attributable to pharmacy staff or a carer. On 16 occasions problems using the MOD were cited as a reason for an error; however, the nature of these problems was not reported. Factors contributing to the reported problems included patient confusion/distraction and the MOD being inappropriate for the patient. 31 A further Australian audit32 of dispensing errors associated with 6972 dispensed MODs detected an error rate of 4.3%. 32

A 2007 UK evaluation of dispensing error rate associated with the pharmacy usual dispensing process33 reported 1.7% content errors out of 2859 dispensed items. Content errors were errors of omission, incorrect drug, incorrect strength, incorrect dosage form, added or missing dose units and expired medication. A similar US-based study conducted in 2003 reported an identical 1.7% error rate. 33 While general dispensing error rates are 1.7%, there are no UK data for MOD error rates and, therefore, it is necessary to record error rates for dispensing into both MODs and usual packaging.

In summary, comparative data in terms of error rate associated with MODs and usual dispensing are not available. Data regarding the incidence of dispensing errors associated with usual dispensing are available, but may not be generalisable to prescriptions assembled for an older population with more multiple items. A reasonable estimate of error rate requires a large sample size and significant resources. Within a relatively small-scale feasibility study it is, therefore, only appropriate to test tools designed to collect error data and to quantify and describe any identified errors.

Article selection

Information sources

The following electronic databases were searched. Relevant reviews were identified in order to access their bibliographies (see the following for augmented searches).

• The library of the Cochrane collaboration (www.thecochranelibrary.com) including the databases Cochrane Database of Systematic Reviews, Cochrane Central Register of Controlled Trials, Cochrane Methodology Register, Database of Abstracts of Reviews of Effects, HTA Database, NHS Economic Evaluation Database (searched from inception to 2012).

• MEDLINE (via Ovid; searched from 1966 to 2012).

• EMBASE (via Ovid; searched from 1980 to 2012).

• PsycINFO (via Ovid; searched from 1874 to 2012).

• Allied and Complementary Medicine Database (AMED; via Ovid; searched from 1985 to 2012).

• Cumulative Index to Nursing and Allied Health Literature (CINAHL; via EBSCOhost; searched from 1982 to 2012).

• Trials listed as complete in Current Controlled Trials (http://controlled-trials.com/; searched from inception to 2012).

• York Centre for Review and Dissemination databases (Database of Abstracts of Reviews of Effects, NHS Economic Evaluation Database and the HTA database; www.crd.york.ac.uk/crdweb/SearchPage.asp; searched from inception to 2012).

Search terms of medical subject headings, free text and trade names (see Appendix 1) were applied to title, abstract and whole text of articles within the Ovid search tool. For databases outside the Ovid platform, the same terms were utilised but with appropriate translation of syntax (truncation, wildcards, adjacency and Boolean operators). No restrictions were placed on year of publication and the full back-catalogues of the available databases were searched up to the end of 2012; the first search was run on 8 February 2012 and the search was updated on 11 January 2013.

In addition to formal searching of the above databases, searching was augmented via:

• keyword searches in the Google Scholar™ search engine (http://scholar.google.com)

• hand-searching of reference lists of included articles

• hand-searching of identified review articles

• hand-searching of reference lists of articles relevant to the project but excluded because of exclusion criteria (e.g. compared MODs to non-standard care, used additional interventions or measured adherence only via self-report)

• personal communication with packaging companies

• personal communication with research groups with an expertise in adherence.

Study selection

Titles and abstracts of all identified articles were screened for relevance by two reviewers (CA, LC, EP or TB) with any disagreements resolved by discussion. When agreement could not be reached, a third reviewer moderated the final decision. Full-text articles were similarly screened by two reviewers (CA, EP or TB). In the case of articles identified as potentially relevant after full-text screening, data were independently extracted by two reviewers (EP and SW), with any differences resolved by discussion. Authors conducting the review were not blinded to any element of the identified articles.

Data collection process

Data were extracted independently by two reviewers (EP and SW) using a standardised Microsoft Excel (Microsoft Corporation, Redmond, WA, USA) spreadsheet. Table 1 provides a summary of the data that were extracted to the spreadsheet.

Source of funding

Sources of funding were examined to identify any potential conflict of interest.

Risk of bias

Risk of bias was assessed using items from the Cochrane risk of bias tool (randomisation procedure, concealment of allocation, blinding of assessors, blinding of treatment providers, whether or not attrition was variable between groups and how this was accounted for, whether or not all outcomes were reported, and whether or not there were any other clear threats to validity) as appropriate to the study design. When the study design did not permit randomisation or blinding of assessors, it was noted that such designs have greater risk of bias.

Summary measures

Meta-analysis was intended where measures were consistently reported across trials; however, owing to a lack of comparability, none was performed. Results are tabulated and described according to study outcomes.

Study selection

A total of 8122 studies were identified. After automatic removal of duplicates, 5100 articles were screened. Following this, 272 articles were retained for full-text screening. Three authors (CA, EP and TB) assessed these texts for eligibility and 10 articles were included. Screening of identified review articles and included articles, and those retrieved for detailed review, identified a further 40 potentially relevant studies, which yielded an additional six studies. A flow diagram summarising this process is detailed in Figure 2. No studies declared any conflict of interest.

FIGURE 2.

Flow diagram of articles included in review.

Overview of studies

A summary of included studies is shown in Table 2. Of the 16 included studies, seven were undertaken in the USA, five in the UK and four in Australia and New Zealand. Six of the 10 non-US studies were identified via augmented searching, indicating a possible US bias in the availability of articles via academic databases.

Eight studies were conducted in 2000 or later, and the majority were in domiciliary settings (n = 12). Seven studies did not target a specific illness and the remaining studies targeted hypertension,3 diabetes mellitus, mental health, cancer and human immunodeficiency virus (HIV) infection. One study44 examined the effect of MODs on dispensing procedures only. Nine studies were RCTs. Two studies were audits and so did not assess any adherence behaviour. 31,32 The combined population from all the studies was 1541.

The type and number of MODs assessed varied across studies. Unsealed reusable MODs were used in five studies40,42,49,52,54 and pre-sealed units were used in five studies;42,44,50 in five studies,31,43,45,50,51 the seal type was not specified. One study52 used multiple types of MODs.

Only one study used patient-filled MODs,2 although in Feetam and Kelly42 it was unclear who filled the MODs. In other studies, the units were filled before being supplied to the patient by researchers in one study,43 were filled by an automatic filling device in one study,40 were filled by community nurses in one study51 and were filled by unspecified professionals in the remaining studies. McElnay and Thompson44 timed the filling of MODs by five pharmacists and a pharmacy technician.

Figure 3 provides the number of studies published over each 5-year period. It can be seen that the frequency of relevant studies has dramatically increased over the past 10 years.

FIGURE 3.

Frequency of studies published per 5-year period.

Risk of bias

There was considerable risk of bias in many studies, which is summarised in Table 3. Only three of the nine RCT studies included information on randomisation,2,43,49 and none mentioned blinding. Other risks of bias identified included baseline differences between groups not accounted for in analysis,40,53 minimal participant information42,52 and minimal information on analysis. 42 Levings et al. 31 performed an audit of errors reported in the Australian Incident Monitoring Study and noted that some types of errors may be more likely to be reported than others (e.g. overdose). It is therefore difficult to determine whether or not the errors identified are representative of all errors. McElnay and Thompson44 determined the time taken to fill MODs, but they used fictional patients with only three medicines which may have led to an underestimation of the actual time required to fill.

Effects of medication organisation devices

Method

A review of peer-reviewed and grey literature using appropriate search terms to identify existing knowledge regarding the key design features was supplemented with expert opinion. The design elements investigated were identification of appropriate participants; characterisation of participants’ cognitive, manual and visual ability; selection of appropriate medication organisation devices; definition of standard care and the intervention; objective adherence monitoring; optimum recruitment strategies; and a suitable database.

Results

Selection of appropriate medication organisation devices

A survey of pharmacies to establish the MODs that are supplied had previously been undertaken by DB. This identified three MODs most frequently used: dosette boxes, Nomad Clear and Venalink. Figure 5 provides the specification for each of these. The dossette box, although frequently recommended by pharmacies, does not conform to recommendations for MOD characteristics as it is not a sealed unit; thus, medicines dispensed into the MOD are not protected from water vapour and atmospheric gases. 72 This means that medicines dispensed in the MOD are at increased risk of degradation. The Nomad Clear and Venalink are both sealed containers and thus afford greater protection to the medication. Furthermore, it is recommended that MODs should have tamper-evident seals, which dosette boxes do not have.

FIGURE 5.

Medication organisation devices.

Nomad Clear is a lidded, clear rigid plastic tray of pre-formed compartments. It is cold sealed using a flat plastic adhesive film through which pills are accessed. The film is labelled with days of the week and times of the day and is perforated at the perimeter of each compartment to facilitate ease of access. Nomad Clear XL™ (Surgichem Ltd, Cheshire, UK) is identical to Nomad Clear but, having larger compartments, is better suited to those prescribed more medicines. The Venalin is a cold-sealed device which closely represents blister packaging. It comprises a non-rigid plastic tray into which pills are dispensed. This non-rigid tray is then inserted into a cardboard wallet labelled with days of the week and times of the day and sealed using an opaque adhesive cover.

Primary outcome

The primary outcome measure, as stipulated by the funding agreement, was EAM. EAM overcomes the problem of self-presentation bias which is associated with self-reported adherence. EAM may, however, be susceptible to reactivity bias as the device used for EAM acts as a constant reminder of trial participation and thus may alter usual patient medication-taking behaviour, resulting in artificially inflated adherence. 73,74

Broad-ranging web-based searches were carried out to identify candidate objective adherence monitoring technology and manufacturers were approached to determine whether or not they could supply suitable technology for these purposes.

These searches identified that technological advances mean that it is now possible to objectively measure adherence. This type of monitoring is less susceptible to the Hawthorne effect by virtue of being less intrusive and less conspicuous to the patient than direct adherence measures or DUCs. Electronic medication monitoring systems were initially developed as bottle-based systems containing a switch and data logger in the cap. Such electronic medication monitoring systems have been widely used in clinical trials to assess medication adherence. 11,24–26 The data logger records the date and time of each bottle opening event. However, usual dispensing is now generally in manufacturer issued packaging which is in blister pack form. Trials have, therefore, generally approached this issue by decanting medication from usual packaging into the monitoring bottles. This has the limitation of not assessing adherence in a naturalistic setting.

Similar technological principles have been applied to blister packs; however, there is a further challenge to then translate this to electronic monitoring of manufacturer supplied blister packaging. There is no standardisation to the size and shape of solid oral dosage forms; thus, there is a huge array of different-sized and -shaped packaging.

The main requirements of an electronic monitoring system are that it is discrete, accurate and robust, and consistent for both MODs and usual-care blister packaging; in addition, the comparator must look and feel like usual care and the intervention should provide a good approximation of the intervention as it would in a real setting.

A number of potential candidate technologies were identified for EAM of usual packaging and three organisations were approached for information or to supply devices for the project [Med-ic MMD Solution™, Qolpac b.v. NL and Future Technology (UK)].

Qolpac b.v. (NL) provides electronics-based e-health solutions. Its Objective Therapy Compliance Monitoring (OtCM™, Qolpac, NL, Amsterdam, the Netherlands) system is a powered, single-circuit and resistor-based system printed using silver or copper ink onto a clear plastic sticky-backed film which records each time a blister seal is broken, assuming that this equates to ingestion. Data are downloaded to computer software which records the blister position and time it was broken to within 10 seconds of the actual time. Figure 6 shows a representation of the Qolpac b.v. OtCM system.

FIGURE 6.

Representation of an EAM system for blister packs.

Future Technology (UK) Ltd (Banbury, Oxfordshire, UK) has recently developed SMARTpack™, which is similar to the Qolpac product in design and data produced but uses mobile phone technology to transfer data from a logging device to a remote server.

GP Solutions (UK) Ltd supplied the E-wallet™ (Manchester, UK): a system that enables medication-taking events from blister packs and MODs to be recorded in the same manner as the previous two systems. Further details of the systems are presented in Tables 8 and 9 and in Appendix 2.

All systems require precision-designed detection circuits which fit discretely to the surface of the blister pack. Each blister pack requires a slightly different design to ensure accurate recording of pill-taking events, that is if the circuitry does not cross the pill position then the removal of a pill will not be recorded. For a study of one medication, such systems are relatively cost-effective; however, in the case of studies involving multiple different medicines, each with individually shaped packaging, the cost of developing a detection circuit bespoke for each different type of medication would be prohibitive.

A 2-month pilot study (n = 52) of this technology to assess feasibility and acceptability reported promising results. Adherence data were obtained from 94.3% of participants, and 67.4% of participants reported that they would consider using the electronic monitoring system for a long-term study. 27 The original protocol for the pilot RCT set out to include adults aged ≥ 75 years if they were prescribed three or more SODF medications which were expected to be continued for 12 months, were registered with one of six participating medical practices and an associated pharmacy and were capable of providing informed consent. However, owing to technical difficulties with the supply of a suitable EAM system, the eligibility criteria were revised. The EAM systems were not universal and because of cost restrictions could be developed for only a limited range of pack sizes. Therefore, eligibility criteria were relaxed to enable adults aged ≥ 75 years who were taking three or more SODFs to be recruited. At least two of the participants’ medications needed to be from a defined list of 11 medications sourced from one supplier and identified by study GPs as those most commonly taken by patients in the over-75-years age group to enable electronic monitoring of adherence to at least two medicines.

Although all components of MODs and usual-care EAM systems are readily available, no prototype units from the preferred supplier were available for testing and no other EAM technology could be provided for this project within the necessary timescale. In the absence of an electronic monitoring system, DUCs are the most appropriate approach to quantitative adherence measurement.

The DUC is based on the assumption that, if the medication is not in the container, the patient has taken it. This is problematic when attempting to identify intentional non-adherence because patients may deliberately remove and discard medication in order to disguise non-adherence. In essence, however, the electronic monitoring systems are open to the same criticism. Retrospective DUCs are, however, less likely to cause reactivity bias, and previous research has demonstrated that conducting DUCs on the older patient population is feasible and acceptable to patients. 15

Accurate DUCs require accurate knowledge of the quantities of medication to which the patient has access. Hoarding of medication is a regular occurrence; thus, accurate DUCs require that patient access to medicine stores be prohibited. However, the removal of what is, essentially, the property of the patient to a place of safety while also guaranteeing, the integrity of the medications during storage and then their safe and ethical return is problematic.

While pharmacies are secure environments with appropriate medication storage facilities and thus the most intuitive solution to storing patient hoarded medication during a trial, liaison with the pharmacies participating in this feasibility study identified that it would not be an acceptable solution. Limited space was a common reason cited plus ethical considerations; it was considered undesirable to remove a patient’s property and then to be unable to return it because it was out of date or because its provenance could not be guaranteed.

These issues may be circumvented by allowing the patient to retain their non-trial medicines within their home but packaging them into a sealed container so that the contents cannot be accessed without destroying the container. The additional merit of this process is that the patient is able to access their medication in case of an emergency.

Secondary outcomes

Introduction

Trial design must be feasible and acceptable from the perspective not only of researchers but also of patients and health-care professionals. Focus groups were carried out to identify design elements that would or would not be acceptable and, when necessary, to refine the experimental design to meet both researcher and stakeholder requirements.

Pre-specified objectives for the patient and carer focus groups were to:

• describe the practical difficulties experienced by patients and carers in adhering to complex medication regimes

• explore the appropriateness and acceptability of MODs

• consider the benefits and disadvantages of MODs including any potential adverse outcomes

• establish the feasibility and appropriateness of trial participant requirements including recruitment documentation, participant information sheets, survey tools and proposed adherence measures.

Pre-specified objectives for the health-care practitioner focus groups were to model and gain consensus expert opinion on:

• the appropriateness and feasibility of the research design

• participant inclusion and exclusion criteria

• recommended MODs and criteria for selection

• any clinically important difference in patient adherence.

Methods

The feasibility study of a RCT was funded by the NIHR HTA programme and started in August 2011 (ref. HTA 09/34/03). The trial was approved by the Cambridge East Research Ethics Committee (ref. 11/EE/0141) and Norfolk Primary Care Trust (ref. 55252). The trial is registered with UK Clinical Research Network (ref. UKCRN 11256). The protocol describing pre-RCT focus group study design is appended (see Appendix 3).

Six medical practices and the geographically close pharmacies in Norwich, UK, expressed an interest in study involvement. Potential participants were identified by medical practice staff via a computerised search and the resulting list was manually screened by either the prescribing team or GP. Three practices recruited patients receiving, or with a history of having received, a MOD, one practice recruited patients not currently receiving, and with no history of having received, a MOD and two practices recruited carers of patients in receipt of a MOD.

Focus group participant identification and recruitment

Patients and informal carers were purposively sampled to represent a wide variety of medical, social and demographic characteristics.

Inclusion criteria for patients were:

• age ≥ 75 years

• exhibiting any medication-taking behaviour (thought to be fully adherent or intentionally non-adherent or unintentionally non-adherent)

• representing a range of regimen complexity (regularly prescribed three or more medicines and prescribed multiple formulations, e.g inhalers, eye drops and cream or ointments)

• currently using MODs or who have previously declined the use of a MOD

• with mild cognitive impairment (sufficient in the clinician’s opinion to allow provision of informed consent and engagement with a focus group)

• with manual dexterity problems.

Exclusion criteria for patients were:

• on the mental health register with, for example, a diagnosis of schizophrenia or bipolar disorder, but not depression

• on the dementia register

• unable to read or speak English and/or to provide informed consent

• deemed by the health-care team to be inappropriate for study inclusion.

Informal carers were defined as friends and relatives aged over 18 years who support patients in their medication organisation and/or taking but receive no remuneration, were required to be registered with participating medical practices and were known by practice staff to support a person aged over 75 years in managing medication.

Patients and carers were invited to take part via a recruitment pack posted from their medical practice which comprised a covering letter from the medical practice, participant information leaflet and consent form. Consenting participants returned consent forms with contact details directly to the researchers. After 2 weeks a follow-up letter was sent from practices to non-responders.

Written informed consent was sought from patients and carers to allow contact from researchers with a view to arranging focus groups. Patients and their carers were not invited to participate together in the same focus group, as this may have prevented participants from speaking freely.

Focus group conduct and data collection

Topic guides were developed from pre-specified study objectives. Meetings were scheduled to last for 90 minutes and were designed to encourage free discussion and to generate a wide range of ideas and opinions. Audio-recordings were made with the consent of participants. A broad introductory session on general areas of organising, giving and taking regular medicines was followed by in-depth discussion surrounding MODs and study procedures. At the relevant points in the discussion, samples of proposed study documentation and a selection of sample MODs were introduced to serve as trigger material to promote discussion of appropriate documentation and MOD types for the proposed study.

Results

Fifteen patients and carers, comprising broadly equal numbers of males and females, participated in two focus groups with 66% having some experience of either self-filled (46%) or pharmacy-filled MODs (20%). Five health-care professionals took part in the primary care focus group and six attended the secondary care focus group. All focus groups lasted between 85 and 90 minutes. Table 10 provides details of participants attending focus groups.

Domains discussed by all groups were medication management strategies and the difficulties associated with taking and giving regular medicines; the use and effects of MODs; and study design, procedures and outcome measures.

Following analysis of transcripts the findings fell broadly into four themes: barriers to adherence; patient autonomy; using MODs; and the acceptability of the proposed pilot study.

Trial design, funding and approval

The RCT was a single-centre (multisite) individually randomised 2 × 2 factorial design comparing the effect of MODs with medication dispensed in usual packaging and of weekly compared with a monthly medication supply. The trial was conducted across six NHS Norfolk GP practices and 14 pharmacies which served the patients registered at these GP practices. Five of the GP practices and 13 of the pharmacies were situated in the Norwich area, including its suburbs, and one GP practice and pharmacy was based in a small, nearby market town.

The trial was funded by the NIHR HTA programme and started in August 2011 (ref. HTA 09/34/03). The trial was approved by the Cambridge South Research Ethics Committee (ref. 12/EE/0251), Norfolk Primary Care Trust (ref. 99999) and additionally by Norfolk County Council (in respect of social services usage data only). The trial is registered with the International Standard Randomised Controlled Trial Register (ref. ISRCTN 30626972) and the UK Clinical Research Network (ref. UKCRN 12739).

A number of changes to procedure were necessary as a result of the failure to obtain a suitable EAM system. Table 15 provides a summary of deviations from the protocol with reasons for the deviation. All deviations were appropriately notified as amendments to the protocol.

Participant eligibility

The pilot had two phases. Phase 1 established patients’ adherence; only patients identified as unintentionally non-adherent progressed to phase 2, which was randomisation.

As described in Chapter 3, refinements to the eligibility criteria were necessary because of constraints of the proposed EAM system.

Eligibility criteria

The process for ensuring that all randomised participants fulfilled the inclusion and exclusion criteria was multistage and is summarised in Figure 7. Some eligibility criteria are repeated at the different stages where appropriate multiple information sources were used to confirm eligibility.

FIGURE 7.

Process for determining participant eligibility.

Inclusion

Patients were included if they were:

• aged ≥ 75 years

• registered with one of six participating medical practices

• prescribed three or more SODF medications expected to be continued for 12 months (of which at least two were from a defined list in Table 16)

• capable of providing informed consent.

Exclusion

Patients were excluded if they:

• were regularly prescribed medication not currently dispensed by a pharmacy recruited to the study

• had a life expectancy of less than 12 months

• were currently involved in a clinical trial

• were not self-administering their medication

• were currently receiving or had previously received medication in a MOD

• were diagnosed with Parkinson’s disease or a severe mental health disorder such as schizophrenia

• were deemed by the health-care team to be unsuitable for study inclusion for other reasons

• were using a medication organisation strategy incompatible with trial participation

• were intentionally non-adherent.

Identification

One member of each of the six recruited medical practices undertook a database search to identify potentially eligible patients. A GP subsequently screened the list to exclude any patients deemed unsuitable. The reasons for exclusion were recorded in order to further refine the exclusion criteria of a definitive study.

Recruitment

Two patient recruitment methods were tested, passive and active, to determine their acceptability and effectiveness. For consenting patients, responses to questionnaire 1 (see Appendix 5) were used to identify whether or not the patient should be excluded on the following criteria:

• were using a medication organisation strategy incompatible with trial participation

• were intentionally non-adherent.

Passive recruitment

Patients identified by GP database search, and subsequently GP screened for suitability, were sent recruitment packs. Recruitment packs comprised a letter of invitation on GP practice-headed paper (see Appendix 6), patient information leaflet (PIL; see Appendix 7), consent form (see Appendix 8), questionnaire 1 (see Appendix 5) and a pre-paid envelope. Participants were asked to return completed documents directly to researchers. Questionnaire 1 was not attached to the consent form and so patients were able to choose to return both, either or neither. Implicit consent was assumed for any questionnaires returned. One reminder was sent to each non-responding patient 3 weeks after initial posting.

A standard operating procedure (SOP) for passive recruitment was prepared (see Appendix 9). Three practices used only passive recruitment which commenced during August 2012 and was completed in February 2013.

Active recruitment

This process involved one of six researchers recruiting patients when they presented for routine appointments at the participating GP practices. Patients identified by GP database search, and subsequently GP screened for suitability, were highlighted on the GP system, which enabled practice reception staff to identify the potentially eligible patients who would be presenting at the medical practice on a recruitment day. Such patients were provided with a PIL and an explanation that there was a researcher present who would like to talk to them about the study if they were interested. Patients who chose to approach the researcher were informed about the study and given a recruitment pack which they could complete at the medical practice or take away. If patients volunteered information indicating that they were ineligible (e.g. they were already using a MOD), they were informed of their ineligibility and thanked for their time. The SOP for active recruitment is provided in Appendix 10.

Three practices used active recruitment for a 3-week period followed by passive recruitment, as previously described. Active recruitment was carried out in 3-week tranches during the period September to November 2012.

Intervention and comparator

Participants were randomised to one of the following arms for 8 weeks:

1. four MODs dispensed monthly either collected by the patient or delivered to the patient’s home, according to usual patient routine

2. one MOD delivered to the patient’s home weekly

3. usual medication packaging dispensed monthly and either collected by patient or delivered to the patient’s home, according to usual patient routine

4. usual-care medication packaging delivered to the patient’s home weekly.

Other prescribed medication that could not be dispensed in a MOD, for example inhalers, creams, liquids and when-required medication, was supplied in usual packaging with the MOD.

Medication organisation devices

Nomad Clear, Nomad Clear-XL and a Venalink device were used. MODs contained all SODFs prescribed to the patient unless they were unsuited to being dispensed in a MOD [e.g. because they are to be taken ‘when required’, their administration requires special processes, such as dissolving or chewing, or their dose is likely to change frequently or at short notice (e.g. warfarin)]. Monthly MODs were either collected from the pharmacy or delivered from the pharmacy to the patient according to the usual routine of the patient. Weekly MODs were delivered to the patient.

Usual care

Medication supplied in the manufacturer’s usual packaging or in accordance with usual dispensing procedures when this is inappropriate, for example prescription requests for a quantity other than the manufacturer’s pack size. Monthly packaging was either delivered to the participant or collected from the pharmacy according to the participant’s usual routine. Weekly packaging was delivered to the participant’s home.

Outcome measures and outcomes

Data collection

Questionnaire data were collected from patients by postal return or in their own homes if participants requested help to complete forms. Data on pharmacy activities and the empty packaging for pill counts were obtained from each participating pharmacy. Patient data on comorbidities and use of health-care services for the period 2 months prior to and 2 months after the start of the trial were obtained from participating GP practices. Data on use of social care services were obtained from Norfolk County Council Social Services Department.

Prevalence of intentional non-adherence

All consenting participants were screened for factors including intentional non-adherence, non-self-medication, current or previous use of a MOD and complex medication management strategies using questionnaire 1. Intentional non-adherence was investigated using eight statements in four pairs across three sections of the questionnaire and an arbitrary cut-off of extreme agreement (strongly agree or ‘yes’ response) with three or more of the eight statements on intentional non-adherence was used to indicate likely intentional non-adherence. Statements included widely accepted reasons for intentional non-adherence in a UK setting and addressed the need for treatment breaks, addiction concerns, cost–benefit, perceived toxicity, medication necessity and side effects.

Questionnaire 1 was also used to identify participants already using MODs or using methods of medication organisation which would either be incompatible with EAM or unethical to remove from a participant.

Participants not meeting the inclusion criteria because of intentional non-adherence, use of MOD or other trial-incompatible medication organisation strategy were thanked for their interest by letter and informed we would be making no further contact with them (see Appendix 11).

Participants identified as potentially eligible after completing questionnaire 1 were contacted by telephone to arrange a preliminary baseline visit in their own home, and their usual pharmacy was identified.

Prevalence and magnitude of unintentional non-adherence

The purpose of the baseline visit was multifold: to meet with patients (often for the first time); to confirm that all non-medical inclusion criteria had been identified (e.g. that there were no obvious unreported medication-taking strategies in place); to remove existing prescribed medication so as to ensure that the patient did not accidentally take medicines from a non-study pack; and to deliver a new 28-day medication supply to be taken over the following 4-week period, which would allow the study team to ascertain whether or not patients may be unintentionally non-adherent.

Prior to the baseline visit, participants were asked to collect all medicines together. At the baseline visit, researchers stressed that they would not reveal information to their GPs about how well participants took their medicine, as researchers were interested only in measuring the participants’ usual medication-taking habits. Researchers recorded name, form and quantity of all existing medicines and then removed them from use by placing them in one or more large clear bags secured with a cable tie. Details were recorded on administration form 1 (see Appendix 12) and participants were asked to endorse the record. Medications no longer required or expired were returned to the pharmacy for destruction with participant consent.

Each secure bag was labelled clearly with the participant’s name and instructions not to use the contents unless absolutely necessary. The sealed medicines were returned to the participant for safe keeping and participants were asked to inform researchers if they needed to open the bag during the study. A follow-up appointment was arranged for 3 weeks (± 2 days) after the initial visit.

The assessment of unintentional non-adherence was made by DUC. If a participant had taken fewer than expected SODFs, they were identified as unintentionally non-adherent. Researchers also confirmed the participant’s willingness to continue in the study and carried out randomisation. Participants were informed of their randomisation status. If participants were randomised to receive a MOD, they were shown the three study MODs (Nomad, Nomad XL and Venalink) and asked to select their preferred MOD. Their choice and confirmation that they were able to use it were recorded using administration form 2 (see Appendix 13).

Irrespective of adherence status or willingness to continue in the trial, participants were asked to continue to take the 1-week (approximately) medication supply remaining in the packs delivered at the baseline visit and then to continue to obtain their prescribed medications according to their usual supply routine. Pharmacists were informed of randomisation status for all participants.

Adherence monitoring

In the absence of a suitable EAM system, DUC was used to determine adherence to prescribed SODFs. Patients were provided with a number of large, clearly labelled, sealable plastic wallets in which to store used medication packaging. Wallets were returned to pharmacies by patients or delivery drivers coinciding with usual collection from the pharmacy. Researchers contacted patients for whom medicines had not been returned to the pharmacy. The number of doses returned for each medicine was counted and recorded.

Characterisation of the participant population

Cognitive function was assessed using the 11-item MMSE. It is scored out of 30, with scores of 23 and above indicative of cognitive function within the normal range. 65

Manual dexterity was assessed using the 9-HPT, which was scored by measuring the period of time, in seconds, taken to move pegs, similar in dimension to medication capsules, from a tray into a 3 × 3 array of holes which accommodate the pegs in a vertical orientation, and then to remove them and place them back into the tray. 76

Visual acuity was assessed using the smallest font size that could be read on the Bailey–Lovie Near Chart. 71 Participants used their usual visual aids and held the chart at their normal reading distance (approximately 40 cm). The test took place under the lighting conditions that patients would usually be exposed to when taking their day-time medicines.

General practitioner records were accessed by a member of the research team in order to obtain participant comorbidities.

Participants also completed baseline health and quality-of-life measures as described in Chapter 7.

Randomisation

Randomisation was carried out at the participant’s home by researchers. Participants identified as unintentionally non-adherent were randomised to phase 2. Researchers used mobile internet to connect to the secure study database. The randomisation sequences were generated automatically using an online system developed and managed by Norwich Clinical Trials Unit. Allocation was automatically recorded in the study database. Randomisation was done on a permuted block basis. There were six strata, one per GP medical practice. There were four randomisation arms (usual supply weekly, usual supply monthly, MOD weekly, MOD weekly). The codes for each stratum were blocked randomly into groups of four or eight to try and even out the distribution across randomisation arms.

Blinding

Given the nature of the interventions under study (MOD and frequency of dispensing), blinding was not possible for patients, researchers or pharmacists involved in the study.

Administration and timings

At commencement of the intervention, researchers visited the participant and explained procedural details emphasising how and when medicines would be supplied and asking that all medication packaging was retained for researchers. It was again stressed that researchers would not feed information back to GPs about how well participants were taking their medicine and that they were only interested in measuring the participants’ usual medication-taking habits. At this visit, researchers delivered the first medicines of the intervention and follow-up questionnaires: EQ-5D-3L combined with the Investigating Choice Experiment CAPability measure for Older people (ICECAP-O, questionnaire 2), a patient satisfaction survey (questionnaire 3, see Appendix 14). A further questionnaire and letter of invitation for any person living with or caring for the participant (questionnaire 4, see Appendix 15) was also left with the participant. Researchers agreed to telephone participants 1 week prior to completion of the intervention to explain the procedure for returning to usual care.

General practice and pharmacy participation

General practice participation at this stage of the trial was limited to routine prescription preparation and required only minimal interaction between researchers and prescription managers.

Pharmacists were informed directly after randomisation of the randomisation status of their participant. They were asked to follow the instructions provided in the SOP for pharmacies (see Appendix 16). Briefly, pharmacists prepared medicines according to randomisation. In the case of medicines that were to be dispensed in MODs, after the first delivery (carried out by researcher), these were delivered to the participant by the pharmacist via their usual delivery service for weekly supply or were collected by participants. Medicines that were dispensed in usual packaging were delivered weekly to patients randomised to usual care and delivered or collected in accordance with usual practice for patients randomised to monthly supply.

Pharmacy data collection

Pharmacists were asked to record participant details, medicines supplied, time taken to dispense medicines and any near-miss dispensing or prescribing errors identified on administration form 3 (see Appendix 17), with further details, if necessary, on pro-forma follow-on sheets. They were also asked to record any other comments about the trial.

Adverse events

This pilot trial was not a clinical trial of an investigational medicinal product, and thus no formal process for recording adverse events (AEs) was implemented. Any potential AEs identified were defined in accordance with European Clinical Trial Directive 2001/20/EC as new or the worsening of pre-existing symptoms.

Serious adverse events

Serious adverse events (SAEs) are defined by European legislation as the development of an undesirable medical condition or the deterioration of an existing medical condition following or during exposure to an investigational medicinal product, whether or not it is considered causally related to that product, which results in hospitalisation or prolongation of hospitalisation; immediately life-threatening illness; persistent or significant disability; and incapacity or death.

Descriptions of all AEs were recorded using UEA SOP 206 with associated SAE form (see Appendices 19 and 20). Reports were made to the UEA research sponsor and local research and governance representative within stipulated time limits. Governance committees assessed the reports. All AEs identified were also reported to management and steering committees. The participant’s usual GP was contacted for an opinion on probable causality, that is whether the GP considered it unlikely, possible or likely that events occurred as a result of trial participation.

Sample size

The sample size was determined as follows. A total of 120 participants were required for the RCT. With four groups of 30 participants, the coefficient of variation of the SE estimates for any continuous outcome measure for each group was expected to be about 13% using a linear approximation and assumption of normality. These SE estimates, would have been used to inform the power calculation of a definitive trial and would have been within about 25% of their true values based on the expected half-width of a 95% CI. It was estimated that, in order to obtain 120 participants who are primarily unintentionally non-adherent, 720 potential participants needed to be screened using a self-report questionnaire. This estimate was based on the assumption that approximately 20% of participants would be intentionally non-adherent17,18,77,78 and thus excluded. The remaining 576 participants would be monitored for non-adherence using DUC. This stage was expected to result in 30% attrition, and thus 403 participants would remain, of whom 160 (40%) were expected to be non-adherent. These participants would then be randomised and monitored for 3 months, during which process a further 25% attrition was expected, leaving 120 participants. (See Figure 8 for the flow diagram summarising participant pathway through trial.) Participants suspected of intentional non-adherence were advised to consult their pharmacist for a medicines use review or their GP.

The primary outcome measure was the mean difference in per cent adherence (measured by pill count) between participants in receipt of a MOD and participants who received their medication in usual-care packaging. This should provide an estimate of the effect of the factor packaging. As there are no reported trials with a similar population, that is pre-screened to remove participants likely to be adherent or demonstrate intentional non-adherence, data were not available to estimate the variance of adherence measured in this population. However, an upper estimate of the precision was determined by considering the dichotomisation of this outcome to the binary measure adherent versus non-adherent. Making the conservative assumption that 50% of participants would be adherent and with 60 participants receiving a MOD and 60 receiving usual-care packaging, the half-width of a 95% CI around the difference between these groups should be less than 18%.

No interim analyses or stopping guidelines were included in the study design.

Statistical methods

A dedicated study database was designed, built and maintained by Norwich Clinical Trials Unit, data were analysed using SAS version 9.2 (SAS Institute Inc., Cary, NC, USA) and PASW statistics version 18 (SPSS Inc., Chicago, IL, USA). Descriptive analyses were used to report recruitment and consent rates, social and health (comorbidities) status, participant functional abilities in addition to prevalence of intentional and unintentional non-adherence. Unintentional non-adherence during a 3-week screening period was calculated as follows:

The magnitude of unintentional non-adherence was determined from median and interquartile divergence for all unintentionally non-adherent participants.

Unintentional non-adherence during the 8-week trial period was identified and magnitude calculated as described above for the 3-week screening. Percentage medication adherence was reported for participants randomised to MODs or usual care and weekly or monthly delivery for each medication. This approach took into account that there were varied individual daily dose regimes and that not all participants returned all packs. Analyses were undertaken for any medicines returned by participants for whom at least 1 week’s medication packaging was returned. Previous research indicated that the data may be negatively skewed with minimal deviation from 100% adherence. 15 Secondary analysis was, therefore, undertaken using a dichotomisation of 100% versus less than 100% adherence.

Analyses were carried out with respect to incidence of reported AEs. Fisher’s exact (two-tailed) test was used to determine the statistical significance of the events with respect to type of packaging and frequency of supply.

Although the study was not powered to detect differences of a particular size, and accepting the limitations of a feasibility study, the observed differences between the study groups were presented. Exploratory analyses were also undertaken to identify any indications that variables such as cognitive function, number of prescribed medicines and manual dexterity affect the primary outcome (adherence). This was to identify whether or not any subgroups of participants might benefit from MODs.

Results

The inability to obtain a suitable objective EAM solution also delayed a number of trial timelines. The initial intent was to carry out the 3-week assessment of unintentional non-adherence with immediate randomisation of eligible participants. As a result of difficulties identifying a suitable adherence monitoring device, there was a delay of 2–3 months between randomisation and the start of the RCT.

Recruitment

Figure 8 [Consolidated Standards of Reporting Trials (CONSORT) diagram] illustrates the flow of patients through the study. The most significant attrition during recruitment was at the stages of determining eligibility for EAM, followed by ineligibility as a result of medication-taking strategies that were inconsistent with MODs or electronic monitoring. A completed questionnaire 1 was received from 288 participants, of whom 11 (3.8%) were excluded because of current participation in a medication trial.

FIGURE 8.

Flow of participants.

Participants using a medication organisation device or medication organisation strategy incompatible with trial participation

One hundred and two participants (35.4%) reported in questionnaire 1 using a MOD and were thus excluded. A further 27 (9.4%) participants reported use of strategies to aid correct medication taking which were incompatible with the study design. Six reported that they were not self-medicating, one reported using an alarm, three reported popping out their pills in advance into other containers, for example bottles or cups, which would have been incompatible with EAM. The remaining 17 participants reported using other equipment to organise their medicines. For example, some used a series of shot glasses and others ice cube trays.

Intentional non-adherence

Screening of questionnaire 1 responses identified 71 (24.7%) participants indicating extreme agreement with one of eight statements exploring intentional non-adherence. Eighteen (6.3%) indicated extreme agreement with two of eight statements and two (0.7%) participants indicated extreme agreement with three statements and were thus excluded from the study.

Unintentional non-adherence

All 80 eligible participants presented their medicines to the researchers during visit 1. Some medications were neatly presented with the correct blisters in their original packaging in an organised fashion and others appeared to have been randomly tossed into buckets, bags and boxes. One participant, who was subsequently withdrawn from the study, had very poor eyesight and used the colours and sizes of his medicine boxes to distinguish one from another. This participant had incorrect blisters inside boxes.

Many patients had medicines surplus to reasonable requirement in their possession, as summarised in Table 17. As an extreme example, one participant presented 17 unused tubes of Ovestin cream and seven unopened boxes of codeine phosphate. Many participants had excessive numbers of surplus prescribed painkillers. No patients reported any discomfort with or declined the removal from use of their old medicine stocks.

Dosage unit count data were available for 76 subjects, of whom 41 (53.9%) were ‘fully adherent’ and thus excluded from further study participation. The remaining 35 participants (46.1%, 95% CI 34.8% to 57.3%) were unintentionally non-adherent. Five of these participants withdrew prior to randomisation and one was excluded after visit 2 because of concerns about his mental health on discussion with his health-care team.

Of the 29 randomised participants, three subsequently withdrew prior to receiving the intervention: one participant citing problems with delivery (n = 1 MOD weekly) and two participants stating that they preferred not to take part (n = 1 MOD weekly; n = 1 usual supply monthly). A further patient was withdrawn after randomisation to the usual supply monthly group because his eyesight was extremely poor and there were concerns that he was unable to manage his medicines safely; his health-care team were informed.

Twenty-five participants commenced the intervention, but primary outcome data were not received from three of these participants and thus they were excluded from analysis: one participant was admitted to hospital during the trial (n = 1 MOD monthly), while the other two participants (n = 1, usual care weekly; n = 1, usual care monthly) did not return any medication blister strips to enable the primary outcome measure to be calculated.

Table 18 summarises the baseline characteristics of participants randomised to each of the four groups. It can be seen that the participants were relatively evenly matched across the four groups.

Table 19 illustrates the cognitive function, manual dexterity and visual acuity of participants. No randomised participant had significant cognitive impairment. There was substantial variation in manual dexterity and 12 (41.4%) participants had manual dexterity that was poorer than the normative value for the population aged over 71 years. 79

Relationship of adherence to cognitive and physical function

The relationships between adherence and MMSE scores, dexterity scores and visual acuity were explored for possible factors predicting adherence among the 22 subjects in whom adherence was measured in the trial. Spearman’s rank correlation (r_s) was used to quantify the relationship between adherence and these variables, as shown in Table 21.

One correlation coefficient was statistically significantly different from zero. This was for the relationship between adherence and dexterity of the left hand (r_s = –0.516; p = 0.014). This value was negative, indicating that the longer the time to complete the dexterity test with the left hand, the lower the adherence score was likely to be. No other coefficient was statistically significant.

Adverse events

A total of five AEs (n = 3) or SAEs (n = 2) were identified in the 25 participants receiving the intervention. Events comprised a hypoglycaemic episode, three falls leading to hip fracture, with death in one patient, and one temporary physical incapacitation. All five events were identified in groups randomised to receive MODs. Using Fisher’s exact test (two-tailed), the association between groups (MODs/usual care) and outcomes (harm/no harm) was statistically significant (p = 0.0391). Two events occurred in participants receiving care on a monthly basis and three in those receiving on a weekly basis. Using Fisher’s exact test (two-tailed), the association between groups (weekly care/monthly care) and outcomes (harm/no harm) was not statistically significant (p = 1.000). The SAEs and AEs are detailed in Tables 22–26. No events were identified in participants randomised to usual care.

Pharmacists were asked to record any near misses and the type of near miss for the duration of the RCT. Pharmacy data collection forms were returned for all participants; the two near misses that were reported are presented in Table 27.

Questionnaire data collection

Participants were provided with two questionnaires during the visit at which they were randomised: a participant experience questionnaire and a carer experience questionnaire. Participants were asked to complete and return the participant experience questionnaire immediately after completing the RCT. Participants who did not return the questionnaire within 2 weeks of completion and who were thought to have completed the trial were reminded once by telephone.

The carer experience questionnaire left with participants was accompanied by a letter of invitation and participants were encouraged to give the questionnaire and letter to the most appropriate carer (formal carer, spouse or other relative). The letter invited completion and return of the questionnaire when the person they cared for had completed the RCT. Carers were unknown to researchers and, therefore, were not reminded to complete the questionnaire.

Participant questionnaire content

An 11-item questionnaire was developed incorporating validated and non-validated items. Non-validated items were developed from the phase 1 focus group findings; the trial management committee was used to assess content validity and the independent steering committee comprising lay members was used to assess face validity. Three non-validated items related to satisfaction with the medication packaging (MOD or manufacturer’s packaging) and the extent to which it met the participant’s needs. Responses were sought on a 4-point Likert scale, with the most positive responses having the highest score.

Participants’ autonomy, confidence and anxiety associated with medicine taking were assessed using six items developed through appropriate adaptation of the patient enablement instrument and previous work undertaken by the research team. 80 Responses were sought on a 5-point Likert scale, with the most positive responses having the highest score. The questionnaire is appended (see Appendix 14).

Carer questionnaire content

The carer questionnaire had six items related to autonomy, confidence and anxiety, as previously described for the participant questionnaire. These questions were adapted to be relevant for a carer or relative to complete, for example:

My confidence in their ability to take their medicines correctly is . . .

My level of anxiety about them taking their medicines wrongly is . . .

It also included a health economics section to capture the amount and type of care provided to the person involved in the study. A copy of this questionnaire is provided in Appendix 15.

Post-trial discussions

Two group discussions were convened in order to capture the experiences of patients and health-care professionals involved in the trial. The discussions were structured to gain opinions on each stage of the trial process to identify what worked well and less well with a view to optimising definitive study design. Focus group 5 (FG5) comprised RCT participants and focus group 6 (FG6) comprised a range of health-care professionals involved in the delivery of the RCT.

Participant identification and recruitment

The feasibility study information leaflet and consent form invited participants to express an interest in attending a group discussion convened once the trial was complete. These participants were purposively sampled to provide even representation of all four arms of the trial (MOD weekly, MOD monthly, usual care weekly and usual care monthly) and invited by telephone to attend the group discussion. All health-care professionals involved in the study (pharmacists, practice managers and GPs) were invited to take part via telephone and those confirming interest were mailed recruitment packs (invitation letter, PIL and consent form). Consenting health-care professionals were then purposively sampled to ensure even representation of all groups.

Data collection

Discussion guides were based on activities for each stage of the RCT element of this feasibility study, designed to capture both positive and negative experiences and tailored to the perspectives of each group (Appendices 21 and 22). A copy of study information sheets, consent forms and questionnaires was provided to each participant and example MODs were provided for each group. The group discussions took place at the UEA and were designed to last no longer than 90 minutes.

The discussions were moderated by the principal investigator and a researcher independent of the study. Proceedings were audio-recorded and transcribed verbatim. NVivo version 9 software was used to organise the themes of the discussions. Two researchers (SB and CA) read and coded the transcripts independently. An expert in qualitative research (CS) reviewed the final framework comprising themes and subthemes. A selection of anonymised quotes from the transcripts was extracted to illustrate identified themes.

Participant satisfaction with medication packaging, trial management and participation

Twenty-three participants completed the study, of whom 21 (91%) returned patient satisfaction questionnaires. Table 28 provides a comparison of the patient ratings of the different medication packaging and supply formats trialled. It can be seen that patient rating of their medication packaging was generally high, with MOD weekly achieving a median score of 3 on a 4-point scale, where 4 indicates maximal satisfaction. Although MOD weekly was consistently scored highest or equal highest for all criteria, no significant differences were observed in this small sample.

Table 29 illustrates the distribution in participant responses regarding perceived change in confidence to self-manage their medication and health plus the extent to which the group to which they were allocated affected any stress associated with medication taking. The median score of 2 on a 5-point scale indicates a response of ‘the same’ for the majority of criteria. This in turn suggests that patient confidence and stress remained largely unchanged before and after randomisation.

Table 30 summarises the carers’ responses to the post-study questionnaire assessing their perception of any changes in the participant’s ability to manage their medication or autonomy. From the 23 participants that completed the study, 13 (57%) carer questionnaires were returned. Results indicate that type of device had no significant effect on any of the autonomy items, as rated by the carers, although there was an observed trend to lower scores in the monthly usual packaging group.

Of the 29 participants randomised, 17 expressed an interest in participating in a group discussion; purposive sampling led to eight attending the group discussion.

Participant characteristics

Eight participants of the pilot RCT participated in the first focus group and seven health-care professionals in the second. Table 31 provides a summary of participant characteristics.

Discussion across both focus groups fell into four broad areas: the general experience of taking part; medication-related issues; comments about specific tools used within the study; and comments regarding a future trial. These are summarised with illustrative quotes, with identifiers (P for patient and HCP for health-care professional), reported in boxes at the end of each section.

General experience of taking part

There were five subthemes under the overarching theme of general experiences: reasons for taking part, communication about the study, patient behavioural changes, impact of study on medical practices and pharmacies, and feedback to pharmacists.

Varied reasons were provided for taking part by patients and health-care professionals. Patients wanted to take part to help others (P7 FG5), to help themselves (P8 FG5), to give payback to the NHS (P2 FG5, P4 FG5), to avoid stockpiling (P2 FG5), P3 FG5) and because it is a good thing to do (P7 FG5). Pharmacy-based health-care professionals were interested from a health-economic/health-benefit perspective (HCP4 FG6, HCP2 FG6, HCP3 FG6, HCP6 FG6) and wanted to know whether or not the resource they put into preparing MODs was worthwhile with respect to patient benefit. Practice-based health-care professionals had chosen to support this study within their Primary Care Research Network (PCRN) research remit and because they were keen to know if MODs (trays) do benefit their patients (HCP1 FG6, HCP7 FG6). Box 7 provides statements of reasons for taking part for patients and Box 8 provides them for health-care professionals.

Communication about the study

This was a complex study and frequent and effective communication was required. Most communication was required between researchers, pharmacists and patients. Representative statements from focus group participants are presented in Box 9. The GP participant did not experience problems with communication (HCP7 FG6). Problems caused by the inability to source a suitable EAM caused delays to project timelines, and this was reflected in negative comments (HCP5 FG6). All patients were signed up to the automatic prescription order and collect system for preparation of medicines during the trial so that they would seamlessly revert to usual care at the end of the 2-month intervention. The process was not clearly defined in the study SOP and was not clear to all pharmacists as reflected in example comments (HCP4 FG6). However, some found the process very smooth (HCP6 FG6).

Patient behavioural changes

There is a body of evidence suggesting that participants do change their behaviour when they know they are being observed. It was therefore a concern that participants may change their usual behaviour as a result of their being observed. Evidence presented in Box 10 suggests that behaviour may have changed as a result of trial participation, with participants applying greater care than usual to take their medication as prescribed.

Impact of study on medical practices and pharmacies

Health-care professionals were asked about the impact of the study on surgeries or pharmacies to identify any knock-on issues. Representative statements are presented in Box 11. No impact on GP medical practices was identified (HCP7 FG6). Pharmacists identified that it had encouraged them to consider practical aspects of provision of MODs (HCP4 FG6, HCP5 FG6). HCP4 FG6 identified that a benefit of taking part would be patient loyalty to their pharmacy. Pharmacists reported taking a closer look at patients’ needs as a direct result of the trial (HCP5 FG6).

Pharmacists were required to provide study participants with their medicines according to randomisation group and to complete a proforma providing error data and resource data for the health economic analysis. These activities were time-consuming, but, once systems were established, both pharmacists and GPs found taking part in the study to be easy and acceptable, as evidenced in the interactions presented in Boxes 12 and 13.

A vehicle for patients to feedback to pharmacists

A number of comments, presented in Box 14, indicated that taking part in the study had an effect on the pharmacy. Some patients who took part in the trial had asked to stay on MODs (HCP5 FG6). There was consensus that pharmacists rarely get feedback, particularly when patients have medicines delivered to them (HCP4 FG6).

Medication-related issues

Under this theme, the subthemes emerging related to the MODs, the removal of medicines, the delivery of medicines, the return of packaging and wasted medicines.

Medication Organisation Devices

Patient perspectives on MODs are presented in Box 15. Medicines not included in the MODs were of concern (P6 FG5, P2 FG5), so it may be that better arrangements need to be made to integrate them into the system, such as synchronisation of supply. In addition to this, there were comments on difficulties in getting pills out of the box (P4 FG5, P1 FG6) and the usefulness of the MOD as an aide-mémoire.

Isolation of non-study medicines

Medicines were isolated from participants’ immediate use to prevent inadvertent use of previous supply instead of the study supply. They were placed into clear plastic bags which were sealed with a cable tie and stored in the usual place that participants stored medicines. The idea was that patients would cut through the bag to release medicines at the end of the study or could do so during the study if for any reason their planned medication supply was disrupted.

Removal of medicines was expected to cause participants the greatest concern. However, no negative sentiments were expressed, participants seemed relaxed about the medication removal process (P2 FG5) and it was readily accepted (Box 16). However, the focus group identified that some patients had difficulty getting into their medicines after the study (P4 FG5) and that inadvertent or even deliberate taking of removed medicines during the study would be very difficult without it being obvious to researchers (P7 FG5).

Removing old medicines was another important change. Patients on monthly medicines, by virtue of non-adherence and mixed pack sizes, for example 28- or 30-day pack, have a range of medicines in their possession that are at varying stages of use (full packs/half packs/nearly empty packs). Maintaining supply can therefore necessitate a high degree of organisation (P4 FG5, P2 FG5). The timing of visits to see the GP when new medicines are prescribed also impacts on this problem (HCP7 FG6). By removing previous medicines and supplying everything as new that degree of complexity has been removed. It is likely that alignment of supply could have a positive impact on adherence. Comments on alignment of medicines are presented in Box 17.

Delivery of medicines

Patients who were randomised to a weekly supply had their medicines delivered to them, patients on a monthly supply who usually had medicines delivered continued to have their medicines delivered to them and those who usually collected their medicines continued to collect them. Medicine deliveries were the aspect of the study eliciting most negative feedback and causing most concern. Comments pertaining to delivery and collection are presented in Box 18. Although the principle of delivery was acknowledged as potentially useful (P4 FG5), in practice, delivery in rural areas was problematic (P2 FG5). Erratic delivery or collection occurred for some (P7 FG5, P8 FG5), whereas others had no problem (P6 FG5) and it is intuitive that there would be more problems for weekly than for monthly delivery (eight deliveries compared with two; P7 FG5). Late and missed deliveries caused real inconvenience to patients (P2 FG5).

Health-care professionals identified confusion in patients who did not know why they had been allocated to delivery when they were happy to collect, annoyance in allocation to delivery in others and a misunderstanding on timing of collections (HCP4 FG6, HCP5 FG6, HCP3 FG6). Importantly, apart from the annoyance of waiting in for a delivery that did not materialise, it was identified that removal of autonomy with respect to collection of medicines made some patients feel vulnerable (HCP5 FG6). One participant suggested that patients should be asked whether they wanted to collect their medicines or to have it delivered.

Return of packaging

For successful adherence assessment in a definitive trial, return of medication packaging to the pharmacy would be required. Return is necessary in the case of DUC so that counts can be made without the need for an additional researcher visit and in the case of electronic monitoring so that data can be downloaded and monitoring units can be recycled onto new packs. Returns would be planned to coincide with usual pharmacy or patient delivery and collection, that is when a patient picks up their new supply they deposit their previous packs. Despite one-to-one and group training sessions, provision of itemised information sheets and SOPs for pharmacies and GPs and provision of containers in which to place medicines for return and explanation to patients at visits, it is clear that this aspect of the study design has been challenging. Information regarding pack return was either not retained or misunderstood by pharmacists and patients alike as evidenced by comments presented in Box 19.

Pharmacists indicated that they were unaware that packaging had to be returned to the pharmacy (HCP5 FG5) and that some patients may have been unaware of the requirement (HCP2 FG5).

Wasted medicines

Quarantining of existing drug stocks at the study start highlighted for professional participants the stockpiling and potential waste of medicines in patients’ homes and reminded them of Christmas and Easter ordering patterns. This is illustrated by comments presented in Box 20.

Study tools

This theme included the subthemes of participants’ comments on the test of participants’ ability, electronic monitoring and the return to usual care.

Tests of participant ability

Participant functional abilities were tested in order to assess the appropriateness of MOD supply. Comments on the tests used are presented in Box 21. No negative comments were received.

Electronic monitoring

Electronic monitoring was part of the original study design but was not implemented during the trial. Exploration of electronic monitoring issues identified mixed feelings. These are represented in Box 22. One interaction correctly identified that monitoring of packs cannot identify that a patient has ingested the medicines and therefore could be considered an expensive waste of money (P7 FG5). One indicated that they thought its use showed a lack of confidence in the patient but it is not clear whether this was a lack of confidence in ability or honesty (P4 FG5). One participant thought it would be helpful for people with dementia (P4 FG5), while another thought it would be worrying for a person with ‘real’ (perhaps meaning severe) dementia (P6 FG5).

Health-care professionals were very much in favour of electronic monitoring and identified with its clinical utility rather than research application. They expressed an interest in identifying the details of what people were really doing and the importance of having an objective way to determine that.

Return to usual care

Health-care professionals discussed difficulties surrounding the return to usual care at the end of the study. The possibility that patients may wish for continuation of their medication in a MOD had been carefully considered and ethical approval had been given for participants expressing such a wish to be referred to the Norfolk Medicines Support service by researchers or pharmacists. There was no plan to ask patients in receipt of a MOD for the study to be approached to determine whether or not they would like to continue on one. This position was clearly documented.

Most (22 out of 23) participants completing the RCT continued in the care of their usual pharmacist for the duration of the trial. All had been enrolled on the automatic order and collect scheme on recruitment to the study. They were informed that researchers would deregister them from the scheme on request. No participants requested deregistration. Pharmacist perceptions were that ‘the end of the trial has been very poorly communicated’ (HCP4 FG6) and this would include return to usual care. Comments on return to usual care are presented in Box 23. Although in some cases the return to usual care was smooth (HCP6 FG5) there was some consensus that a final visit or post-study pharmacy consultation with the patient would have been beneficial (HCP5 FG6, HCP4 FG6, HCP5 FG6).

Definitive study design considerations

In this final section, comments which would inform a future study are reported under the subthemes of issues for future participation, the types of patients to include and the way the study should be managed.

Issues for future participation

General experiences were mixed, but the overall response to the question ‘So then, now, having been through it, if you had your time over again, would you still have said yes?’ from all patients that commented was a resounding ‘yes’, indicating that, in general, the experience had been a positive one. Participants indicated willingness, ability and capacity to take part in a definitive study. While, overall, there was probably a net benefit to GP practices in terms of time, this was not the case for pharmacies. Should larger numbers of patients be involved per pharmacy in any future study, it would be challenging from both organisational and financial perspectives (Boxes 24 and 25).

Types of patient

Health-care professionals described the type of patients who should be considered for additional help with adherence, that is who are suited to MODs and therefore to a definitive study. Comments are presented in Box 26 and indicate that those who collect medicines on a haphazard basis (HCP7 FG6, HCP2 FG6) or who run out of medicines (HCP7 FG6) should be targeted.

Study management

Some comments were made on study management. Exchanges are presented in Box 27 and indicate that the relationship between patients, researchers and health-care professionals, albeit not perfect, was acceptable. Maintaining this good relationship in a larger study would be essential for successful completion.

Passive recruitment:

• Mailshot recruitment pack sent to eligible patients from the patient’s GP, comprising PIL, consent form, request and collect repeat prescription form, invitation letter (on headed paper), questionnaire 1 and a pre-paid return envelope. A reminder letter, on headed paper, was sent to non-responding potential recruits.

Active recruitment by general practitioner and researcher:

• Practice staff flagged any attending patients who were identified by the aforementioned search on the patient database.

• On attendance the GP/nurse briefly explained that there was a trial going on, provided a PIL and asked the patient to see the researcher in the waiting room if interested in taking part in the trial.

• Patients who chose to speak to the researcher and were interested in taking part, they were provided with a recruitment pack (as for passive recruitment but without GP invitation letter) to fill in at the surgery or to take away with them as they preferred.

Active recruitment took place over a 3-week period for each practice. At the end of that period all potentially eligible participants who had been identified by GP searches were posted recruitment packs. Some participants who were registered in the three active recruitment practices were therefore recruited via the mailshot process. These participants are not included in the analyses, which sought only to compare the efficiency of the two recruitment strategies.

For each recruitment strategy, efficiency was assessed in relation to two measures:

1. the number of people who consented to take part

2. the number of people who were randomised.

The former was chosen to enable simple comparison of consent rates between strategies. The latter was chosen to examine eligibility rates for consenting participants between recruitment strategies. Consenting participants may exhibit one of three adherence types: intentional non-adherence, full adherence or unintentional non-adherence. Only unintentionally non-adherent participants were eligible for randomisation. The recruitment strategy may differentially affect consent and eligibility rates, and therefore randomisation rate, and it may be that different proportions of consenting participants in each recruitment group are later identified as ineligible. For example, it may be that by explaining the study to the patient in person, a lower percentage of non-eligible people (e.g. intentionally non-adherent) will give consent.

Consenting eligible participants were subsequently visited, and provided with 1 month’s supply of medication (visit 1). They were visited again 3 weeks later to determine adherence (visit 2). If they were < 100% adherent, were still otherwise eligible and wished to continue in the study, they were randomised.

Costs associated with the activities listed for each of the recruitment strategies, plus costs associated with both visits 1 and 2 (excluding medication prescribed) were estimated. Medication costs were excluded as, although participants were given 1 month’s supply of medication at visit 1, this was akin to them receiving their next month’s supply earlier than would otherwise have been the case (they continued with their old stock after visit 2 if they were not randomised into the study). Costs were based on the 2012/13 financial year, as this was the time of recruitment (active recruitment took place between September and November 2012, and passive recruitment was undertaken in the three practices between August and September 2012). At the time of analysis, health-care professional costs were available only for the financial year 2011/12 and, thus, these were inflated by 2.6% (the rate of Hospital and Community Health Services pay inflation between 2010/11 and 2011/12) to provide estimated 2012/13 costs. Subsequently, costs associated with each strategy (excluding visit costs) were compared with the number who consented via each strategy (visit costs were excluded as the visits took place after consent was provided), and when visit costs were included the overall costs associated with each strategy were compared with the number who were randomised.

The costing method used varied according to the item costed. For example, when assigning the researcher time in the surgery to the individual participants, the top-down costing method81 was used, whereby the total cost of the researcher time was equally apportioned across the appropriate participants. Conversely, when estimating the costs associated with a visit to the participant’s home, the method was more akin to the bottom-up approach,81 as the resources associated with different cost items were estimated on an individual participant basis (although no particular questionnaire was completed prospectively to monitor these items).

Passive recruitment

Passive (postal) recruitment took place in three practices and a total of 995 patients were identified as potentially eligible, based on a search of the three practice lists. Each of these practices was paid a £175 set-up fee. GPs excluded 185 of the 995 patients (no additional cost to the aforementioned £175 was added for this activity) and subsequently 810 were sent recruitment packs. The cost of each recruitment pack sent was estimated to be £1.28 (including postage, but excluding the cost of return postage), which included the PIL (£0.12), consent form and request and collect repeat prescription form (£0.05), questionnaire 1 (£0.16), a pre-paid return envelope (£0.04) and postage (£0.90). Thus, the total cost of sending all recruitment packs amounted to £1036.80 (Table 32). After a further 2–3 weeks, a total of 703 non-responders were subsequently sent a reminder recruitment pack (unit cost = £1.28). In total, 152 people responded (return postage unit cost = £0.48), of whom 135 returned questionnaire 1 only and 113 returned both questionnaire 1 and the consent form. Table 32 shows the costs associated with each aforementioned component part. When these were assumed and divided by the number who consented, the cost per consent obtained amounted to £22.40.

Active recruitment/consent rate

Active recruitment was also undertaken in three practices, where 987 patients were identified as potentially eligible, based on a search of the three practice lists. Again, each of these practices were paid £175 set-up fee and the cost of the search was assumed to equate to this value. When any of the 987 patients who were suitable to participate made a routine appointment to see their GP, the researcher was informed and attended the relevant surgery. The average distance to each surgery was estimated to be 10 miles (mileage was reimbursed at £0.45 per mile in the study) and, thus, the travel cost associated with each GP surgery visit equated to £9.00. Across the three practices a total of 45 visits were made, giving an estimated total travel cost of £405.00.

The average salary of the researchers who undertook the surgery visits in this study was estimated to be £26,000. Assuming salary on-costs (employers’ national insurance and superannuation contribution) equated to 24% of the salary cost and that they worked 37.5 hours per week 42 weeks per year, the cost per hour of researcher time was £20.47. Based on an average speed of 29.9 miles per hour (mph), which is the average free-flow vehicle speed in a 30 mph limit,82 then it would take 2.01 minutes to travel 1 mile and the researcher time associated with the 20-mile round trip would thereby be 40.13 minutes. The cost of the researcher time associated with travel thereby amounted to £616.15 (see Table 32).

The time associated with each visit to the practice was as follows. The researcher was assumed to arrive 15 minutes before the time of the first pre-arranged participant appointment. This was important, as older patients often arrive well in advance of their appointment. The researcher would then be visible at the time the receptionist made the patient aware of the presence of the researcher. It also allowed time for preparation (arranging desk/chairs in position, sorting paperwork and checking for cancellations with the receptionist) so that researchers were available to speak to any potentially eligible participant who approached them after their appointment with the GP/practice nurse. Researcher time at the practice was dependent on how many potentially eligible participants had appointments in that session. Additionally, it was assumed researchers would stay an additional 10 minutes from the start time of the last appointment of any potentially eligible participant (to allow for the consultation with the GP/nurse to finish), 30 minutes to explain the study to any potentially eligible participant that approached them (if applicable), 10 minutes for discussion with practice staff (no additional payment, beyond the £175 practice payment, was made to the practice for this time) and a further 15 minutes to clear up (reinstate the room and clear paper work). The total attendance time (per session) was thereby equal to the time between the appointment times of the first and last potentially eligible participant plus 80 minutes (15 minutes’ preparation, 10 minutes’ appointment time, 30 minutes for discussion, 10 minutes’ discussion with receptionist and 15 minutes to clear up) if the last potentially eligible participant approached them (this applied to 38 of the 45 sessions) and 50 minutes (15 minutes’ preparation, 10 minutes’ appointment time, 10 minutes’ discussion with receptionist and 15 minutes to clear up) if the last potentially eligible participant did not approach them (this applied to 7 of the 45 sessions). The average time between the appointment times of the first and last potentially eligible participant was 82.44 minutes. Over the 45 sessions, the total time the researcher was in the surgery amounted to 7100 minutes (118 hours). At a cost of £20.47 per hour this gave a cost of £2422.26 for the time in the surgery (see Table 32).

Within these 45 sessions, a total of 98 potentially eligible participants approached the researcher, 52 of whom took a recruitment pack containing PIL (£0.12), consent form and request and collect form (£0.05), questionnaire 1 (£0.16) and a pre-paid return envelope (£0.04). Of these, 32 responded, 21 of whom completed both questionnaire 1 and the consent form. Table 32 shows the costs associated with each component part of the active recruitment; the cost per consent obtained amounted to £190.63.

Passive and active recruitment post consent

Those who consented and were deemed not intentionally non-adherent or otherwise ineligible were visited in their own home by the researcher (visit 1). The purpose of this visit was to provide the participant with 1 month’s supply of medication and to remove old medication stocks so that they could be visited 3 weeks later (at visit 2) to assess adherence. Thus, prior to the visit it was necessary for (1) the researcher to call the pharmacist, in order to arrange to collect medicines for the participant to be visited (10-minute telephone call), (2) the pharmacist to prepare the medication and (3) the researcher to collect it. The pre-visit pharmacy time for these three activities was assumed to average 45 minutes [at a cost per hour of employment of £51.30 (£50.0083 inflated by 2.6%)]; thus, the pharmacist cost would be £38.48 per participant. Assuming that the aforementioned travel costs to the GP practice also applied to the pharmacy (20-mile round trip, with an average speed of 29.9 mph), the total researcher time per pharmacy visit would be 40.31 minutes, which gave a total researcher time of 50.13 minutes when the 10-minute telephone call was included. Based on the aforementioned £20.47 per hour of employment, the pre-visit 1 researcher cost was £17.10. Thirty-nine participants from the three passive recruitment practices received visit 1, hence the total pre-visit cost for these participants was £2167.58. Eight participants from the active recruitment practices received visit 1, hence a total pre-visit 1 cost of £444.63 (see Table 32).

At visit 1 participants had their current stock of medication removed and were given 1 month’s supply of new medication. This visit was assumed to last 1 hour (excluding research-related time) and have the same travel time and cost as that for the aforementioned GP visits (although participants often lived further away, it was possible to combine some of these visits). Total researcher time per visit was thereby 1 hour 40.13 minutes at a cost of £34.16, with travel costs of £9.00, giving a total visit 1 cost of £43.16. Total visit 1 costs for the eight active recruitment participants and the 39 passive recruitment participants are shown in Table 32.

Visit 2 took place 3 weeks after visit 1 and was primarily undertaken to monitor adherence by pill count. If participants had taken either more or less than 100% of their medication they were deemed unintentionally non-adherent. After excluding the time associated with research activities (e.g. MMSE, vision test, dexterity test, pill count, randomisation), the time for assessment and to ensure that the participant could use their allocated medication packaging amounted to 30 minutes. Using the same travel costs as visit 1, the total cost per visit 2 amounted to £32.97, which was applied to eight active recruitment participants and the 37 passive recruitment participants.

When the costs associated with each of the aforementioned activities (both pre and post consent) are summed together, the total cost was estimated to be £7600.56 and £5056.64, for passive and active recruitment, respectively (see Table 32). At the end of visit 2 a total of 15 participants were randomised from the three passive practices, compared with two from the active practices, giving a cost per-participant randomisation rate of £506.70 and £2528.32, respectively.

Measuring costs

Assigning costs to items of resource use

Costs were estimated at 2011/12 financial year levels, as such values were reported in the latest edition of Curtis85 available at the time of analysis.

Overall and incremental costs

Overall, mean costs for each of the aforementioned main cost categories [intervention costs (medication costs were reported separately) and other costs] were estimated over the 2-month intervention period. These are reported for each of the four trial groups, as well as overall (undertaken because of the small numbers), in order to provide an indication of the main cost drivers for the population group and, in particular, the proportion of overall costs that the intervention costs constitute.

Measuring outcomes

To estimate the impact on health-related quality of life, participants were asked to complete the EQ-5D89 at baseline and 2 months post randomisation. The EQ-5D has five questions, which ask the respondent to report the level of problems they have (no problems, some/moderate problems or severe/extreme problems) with regard to mobility, self-care, usual activities, pain and anxiety/depression. 89 The three-level version of the EQ-5D (EQ-5D-3L) was used. Responses to the five dimensions are converted into one of 243 different EQ-5D health state descriptions, which range between no problems in all five dimensions (11111) and severe/extreme problems in all five dimensions (33333). A utility score (a scale where death is equal to 0 and full health 1) was assigned to each of these 243 health states using the York A1 tariff91 (producing EQ-5D scores ranging from –0.594 to 1.00). The EQ-5D is accompanied by a visual analogue scale (European Quality of Life-Visual Analogue Scale, EQ-VAS), on which people are asked to indicate how good or bad their health state is (on the day they complete the questionnaire), where zero corresponds to worst imaginable health state and 100 to best imaginable health state. 89

In addition to the EQ-5D, participants were also asked to complete the ICECAP-O92 at baseline and at the 2-month follow-up. The ICECAP-O is designed to capture both quality of life and capability, for which the focus is on an individual’s own perception of their capabilities on the five attributes of attachment, security, role, enjoyment and control. 92 Each of the attributes has four levels: the most preferred state, all of . . . (e.g. the love and friendship) I want (4), a lot of . . . (3), a little of . . . (2) and none of . . . (1). 92 The best–worst scaling technique was used to assign a value to each of the capability descriptions that are ascertained from the ICECAP-O, where these range from 0 (no capability: 11111) to 1 (full capability: 44444).

Participants

Participant allocations and characteristics are summarised in the CONSORT diagram (see Figure 7) and baseline characteristics table (see Table 18). It should be noted that 3 of the 29 randomised participants withdrew consent prior to receiving the intervention (two from the MOD weekly arm and one from the usual supply monthly arm). After the point of withdrawal for these three participants, no further data could be collected. Thus, dispensing time data were not available for these three participants and their primary record data had also not been accessed at this point; consequently, data from these three participants are not used in the subsequent costs analyses. Additionally, one participant from the MOD weekly arm died; data from this participant are used up to the point of death. Resource use data were also available for the participant who had poor eyesight (usual supply monthly arm). Cost data were thereby available for up to seven participants in the MOD weekly arm, six in the MOD weekly arm, six in the usual supply monthly arm and seven in the usual supply weekly arm. With regard to outcomes, it was possible to use the baseline data for the 29 randomised participants (the three who withdrew completed the measures prior to withdrawing consent). Follow-up data were, however, only available for up to 26 participants. These numbers differ from those in the CONSORT, as the CONSORT is based on the primary outcome (pill count), for which levels of missing data differed from resource use and quality-of-life data (all available data were used in the economic analysis).

Intervention costs

In this study each MOD covered a 1-week period and had a cost of 40p (this cost was assigned to all types of MODs used). Two participants were admitted to hospital (where they did not use a MOD as they were not self-medicating); the remainder of those in both the MOD monthly and MOD weekly arms were provided with eight MODs in the study period. Thus, the mean number of MODs used in each of these arms was eight (the one participant admitted to hospital in this study arm was in the last week of the 2-month study period, so the cost of eight MODs was still included for that participant) and 7.36 (one of the six participants in this arm was admitted to hospital in their fifth week in the study), respectively, giving mean associated MOD costs of £3.20 and £3.00, respectively.

Table 33 summarises the direct costs associated with the four study arms. Medication delivery costs were £5 per trip, and the mean cost associated with these is also shown in Table 33. Dispensing times were recorded for 25 of the 26 participants. The mean time taken to dispense 1 month’s supply of medication (including filling the MOD, if applicable) varied between 19.50 minutes for usual supply monthly (n = 5) and 43.00 minutes for MOD weekly. Once the appropriate figures for pharmacist/pharmacist technician cost per hour of employment (Table 34) were assigned to these costs, it can be seen that (for both usual supply and MOD) mean dispensing costs were higher for those in the weekly arm than for those in the monthly arm and that mean costs in both MOD arms were higher than in the usual supply arm.

Pre-intervention, the mean number of medications per participant varied between 5.50 (MOD weekly) and 7.86 (MOD weekly), where the associated cost also varied between £31.45 and £70.48, respectively (see Table 33). Within the 2-month intervention period, four of the 26 participants had a change in their medication recorded by the pharmacist, where this was the addition of one medication for two participants, the addition of two medications for one participant and the removal of one medication for one participant. Costs/levels of resource use were only estimated for prescribed SODF medication that was prescribed as a regular dose.

Other costs

Medical records were accessed for each of the 26 participants. The mean number of primary/community care visits, outpatient visits, A&E visits, day-case attendances and hospital admissions, within each of the four groups are summarised in Table 35. Unit costs (see Table 35) were subsequently assigned to each of these items of resource use, in order to estimate the mean costs for each of these categories (see Table 35). It can be seen that many of the values are low, and this can be explained by the fact that the most common value for each item was zero. Moreover, it can be seen that one or two individuals have quite an effect on the overall mean, for example there was just one participant who was admitted to hospital in the MOD weekly arm (their length of stay was 32 days).

Nine carer questionnaires were returned at the 2-month follow-up point, although in one of these questionnaires the person reported that they were the daughter of the participant and not a carer (they met with the participant on a weekly basis); consequently, this participant was assumed to receive no help from a carer. All of the remaining eight carers reported that they were family members, seven of whom lived with the participant. Six of the carers answered the question regarding whether or not they helped the participant with organising or taking their medication, three of whom gave the answer of ‘yes’ (one of these three reported that they received payment for this help). Two of those three were in the MOD weekly arm; of these two, the reported time providing help was once per week for 5 minutes and seven times for 3 minutes on each occasion. Assuming these times applied to the whole of the 8-week period, this would amount to 0.67 hours and 2.80 hours, respectively, and lost productivity costs of £9.84 and £41.33, respectively, when assigned the rate of average hourly earnings in 2011 (£14.76). The other carer who reported that they helped with organising or taking medication was in the MOD weekly arm and reported providing help three times per week (for 2 minutes on each occasion); this equated to a lost productivity cost of £11.81.

In order to calculate average levels of lost productivity, it was necessary to make assumptions with regard to those for whom the carer questionnaire was not returned, specifically whether or not this was because they did not have a carer (and consequently received no such help), and whether or not carer data should be treated as missing. Participants were asked to return the carer questionnaire at the same time as the EQ-5D questionnaire. Thus, we assumed that those who returned the EQ-5D questionnaire, but not the carer questionnaire, did not have a carer.

This applied to 13 participants. For the remaining participants who did not return the EQ-5D questionnaire, or who returned the carer questionnaire but did not complete the question with regard to help regarding medication (n = 2), we assumed the data were missing with regard to whether or not they had a carer and received any associated help. The estimated mean lost productivity costs associated with organising or taking medication across each of the four groups are shown in Table 36, in which the three aforementioned lost productivity values are combined with values of £0.00 for the three participants who reported that they did not provide such help, the one person who returned the carer questionnaire but reported that she was not a carer and the 13 who returned the EQ-5D questionnaire but not the carer questionnaire.

Seven of the eight responding carers answered the question regarding other types of help, six of whom reported that they provided such help (two of the six reported that they received a payment for such help). One of those six failed to complete the questions regarding the frequency/length of help, and another (from the usual supply weekly arm) reported that they felt unable to answer this question as they provided help ‘all day’ (providing help 14 hours a day for 7 days a week would equate to a lost productivity cost of £11,571.84 over 2 months). The times reported by the remaining four ranged between 2.5 minutes twice a week (to remind the participant about appointments) and 1 hour 7 days a week (for personal care). The inferred lost productivity costs for these four are combined with the 15 who were assumed to have received no such help (the one carer who reported that they provided no help, the one who returned the questionnaire but reported she was not a carer and the 13 who returned the EQ-5D questionnaire but not the carer questionnaire) in Table 36 (for the purposes of this analysis the responses for the one carer who reported that they provided help ‘all day’, the carer who reported they provided help but did not report the frequency/length and the one carer who returned the questionnaire but did not answer this specific question were treated as missing data).

Missing data

The following assumptions were made in order to enable all 26 participants to be assigned a cost for each component cost. The one participant (from the usual supply monthly arm) for whom the method of delivery/dispensing times were not recorded had been found to have difficulties with regard to eyesight and been withdrawn from usual supply by his GP [he was instead provided with a MOD (by the NHS outwith this study)]. In line with the intention-to-treat principle, we included this participant in the analysis, in the arm to which they were originally allocated. The costs associated with the method of delivery/dispensing times for this participant were assumed to equate to the mean for the seven participants in the MOD weekly arm. Similarly, in line with all participants in the MOD weekly arm, he was assumed to have used eight MOD boxes within the study period.

When the costs associated with the intervention [MOD boxes (if applicable), dispensing and delivery (if applicable), but excluding medication] are summed together, it can be seen that these are relatively small compared with other NHS costs. Over the 26 participants, on average, they can be seen to be approximately 10% of the overall cost. In terms of the MOD intervention itself, it can be seen that the mean intervention cost for those in the MOD monthly arm is just over £20 more than that in the monthly usual-care arm, with a similar, although slightly smaller, mean difference in the intervention cost between the MOD weekly arm and usual supply weekly arm (see Table 35). There is also no evidence to suggest that the higher intervention costs are off-set by lower other NHS costs; indeed, the mean other NHS costs for both usual supply arms are lower than those for both of the MOD arms. It should, however, be noted that these costs are highly influenced by the two participants who were admitted to hospital (both of whom were in one of the MOD arms).

Outcomes

At baseline, the EQ-5D, EQ-VAS and ICECAP-O were completed by 28 of the 29 participants (see Table 37). Follow-up scores were completed by 22 of the 29 participants (84.6% response rate). Looking at the change in scores, it is difficult to draw firm conclusions, but there is little to suggest that the mean scores for those who received a MOD, whether this be on a weekly or monthly basis, were superior to those who received usual supply. Indeed, of the four groups, a usual supply arm had the highest mean change according to the EQ-5D (usual supply weekly), EQ-VAS (usual supply monthly) and ICECAP-O (usual supply weekly).

Cost-effectiveness analysis

Given the small numbers, formal cost-effectiveness analysis is considered inappropriate. There is, however, little to suggest that levels are greater for the MOD intervention, as for each of the measures, based solely on the mean unadjusted results, it can be seen that a usual supply arm dominates (has lower mean costs and higher mean effects) both of the MOD arms (see Table 36 for costs and Table 37 for outcomes).

Value of information analysis

We based our value of information calculations on those participants for whom complete cost and effect data were available. The resulting estimates of the within-trial mean cost and QALY gain/loss for each of the four groups are shown in Table 38. The option of usual supply weekly was optimal (had the highest ICER below the cost-effectiveness threshold) for values of λ ≥ £119 per QALY. Consequently, the CEAF (shown in Figure 9) was equal to the probability that usual supply weekly was estimated to be cost-effective for threshold values greater than this value of λ and took the value of the probability for MOD weekly for λ values < £119. The per-participant EVPI was subsequently calculated and is shown in Figure 9. The EVPI never exceeded £50 and is equivalent to an estimated value of £2.91 at λ = £20,000 per QALY (see Table 38). The EVPI is equivalent to the probability of making the wrong decision multiplied by the consequences of that wrong decision, and this low EVPI estimate can be largely explained by the fact that the probability of making the wrong decision (as depicted by the CEAF in Figure 9) was ≤ 15% for values of λ > £10,000 per QALY and ≤ 5% for values of λ > £20,000 per QALY.

FIGURE 9.

Estimates of the CEAF and EVPI at different levels of the cost-effectiveness threshold.

If one took the EVPI per-participant value of £2.91 at face value, then one would probably conclude that the potential value of undertaking further research is relatively low. However, there are a number of reasons why we consider that the EVPI estimates produced here should be treated with caution. First, although complete data were available for a large proportion of the sample (22 out of 26), with such small numbers in the sample it can be seen that the mean estimates for those with complete cost and effect data (see Table 38) are somewhat different from those for participants for whom only cost data were complete (see Table 36). Contributory factors include the fact that the participant with the highest NHS cost in the monthly MOD group failed to complete the EQ-5D, as did the participant with the lowest NHS cost in the MOD weekly group. Thus, the mean cost estimates, based on those with complete data, may underestimate for the monthly MOD group and overestimate for the MOD weekly group. Furthermore, a key assumption of bootstrapping (which was used to produce the iterations on which the mean cost and effect data are based) is that the observed data are representative of the population and that estimates of the mean of the population can therefore be made by sampling from the observed data. Given the small numbers in this study, and the fact that those who have missing data may differ from those with complete data, the above assumption is unlikely to hold and the use of bootstrapping may therefore be inappropriate. A further issue is that the mean cost and QALY estimates here were based on the within-trial period of just 2 months. As such, they may underestimate the changes in lifetime costs and effects (we thought it too speculative to produce such estimates based on the data available to us) and the resulting estimates of the consequences of making the wrong decisions which is used in the EVPI calculations. In the light of the above, we consider that the value of information results produced here should be treated with caution. Indeed, similar to other pilot studies that have included an economic component, for example Clarke et al. ,100 we would prefer to concentrate on the completion rates for the different measures (to assess whether or not it would be feasible to use these measures within any subsequent definitive trial).