Immediate oral versus immediate topical versus delayed oral antibiotics for children with acute otitis media with discharge: the REST three-arm non-inferiority electronic platform-supported RCT

Previous evidence and experience

Data standards for research data collection have been formulated by the clinical trials community via The Collaborative Data Standards Interchange Consortium (CDISC) over several decades, with an established pathway for data management from source to submission for regulated clinical trials. Using CDISC standards (www.cdisc.org), there has been a steady move away from paper case report forms (CRFs) towards electronic data capture (EDC) systems. Given the rapid expansion of the use of electronic health record (EHR) systems in clinical settings, it has been proposed that EHRs could be the primary point of data entry for a clinical trial. However, direct collection of data into a digital form, referred to as electronic source (eSource), can be achieved only if the EHR is able to support research-quality data collection. Good clinical practice (GCP) principles need to be adhered to ensure that the requisite standards are in place for eSource, and changes are needed to the data collection process and governing regulations to fit this electronic context. 35

There are three models of eSource currently being explored: (1) entry into a Clinical Trial Data Management System (CTDMS) with transfer to the EHR; (2) entry into a CTDMS with copying to both the EHR and the EDC form; and (3) collection within the EHR with transfer to the EDC form. Local preferences, maturity of EHR systems and sponsor requirements are likely to maintain this heterogeneous approach, emphasising the paramount importance of adherence to standards. The Integrating the Healthcare Enterprise (IHE) collaboration36 has developed a set of profiles for eSources:37 the Retrieve Form for Data Capture (RFD) and Retrieve Process for Execution (RPE) specify forms and workflow, respectively. Several proof-of-concept studies using IHE profiles have been completed. These include integration of common data elements from the National Center for Biotechnology Information (NCBI) Cancer Biomedical Informatics Grid (CaBIG) Enterprise Vocabulary Services (EVS) into the RFD profile,38 and STARBRITE (Strategies for Tailoring Advanced Heart Failure Regimens in the Outpatient Setting: Brain Natriuretic Peptide versus the Clinical Congestion Score), a single-site proof-of-concept implementation within a heart failure clinical trial. 39

Within the academic and pharmaceutical trials world, there has been a move to ‘real-world’ clinical trials as a means of gathering a larger number of data more quickly on the likely effectiveness of treatments, to satisfy increased regulatory requirements in this area. 40 It is proposed that embedding and integrating research into electronic record systems would enable automation of some elements of the trial’s screening process, with eligibility criteria matched directly with EHR-held data. 41 Potential participants who match the exclusion criteria need not be flagged. In addition, for those who are potentially eligible, data held in the EHR can be used to pre-populate the eligibility form. A similar process can be used to pre-fill electronic case report forms (eCRFs). In a reverse of this process, trial data can be added back to the EHR. 42 The ability to place trial information in routine EHRs at the point of collection would be a significant step towards safer and more efficient clinical trials. 43

Real-world trials have not yet progressed to using eSource by default, still requiring a large investment in data collection and validation. 44 Closing this gap would go a long way to providing an end-to-end ‘research and learning’ continuum for a learning health system (LHS), in which research and knowledge translation are routinely transacted using information technology (IT) systems. When interacting with EHRs, the use of robust data standards, such as the CDISC suite, is essential to the operation of the LHS to overcome the ‘silo of excellence’ culture prominent in health-care research and lower the barrier to entry for traditional clinical environments. 35,45

For the past 10 years, the US Food and Drug Administration (FDA) and the European Commission have been advocating the use of electronic platforms to manage clinical trials, with the source data obtained directly from the EHRs. 46 Advantages of this include:

• increased data accuracy, including anonymised recording of the characteristics of eligible patients who decline to participate, thereby providing a greater understanding of final sample representativeness

• reductions in data management

• increased safety, by ensuring that trial data are within the clinical record

• easier and, therefore, more efficient trial monitoring

• EHR management of trial workflow, prompts and alerts for recruitment and follow-up, and patient-reported outcomes47

• the use of CDISC standards for data capture. 48

Participant identification

Within-consultation ‘hot’ recruitment of patients with incident conditions is significantly more challenging than ‘cold’ recruitment of patients with prevalent conditions, as the latter group of patients can be contacted electronically or by letter. 37 The extra workload and time needed to set up and recruit within a normal consultation are major barriers to participation by GPs and general practices. 37

Participation can be increased when there is perceived clinical value and/or benefit to patients, adequate remuneration for time and streamlined recruitment processes that minimise workload. 49 One approach to overcoming these barriers is through the use of a trial platform.

REST was designed to collect quantitative data using the TRANSFoRm platform, originally developed as part of the EU FP7-funded TRANSFoRm Programme (i.e. the seventh framework programme of the European Community for research and technological development including demonstration activities). 31 The TRANSFoRm platform is designed to integrate with primary care EHR, ensuring data validity and accuracy and facilitating the nationwide engagement of the large number of primary care sites needed for REST. An additional module enables patient-reported outcome measures (PROMs) to be recorded by parents using the internet and smartphones [iOS (iPhone operating system; Apple Inc., Cupertino, CA, USA) and Android (Google Inc., Mountain View, CA, USA)]. The system was fully GCP validated as part of a European trial on gastro-oesophageal reflux diseases and registered with EudraCT (number 2014-001314-25).

The basic components of the system are:

1. a TRANSFoRm Study System (TSS) that manages projects, sites and workflow

2. middleware that manages authentication and messaging

3. a system for triggering and storing PROMs

4. data node connectors (DNCs), specific to each EHR system, that link clinical systems to the TSS via their application programming interface (API)

5. an online back-up data collection system.

For REST, relevant data elements were captured using a set of extensible markup language (XML) files, and linked to the TRANSFoRm clinical data information model (CDIM), structured in accordance with the CDISC Operational Data Model (ODM), and the timeline according to the CDISC study data model (SDM). Further ODM files containing questions for the PROMs were developed in combination with structured searches for data elements that were pre-populated by the data in the EHR.

Two areas of the added value of the TRANSFoRm platform specific to REST were (1) the triggering of ‘real-time’ eligibility reminders when potential recruits were being seen by clinicians and (2) streamlined study processes after identification, with access to REST-specific documentation, pre-populated consent forms and ‘real-time’ randomisation.

Oral antibiotics

For the oral antibiotics (immediate and delayed), we wanted to reflect routine care and appropriate bacteriological cover. A 2010 study of 256 children with AOM recruited from primary care3 showed that 84% of the 38 children with AOMd received an immediate prescription for amoxicillin, with a further 5% receiving oral erythromycin, 3% receiving topical gentamicin and 8% receiving no antibiotic. In the majority of children [22/38 (58%)] a recognised bacterial pathogen was isolated: Streptococcus pneumoniae (n = 5), group A Streptococcus (n = 7), Staphylococcus aureus (n = 7), Pseudomonas (n = 2) and Haemophilus influenzae (n = 3). 3 A study of 177 children with AOMd isolated single pathogens in 70 (39%) samples, whereas two, three and four bacterial species were detected in 54 (30%), 20 (11%) and 7 (4%) cases, respectively. 50 Non-typeable H. influenzae was the most common and was identified in 90 children (51%), followed by Moraxella catarrhalis (35%) and S. pneumoniae (27%). 50 Children with co-infections, including non-typeable H. influenzae, had significantly more frequent recurrent AOM (adjusted odds ratio 6.6; p = 0.029). 50

Our 2015 audit (33 general practices; 56,251 children) confirmed immediate oral antibiotics as usual care for AOMd: 88% were given oral antibiotics (of which 97% were immediate), with amoxicillin being the most widely prescribed antibiotic. UK primary care practice is to prescribe amoxicillin ‘dose-by-age’. However, we used the latest British National Formulary (BNF) for Children51 prescribing guidance to prevent underdosing and overdosing the oldest and youngest children, respectively. Clarithromycin is a commonly used and well-tolerated alternative for penicillin-allergic children; hence we selected oral amoxicillin, with the option of clarithromycin in the event of recorded penicillin allergy.

Topical antibiotics

We considered there to be four main advantages to selecting ciprofloxacin 0.3% as the topical antibiotic. First, it has low potential for AMR as bloodstream absorption of ear drop quinolones has been shown to be extremely low, measured at no higher than 10 ng/ml in one study52 of children and adults given ofloxacin 0.3% solution, which is < 1% of the concentration typically seen after oral dosing. We used four drops of ciprofloxacin 0.3%, taken three times daily, and expected to see similar low blood levels to those reported for ofloxacin. Typically a drop is about 50 µl,53 giving a total daily dose of 1.8 mg. Although we are not aware of any studies measuring respiratory or gastrointestinal tract quinolone exposures following ear use, even if 100% of the ciprofloxacin was absorbed and excreted into the gut, this would be < 2 mg per day. By contrast, the typical oral dose for a child is 10 mg/kg, taken two or three times daily – equating to 300 mg daily for a 10-kg child aged 1 year. It is not surprising that evidence suggests that the risk of developing AMR post ear drop application in these locations is very low. 54

Second, ciprofloxacin ophthalmic 0.3% drops are widely available, which was a requirement for this study, as recruitment took place in a large number of primary care sites dispensed by high street pharmacies in response to standard FP10 NHS prescriptions. It is usual practice in primary and secondary care to use the eye drop formulation for ear treatment because the ophthalmic formulation is considerably less expensive and more readily available than the otic formulation. Prescribing in REST was therefore outside the licence, and the study sponsor ensured that the University of Bristol’s no-fault indemnity applied to this form of ciprofloxacin.

Third, the drops are colourless and odourless, so they did not interfere with parental assessment of otorrhoea. Finally, at the concentrations achieved in the middle ear, ciprofloxacin 0.3% drops are active against the most commonly isolated otorrhoea microbes from children presenting to primary care, namely S. pneumoniae, S. aureus, H. influenzae and Pseudomonas aeruginosa. 3

We decided against using a topical aminoglycoside, mainly because of the potential for ototoxicity. Topical antibiotics are thought to penetrate from the middle to inner ear via the round window. This results in high local concentrations in the inner ear tissues and potential for ototoxicity. One systematic review in animals reported widespread ototoxicity with topical aminoglycoside antibiotics,55 and the ototoxic properties of aminoglycosides have been intentionally used to ablate vestibular function in humans with severe middle ear disease. 56 There is also a genetic mutation, occurring in around 1% of the population, which predisposes those with the mutation to ototoxicity at low drug exposures. 57 Although the evidence for aminoglycoside ototoxicity is debated in humans, especially at the time of an active infection,58 and there are other advantages of aminoglycoside preparations (e.g. unlike ciprofloxacin, aminoglycoside–steroid combination preparations are widely available). The BNF59 states that topical aminoglycosides are contraindicated in patients with a perforation of the eardrum or grommets, and such medicolegal concerns would have significantly reduced the willingness of trial clinicians to recruit.

We also decided against an antibiotic–steroid combination drop because, although there is evidence that, for children with grommets, a topical steroid–antibiotic combination is superior to an oral antibiotic alone18 and to topical antibiotic alone60,61 in reducing the discharge, an industry study has shown that ciprofloxacin alone is more effective than a steroid only preparation. 62 This suggests that, although steroids may have additional benefit to antibiotics, antibiotics are an essential ingredient. Some studies performed in animals have shown that an antibiotic–steroid combination slows the healing of perforated tympanic membranes compared with antibiotics alone. 63,64 Although a number of aminoglycoside steroid-containing preparations are available, no combination of steroid with ciprofloxacin was widely available in the UK at the time of trial design, meaning that ciprofloxacin was not only suitable, but the only non-aminoglycoside topical antibiotic option available.

Randomisation

Concealed randomisation stratified by age (< 2 vs. ≥ 2 years) was used to ensure that treatment arms were similar with respect to both measured and unmeasured potential confounders.

Treatment crossover and adherence

In an open-label trial, which we considered necessary for REST, there is a possibility that children would not be given the treatment to which they were randomised. There is no single agreed threshold at which patients are regarded as ‘adherent’ (and it is likely to vary between diseases and medication classes), but 80% is often regarded as reasonable. 67 Higher levels of adherence than this were achieved in the previous open trial of oral versus topical antibiotics for children with grommets and ear discharge:18 88% and 93% fully adhered to oral and topical antibiotics, respectively. Minimising treatment crossover was a key clinician training element, and the TRANSFoRm platform minimised crossover by guiding clinicians to issue the ‘correct’ treatment. Finally, treatment adherence was monitored both by checking the drug prescribed at the notes review and by using the SRQ.

Performance, measurement and attrition bias

Although participants were not blinded to treatment allocation, given current treatment equipoise and the fact that all participants would receive active antibiotic treatment, we did not consider that parent knowledge of treatment allocation would significantly influence their perception of symptom severity. Members of the TMG and the statistical team remained blind to treatment allocation until analyses were completed. REST used outcome measures successfully used in our previous studies,1,27,28 which had been shown to be valid26 and sensitive to change. 1,27,28 We aimed to achieve a minimum follow-up rate of 80%, but with online data collection and telephone support from an experienced research nurse, we anticipated achieving closer to 90% for our primary outcome. 1,27,28

Recruitment reminders

The system was intended to respond to clinicians’ use of predetermined Read diagnostic codes by triggering a reminder ‘pop-up’ alerting the clinician that the child might be eligible for REST.

Read and International Statistical Classification of Diseases and Related Health Problems, Tenth Revision,72 (ICD-10) clinical codes describing members of the AOM subclass (Table 2) were added to the TRANSFoRm platform’s study database. The scope was intentionally broad (i.e. sensitive) to maximise the chances of identifying potential participants using the wide range of codes used by clinicians. This was followed by application of the (specific) eligibility criteria supplied at the beginning of the (electronic) recruitment process to ensure that the patient was suitable for the clinical trial. Specifically, the steps followed were:

1. Identify open consultation. The TRANSFoRm platform’s DNC, using SystmOne^®’s [The Phoenix Partnership (Leeds) Ltd, Leeds, UK] API, asked to extract information from the currently opened EHR, if any. If the request was not successful, then no consultation was in process. Otherwise, a consultation was taking place and a summarised view, containing demographic data and clinical codes inserted in the record on the current date, was temporarily stored in a local folder of the DNC. Then, a call to the TRANSFoRm platform’s TSS retrieved the given list of Read and ICD-10 clinical codes and the age comparison expression.

2. Match patient age. The system matched the patient’s age, as extracted from the summarised view, with the comparison expression. If the matching proved unsuccessful, the recruitment workflow stopped.

3. Match clinical codes. A search took place to match clinical codes from both the summarised view and the data obtained from the study database.

4. Identify prior trial entry. The algorithm searched for any trial-related clinical code that may have been already inserted in previous consultations with the same patient (e.g. Read code ‘XaN0L::Consented’ inserted into record when consent form is submitted by user) or declined (e.g. Read code ‘XaaFk::Declined’ inserted into record when decliners form is submitted by user).

5. Following the sequence 1–4. A confirmation pop-up dialog was displayed to the user before initiating the recruitment process. Simultaneously, a trial-related clinical code was inserted into the patient record to denote potential eligibility to the trial (e.g. Read code ‘XaaEl::Screened’).

6. Pop-up disposal. Disposal of the pop-up reminder window was undertaken by the clinician following Microsoft Windows^® (Microsoft Corporation, Redmond, WA, USA) graphical user interface (GUI) standards, that is clicking on the red cross icon located on the top-right corner of the pop-up window. The same action to close the window was taken regardless of the outcome (user declined, consented or did not follow through with the recruitment workflow). This was because of the complexity of having more than one possible workflow path at a time and the lack of restriction on which electronic forms had to be filled in first.

Consenting

The TRANSFoRm platform prompted and facilitated the clinician to print the REST consent form and, after the parent had signed the form, to indicate consent on the system, initiating the trial’s workflow.

Case report form completion

The eCRFs were presented to clinicians at appropriate points to complete. These were then automatically entered into the study database. Some information was retrieved directly from the SystmOne record and used to partially fill in the form, which could be amended by the user. Once submitted, confirmation was displayed to the user on the screen and completed forms were stored in the study database and the SystmOne record. For SystmOne, a link to the local copy of the completed record was added. To facilitate subsequent tracking of participant progress, trial-specific codes were added to the record (1) when a patient was classified as potentially eligible by the plugin, (2) after submission of the consent or decliner form and (3) on trial completion.

Recruitment process data collection

There were two approaches planned to collect process frequency data. The first involved the recording of information relevant to the submission of eCRFs by the user. Recorded data consisted of information on the closed pop-up window (see Table 2) or submitted eCRF (consented or declined eCRF), that is the identifier of the user interacting with the form, the NHS number of the patient and the current date and time. The recorded data were stored locally in a dedicated folder in the clinician’s or nurse’s computer and could be viewed or shared by them only. The patient’s NHS number was added to account for the possibility of a patient being initially discarded by the clinician as a potential candidate but recruited at a later stage by the same or a different user. This approach was, in the event, unsuccessful, as local files stored during the running of the recruitment process were overwritten each time an upgraded version of the DNC was installed. This issue was due to the architecture of the DNC and the restrictions imposed by NHS computers on installation of third-party software, such as the TRANSFoRm platform (e.g. only Windows-based administration members could permit installation).

The second approach involved auditing the EHR for trial-related clinical codes inserted by the TRANSFoRm platform’s DNC during the recruitment workflow. These clinical codes denote potentially eligible and consented participants, as well as patients who declined participation.

In November 2019, we tried to estimate the frequency of pop-up disposal by contacting 12 randomly selected sites (of the 38 open to recruitment at the time) to request data on the number of times the TRANSFoRm platform pop-up had been responded to (closed without action, declined or consented) in the previous 6 months.

Inclusion

• Aged ≥ 12 months to < 16 years.

• Presenting with recent-onset (≤ 7 days) unilateral AOM with recent-onset (≤ 7 days) otorrhoea, currently visible or seen by parent within the last 24 hours.

• Attending with a parent who is legally able to give consent.

• Parent is willing and able to administer ear drops.

• Parent is willing, able and available to complete the daily SRQ and receive regular telephone calls from the study team.

Exclusion

• Symptoms or signs suggestive of bilateral AOM or AOMd.

• Symptoms or signs suggestive of serious illness and/or complications (e.g. mastoiditis and/or requires immediate hospitalisation).

• Requires immediate oral antibiotics.

• Child is at high risk of serious complications:

  • Significant immunosuppression.

  • Heart, lung, renal, liver or neuromuscular disease comorbidities.

  • Trisomy 21, cystic fibrosis or craniofacial malformation, such as cleft palate.

• Grommet tube in situ in the otorrhoea ear.

• Currently on oral or topical antibiotics.

• Allergy to ciprofloxacin.

• Allergy to penicillin or anaphylactic reaction to another beta-lactam agent, and allergy to clarithromycin.

• Child had taken part in any research involving medicines within the last 90 days.

• Child had already participated in this trial.

Intervention 1

Intervention 1 was immediate ciprofloxacin (0.3%) ear drop solution, four drops given three times per day for 7 days, with an advice sheet on how to administer the ear drops and standardised symptom management advice.

Intervention 2

Intervention 2 was delayed dose-by-age oral amoxicillin suspension, given three times per day (clarithromycin twice daily if allergic to penicillin or another suitable oral antibiotic as chosen by the GP) for 7 days, and an advice sheet containing structured delaying advice and standardised symptom management advice.

Comparator

The comparator was immediate dose-by-age oral amoxicillin (clarithromycin twice daily if allergic to penicillin or other suitable oral antibiotic as chosen by the GP), given three times per day for 7 days, and standardised symptom management advice.

Days 2–14

The SRQ was provided in electronic format (for either web or iOS/Android applications) via the TRANSFoRm platform. Parents recorded the daily presence and severity of the following AOMd symptoms (until cessation of all symptoms without need for analgesia): pain, fever, being unwell, sleep disturbance, otorrhoea, episodes of distress/crying, appetite and interference with normal activities. The primary outcome was collected using the SRQ, along with a research nurse telephone call (we have achieved < 20% primary outcome attrition using this method in other similar trials). 1,27–29 The presence of adverse events, defined as new or worsening of existing symptoms, including otitis externa, rash, fungal ear infections, diarrhoea, vomiting and serious AOM complications, was recorded daily. We recorded daily measures of study medicine and analgesic/antipyretic use on the SRQ. On day 14, parents were invited to record their satisfaction with the trial treatments.

On days 7 and 14, parents were invited to record on the SRQ any use of health-care resources in the previous 7 days, including information about primary care contacts, community care and use of 111, walk-in centres and hospital services. The research nurse clarified details, such as reasons for the consultation, who was seen (e.g. GP, nurse, health-care assistant), the type of consultation (e.g. face to face, telephone, home visit) and whether it was in hours or out of hours. During the final telephone call (day 14), the research nurse reminded parents to send a stool sample [a specimen pot with research laboratory-recommended opaque polythene envelopes and label-compliant Mail Tuff™ (Antalis, London, UK) outer envelopes was sent to parents on day 7] and reminded them about the final questionnaire and stool sample at 3 months. The final questionnaire was intended to be provided in electronic format (for either web or iOS/Android applications) via the TRANSFoRm platform, but was completed on paper and sent by post because this part of the TRANSFoRm platform was not ready. In the questionnaire, parents recorded details of AOM and AOMd recurrence, audiology referrals, use of hospital services, hearing loss (using the OMQ-14)32 and any serious complications.

Symptom duration

Forty-one per cent of parents reported children’s symptoms as ‘normal/none or very slight problem’ by day 3, and 65% reported symptoms as ‘normal/none, very slight problem or slight problem’ by the same time point. The median duration until symptoms were rated as ‘normal/none, very slight problem or slight problem’ for children in all arms was 3 days (IQR 2–4 days; see Table 7); and the median number of days in the immediate oral, delayed oral and immediate topical antibiotics was 3 (3–4), 4 (2–6), and 3 (2–5), respectively. The duration of moderate or worse symptoms is reported by symptom in Table 6.

Satisfaction with treatment

Although numbers are too small for definitive comment, 88% of parents were either ‘extremely satisfied’ or ‘satisfied’ with treatment, and higher rates of satisfaction with treatment were observed in the immediate topical antibiotic arm than in both the immediate and the delayed oral antibiotic arms.

Use of oral analgesics

Use of paracetamol and ibuprofen was poorly reported, with only one participant responding to whether or not analgesia medication was used (yes/no) over the 14 days, with the remainder leaving the ‘yes/no’ response option blank.

Adverse events

There were three reports of new or worsening of existing symptoms within the first 7 days of follow-up: one (‘scratching ear’) in the immediate oral antibiotics arm and two (‘swollen painful eye with headaches’ and ‘eye discharge’) in the immediate topical antibiotics arm (see Table 7). There were three reports of new or worsening of existing symptoms in the second 7 days of follow-up: one (‘ear leaking again’) in the delayed oral antibiotics arm and two (‘sore throat with temperature’ and ‘eye discharge’) in the immediate topical antibiotic arms. The reports of ‘eye discharge’ from the first and second 7 days were from the same participant.

Serious adverse events

No serious adverse events were reported. One child attended the emergency department but did not require hospital admission.

Stool sample microbiological data

Only four (18%) of 22 parents sent stool samples. The research laboratory processed three samples within 48 hours of receipt, with one sample, received on 30 December 2019, taking 8 days to be processed.

The trial addressed an important clinical question

General practitioners felt that the trial addressed an important clinical question that was relevant to their practice and to their patients. GPs described being unsure about when antibiotics were needed and (in different practices) facing pressure from parents to both prescribe and not prescribe antibiotics. GPs felt that evidence about the effectiveness of the treatment options would support better treatment and communication with parents:

I liked the fact that it’s something that we see a lot and, like I say, there’s always this kind of unsureness whether we should be giving these kids antibiotics or not.

GP08

Parents don’t want to give their children antibiotics unless they absolutely have to, so they are quite willing to try some alternatives.

GP01

[W]e do see a lot of children with ear problems and there is always quite a lot of pressure to provide antibiotics from parents for them . . . having either data that says ‘You don’t need to do it’, [or] giving topical drops worked . . . would have the advantage . . . But it’s currently an off-licence use. So if we could get evidence that that was more supportive of its use, then that would be really helpful for ongoing care for patients.

GP09

Normal practice varied between GPs, reflecting a lack of clarity in relation to optimal treatment. Different GPs described normal practice as watch and wait, immediate oral antibiotics and, in some cases, delayed antibiotics:

I think I’ve probably been more likely not to give anything actually, just watch and waiting.

GP02

. . . so, normally, if they’ve got ear discharge present then we would be starting, erm, immediate oral antibiotics.

GP06

. . . if they were completely systemically well and they’ve got a runny ear then I think there might be a bit more of a negotiation about ‘well, is it getting better’, you know, ‘do you want to hang on for a day or two and see what happens, do you want a delayed prescription’.

GP07

Views of treatment options also varied between GPs, and not all were in equipoise, as not all necessarily viewed the different treatment arms as equally valid. Some GPs liked the ciprofloxacin drops because they are applied topically and, therefore, the GPs believed, less likely to cause systemic side effects. Some were familiar with the use of ciprofloxacin drops with adult patients. One GP raised concerns about the risk of toxicity and another about how quickly patients would be able to access the ear drops. Those GPs whose normal practice was to prescribe immediate antibiotics were concerned about delaying antibiotic treatment. Several GPs expressed a desire for clear evidence about the best treatment, to both guide practice and ‘validate’ their practice ‘to the powers that be’ (GP02):

. . . personally, I think [ciprofloxacin drops] are good . . . I have seen patients with recurrent [infections] who have had treatment with antibiotic drops, on occasions recommended by ENT [ear, nose and throat] . . . so I am sort of aware of it as a practice and I’m sort of comfortable with that . . .

GP06

[I]f there was data that said actually giving topical drops worked, then that would have the advantage that you don’t have the systemic side effects.

GP09

I was always a little bit hesitant to use [ciprofloxacin drops] to be honest, just because of the risk of toxicity . . . generally, if I thought there was perforation I’d stay away from drops, generally.

GP03

[S]ometimes it’s a problem actually getting them, so sometimes we’ve sent them to the chemist and they’ve commented that the chemist hasn’t had them in stock and, therefore, we’ve been rung up to say, you know, we want an alternative and I think then we’ve gone to oral antibiotics again.

GP02

I think it’s risky because . . . my fear is, is that what if a patient can’t then get in and if they can’t re-present or if their situation’s changed and . . . so I just feel uncomfortable with this delayed concept.

GP04

. . . it validates to the sort of powers that be that actually antibiotics, topically, are indicated so we are treating appropriately . . . and we wouldn’t get the pressure to, you know, from the kind of prescribing committee, we’d be able to validate the fact that, actually, we are better treating this with something rather than nothing.

GP02

The parents who participated in the trial were happy with the treatment options. The topical ear drops were seen as a good option by both parents who did want and parents who did not want antibiotic treatment:

I’m quite intrigued, I quite [like] the idea of the ear drop antibiotics for ear infections, I think they [. . .] might work.

P02

I was happy to do it if it helps, but also because he’s had so many antibiotics and [I] was happy for him to not have them . . . to be involved in something which would almost limit the use of them. Because I have been worried about the amount that he’s had to have for his ears over the years. So, yeah, the idea that there might be another option eventually . . . I think that would be really good.

P05

I thought it was good. I did want her to have some kind of treatment, I didn’t want just to leave it because I think it had been like 5 days or something at this point . . . I thought . . . there was a two out of three chance that she get a medicine. So I thought that was probably a good idea and yeah it was really easy to do and I was really pleased. And I got the drops . . . it was really simple.

P04

Clinician views of training materials

Clinicians described the training videos as detailed and comprehensive, if quite long. The length of the training videos may have meant that fewer clinicians were willing to complete the training to become recruiters, particularly as the expected number of recruits per practice, and, therefore, the recruitment-based payment to the practice, was quite small. Because of delays due to difficulties in getting the TRANSFoRm platform software to work, some clinicians reported a significant period between completing the training and actually being in a position to recruit:

. . . they were very detailed and comprehensive, I seem to remember at least one of them was quite long . . . but they were good in terms of guiding me through both the study and the sort of use of the app[lication], so what I was doing was watching little bits of it and doing little bits . . . and then pausing the video rather than watching it in one whole lot, so I watched it in piecemeal fashion as I was going through, sort of thing . . . so it did work quite well.

GP06

[T]here were quite a few . . . anything like that is going to put off some people who don’t have protected time for research . . . in a practice, you’re expecting maybe five or six doctors . . . to be recruiting, then that is going to limit because, you know, we just don’t have the time to do that . . . the problem as well is that it’s not a high recruiting study so if, for example, the practice was going to be recruiting 20 or more patients, then it would be worth watching those videos because you would get, you know, more money for doing that sort of number, but I think that you’re only expected to get quite a small number per practice so you might watch videos and do one or two patients and then that . . . you know forget really.

GP01

. . . certainly between the time that I think the training for the online software, so the integrated software rather, and then the, erm, the time the study actually went live, it seemed to be a sort of significant gap.

GP07

Frustrations of trying to get the TRANSFoRm platform to function

General practitioners and non-clinical staff involved in installing the TRANSFoRm platform and getting it to work described a long and frustrating process of troubleshooting and multiple reinstallations. Many were initially keen on the idea of the TRANSFoRm platform, as they hoped it would make the recruitment process quicker and easier. However, participants described encountering multiple problems and spending a considerable amount of time troubleshooting, seeking support from the trial team and reinstalling the programme:

I think as an idea it’s brilliant . . . it means you haven’t got piles and piles of paperwork and . . . paper forms that you’ve then got to somehow get scanned to e-mail through . . . it self-populates . . . it puts the details in, which is time saving. ‘Cause time is one of the big things in general practice.

GP09

I thought it was really interesting, I was really excited and then it’s made me a bit despondent really.

GP02

[T]he software didn’t work the first time for us, so I had quite a lot of communication with [the trial team] . . . they’d telephone and speak on the phone about working through different ways of doing it . . . to get that to actually work was quite hard.

Practice data manager, ITA01

[T]o be brutally honest [. . .] it was quite a nightmare . . . I’ve probably spent about 10 hours in total trying to install the piece of software on one computer. Um, quite often there would be loads of errors with it installing, with it not working. Um, I’d then have to send e-mails to the people that were dealing with it . . . it’s definitely taken so much longer than what we thought it was going to take.

Practice operations manager, ITA02

[I]t’s been very frustrating . . . communication [with the trial team] has been very good, but the problem has been with the REST TRANSFoRm platform . . . upset’s not the word, but it has taken an awful lot of time, to the point where the practices are saying ‘we don’t want to do this’.

Research nurse, ITA03

The software had to be installed individually on each recruiting clinician’s computer, and limited access to these computers led to delays. It was difficult to find time with a clinical computer, particularly for the early installations, which took several hours. If troubleshooting was also needed, then finding a time when the clinical computer and someone from the trial team were available led to delays in addressing problems:

[T]he major problem, um, of the installation is actually getting time to get into the GP’s room . . . we’re really limited on space so if that GP isn’t in, there’ll be a locum in their room . . . from 8.00 in the morning ‘til 6.30 or 7 o’clock at night. So actually trying to get in to have . . . 2 hours . . . is virtually impossible.

Research nurse, ITA03

[T]he software was only being installed on one particular PC [personal computer] at our practice . . . The time we had when his room was available and getting IT involved was always quite tricky. . . . we’d have to come back to the office, e-mail, wait for a time when we could speak to somebody . . . then you’re reliant on having to go back into a clinician’s room to go in and . . . use their PC and . . . they are seeing patients all day, every day so you’re trying to combine your REST study person with me . . . and then the clinician that’s in the room, to try and combine the three . . . how long these things take.

ITA04

Those who were involved early in the study and who were, therefore, installing an early version of the software felt that it was not sufficiently developed for use in practices. Participants were concerned about the time it was taking away from normal duties and the potential risk posed to practice computers:

The problem is it was rolled out far . . . too early with far too many problems and hadn’t been tested widely enough.

GP09

It basically felt like it was a prototype; it weren’t like the finished product.

Practice operations manager, ITA02

. . . it has got certainly the admin staff and the practice managers to the point of saying ‘do we want this on our practice computers? . . . is it going to cause more problems?’ . . . we have not got the time to be spending installing, take hours out installing this platform . . . it’s a very difficult situation and I’m getting everyone whingeing at me.

Research nurse, ITA03

In addition, during the trial, there was a widespread transition to Windows 10 in primary care practices and this caused some problems. Practices that had installed the early version of the TRANSFoRm platform software reported that it did not work properly after the transition to Windows 10. It also made troubleshooting more difficult as the person in the practice was using a different version of Windows to that used by the trial staff providing technical support:

. . . halfway through signing up for the REST study all our computers were then transferred over to Windows 10, so that caused a bit of an issue apparently . . . [trial staff] said that . . . [the trial had] programmed all the software for Windows 7 . . . I think the knowledge on Windows 10 wasn’t there because their systems are Windows 7 . . . so, therefore, it was very difficult for me to tell them what wasn’t working on our system because of the difference in the systems.

Practice data manager, ITA01

. . . when we went onto Windows 10, which was after the first recruitment, it’s not really settled down . . .

GP02

Some participants reported that later versions of the software worked well. The IT support manager in a practice that installed the final version of the software described the installation process as quick and easy. A GP who had not been involved in the installation process felt that the final version of the TRANSFoRm platform software worked well and had found it straightforward to recruit patients:

And now it nearly works. As in as long as you do this trick to get it to load properly, it works.

GP09

I installed [it on GP and nurse] PCs and it went absolutely fine, there was no problems with it and it just, yeah, installed . . . literally minutes . . . it was absolutely fine. You know it was just pressing next, next, next and finish and, yeah, it was no problem at all.

IT support manager, ITA07

I didn’t really do the installing. That was done by our research nurse and my practice manager . . . I do think the actual computer recruiting system, I think, works really, really well.

GP08

In most practices, however, participants described the final version of the software as unreliable and unpredictable. Participants reported that it worked for dummy patients but not real ones, or for some patients or staff but not others, or in some sites but not others. Participants felt that they did not understand why the software appeared to work in only some circumstances and this sense of unpredictability contributed to a lack of confidence in the software:

I think one would follow all the instructions and it seemed to say, yes, it was working, erm, but then it wasn’t when you went into it the next time and lots of shutting down; you kept having to shut down Windows in order to try it again.

Research nurse, ITA06

[W]e tried doing a dummy patient this morning again and it seemed to open up, but only randomly . . . even if we have it set up, it’s going to disappear again and my admin person comes down and sets it all up again and she can’t get it to work intermittently either.

GP02

[I]t just wasn’t working on [eligible patient] . . . but it was working on most [patients] that we tried. I have no idea why.

Research nurse, ITA06

[E]ventually . . . we were able to use it, but the functionality of what the REST study app[lication] was meant to be doing wasn’t happening . . . when [GP] was trying to, like, put in the code on SystmOne, the pop-up was meant to appear . . . and apparently that pop-up was never coming up, even though it was running in the background . . . On some days, it would be behind; other days it wouldn’t come up at all . . . no apparent reason.

Practice operations manager, ITA02

I still have this background lack of confidence that its going to if I needed to.

Research Nurse, ITA06

Effects of the study on primary care practices

The biggest impact of the study was the time taken to get the TRANSFoRm platform working in the already very time-pressured context of primary care. This had an impact on the other work taking place at the general practices, including preparing data for the Quality and Outcomes Framework (QOF) and, in one case, seeing patients. When the time commitment started to impede essential work, then practices started to consider withdrawing from the study:

I mean, all of the surgeries, all of the, all of the admin staff, practice managers, the secretaries . . . GPs, there’s not a spare minute in primary care at the moment . . . Research in primary care, I think we’re, I think we’re struggling a bit . . . There’s such a burden on, on GP time for major problems that research . . . the GPs would look at the studies and say, ‘yeah, I don’t have time to do this’.

Research nurse, ITA03

I’ve spent literally hours on this trying to install the software, hours. I think our clinicians have said that, you know, enough’s enough. They don’t want me to spend any more time on it . . . they [GP partners] just [got] cross ‘cause I wasn’t doing other things . . . I do all the QOF stuff so all the quality registers and things and all the statistics, all the claims . . . all that sort of was a bit on hold really.

Data manager, IT01

At one point, I started to refuse to install stuff because we were having so many problems with what it was doing to our computers . . . We wanted to wait until they’d got it more sorted because it was just eating so much time . . . it was probably towards the end of the QOF year last year . . . that I said I wasn’t prepared to put it back on until after we finished the QOF year because I couldn’t risk the machines not working.

GP09

When practices had staff with protected research time, participation in the project was more possible. In most of the practices, the recruiting clinician was a research lead with protected time for research. Some of the practices had other research support staff, including research nurses and, in a highly research-active practice cluster, a research co-ordinator:

. . . had I not been doing [a role with protected time for research] I would not have had the time to persevere and so the trial might not have been able to recruit . . . from my practice . . . it was . . . 4, maybe 5 hours that I spent . . . I do have protected time to do it so it does enable me to do things in a slightly different way, which I don’t think might be sort of rolled out for other practices . . . if you think they’ve just got to fit it in in their lunch breaks.

GP01

. . . one of them that tried a lot was our research lead . . . the other [clinicians] have basically really given up. I can’t get their computers to work now.

Data manager, IT01

Some participants reported financial costs to the practice as a result of participating in the study. At least one had paid their IT support person for extra hours to try to get the TRANSFoRm platform to work. Several participants felt that financial support to practices did not compensate for the time spent on research activities for this study:

I’ve worked extra time to do it as well. I’ve, they’ve actually paid me extra to come in and do the REST software, so I think that’s sort of annoyed them a little bit.

Data manager, IT01

[W]e’ve asked about, erm, resources for the amount of time that it spent us to install this software and provide the feedback that they’ve asked for and answered the questions they’ve asked for . . . the practice has actually lost quite a lot of money in terms of their time by trying to engage with this and that just puts people off . . . we do expect our costs to be covered. We do research ’cos we’re interested in research. If we were interested in making money, we would do pharma, but we don’t. But we are interested in not making a loss on it and not making a significant loss on it, which is what we’ve done with this.

GP09

Primary care information technology context: key challenges

Participants identified challenges with practice, CCG and NHS IT that contributed to the difficulties in getting the TRANSFoRm platform to function.

At the practice level, there was limited IT capacity and expertise. There were issues with outdated hardware and software and with the way in which individual GPs had adjusted SystmOne settings. There was varied IT expertise and capacity in individual practices, with many reliant on a GP, manager or administrator with only modest knowledge of IT. Computer administrator rights (which are needed to install software) were usually restricted to a small number of staff and not necessarily those with the time or responsibility for setting up research studies. This meant that, in many practices, the person tasked with undertaking the work to get the TRANSFoRm platform to function often struggled with the tasks and understanding the various problems encountered:

[The TRANSFoRm platform is] designed for really up-to-date computer systems and primary care is running on . . . old equipment and it just couldn’t cope with it . . . not only is it old, but it’s very old [and] not all on the same Windows.

Research nurse, ITA06

[T]he user guide was relatively comprehensive, but I would say not written for a user . . . that had basic IT skills. I think what they were asking for [was] somebody who was fairly IT literate to install the software, which I certainly wasn’t . . . We don’t have a particular IT person that can just go along and install . . . this sort of software.

Assistant practice manager, ITA04

[W]e are a fairly small practice . . . it was pretty much . . . me on my own . . . just trying to go through the installation step by step to work out where it wasn’t working and then trying to work out why, so trial and error.

GP06

[W]e’re not all IT proficient and that’s the problem . . . large practices, they do [have IT expertise] . . . but they don’t necessarily have the admin rights . . . these [small] practices don’t [have IT expertise]. So it’s the practice manager or . . . the secretary or someone who does it.

Research nurse, ITA03

. . . we didn’t locally have full admin rights. Well, the practice manager did, but, you know, to get her to sit down for a couple of hours and set it all up was very difficult; she didn’t have a couple of hours.

Research co-ordinator, ITA05

There were issues with obtaining help from outsourced CCG-funded IT support. All practices had some IT support provided by an external body, sometimes a private provider and sometimes a CCG or NHS provider. Five different providers were mentioned by our small number of participants, and IT support arrangements varied with respect to whether the external body held exclusive administrator rights for practice computers or supported research IT. Several practices reported that their external IT support provider would not assist because the software was not on the CCG’s approved list. When they were asked to provide support, these external bodies often raised concerns about the unknown TRANSFoRm platform software, were usually unfamiliar with software for research projects and were slow in providing support because of limited capacity. Only a research co-ordinator at a very research-active practice reported getting help easily, which she attributed to a good relationship with a particular person:

[W]e’re not sort of in charge of our own IT; the IT goes out to another company . . . our IT people who are [company name A], they’re not really supposed to give admin rights to anybody in a practice . . . [company name A] will not get involved with other people’s software . . . they have a list of software that’s allowed on the system and if we’re going to put some other software onto it they will not support us installing that software.

Practice data manager, ITA01

[W]e do have [company name C], but I didn’t get them involved in it . . . ‘cause . . . [company name C] wouldn’t help with it anyway . . . because it hadn’t been signed off by our CCG so we shouldn’t be installing it on our computers.

Operations manager, ITA02

[T]here’s been problems and we’ve had to go to our CCG IT team. And I’m just looking at an e-mail that’s come in this morning and their IT team are saying ‘what is this application? Is it a trusted . . .?’

Research nurse, ITA03

. . . [company name D] so they’re IT, they’re all NHS staff, but they’re a helpline, so you ring them, erm, with an IT queries . . . they were very dubious, actually, [about the TRANSFoRm platform] . . . I’m on, I think, my third or fourth call now with our IT about it. [It] wasn’t always easy to try and co-ordinate them ringing with then trying to install the software.

Assistant practice manager, ITA04

Some practices reported that they had to obtain permission from their CCG before installing software on their practice computers. The transition to Windows 10 during the trial was linked to the loss of practice-level administrator rights over computers in some CCG areas. Whether or not practices retained some administrator rights over their computers (and, therefore, the ability to install software) varied across recruited practices. When practices had to obtain permission from their CCG to install software, this could be a lengthy process. CCGs had questions about the risk that this unknown software could corrupt NHS software or practice computers and concerns about patient data crossing the NHS firewall. The centralisation of management to the CCG was seen as supporting initiatives such as the single domain, which allows better sharing of patient notes between different types of practitioners in primary care. However, it also had the unintended consequence of restricting the installation of study-specific software. Although some practices appeared to be able to install software freely, most reported having to defer to the CCG and wait for their approval. One practice reported that their CCG had required modification of the software to increase data security before permission was given:

[W]e had a big change at our practice, erm, something called single domain, which basically means that they’ve taken a lot of admin rights away from a lot of the users, including me . . . ‘cause I think it was becoming problematic across the practices that, you know, we had free rein really. And that’s going to cause a problem with things like REST because we can’t install it; so you give us a set of instructions and we won’t be able to do it because it has to go to our localised IT who has to verify they’re OK with it first.

Assistant practice manager, ITA04

[T]he CCG took it upon themselves to be responsible for all of our hardware and software, so when Windows 10 came for the whole of the CCG, they then took charge of everything really, which in a way makes sense because they paid for it and therefore they should control it and the flow of information that’s available and try to link it all up with other bits of the NHS, but as a result things . . . fell by the wayside, unfortunately.

GP05

. . . because of the way that the NHS is set-up we had to get firewalls opened, which wasn’t something we were told about in the beginning, to enable the software to contact [trial database] and then also for them to contact back through to our software; so, basically, you had to go through the firewall through a different port. And because we, we’re not sort of in charge of our own IT, the IT goes out to another company, so that was quite complicated at the beginning, having to go through these firewalls by logging it with our IT and then our IT doing it and that took a while. So that was the first sort of big issue we came across.

Data manager, ITA01

I think [the CCG] are quite – lax might be the wrong word – but we can install software and we do install software. So we’ve installed software for other research studies with no problems.

GP09

. . . we were all set to go and then we . . . needed to change our operating system . . . [to] Windows 10 so the original . . . [the TRANSFoRm platform] downloads would no longer work . . . to get them to rework . . . we couldn’t do it ourselves any more, we had to get the CCG computer boffins in to do it for us, they didn’t want to do it because they said its software may corrupt the NHS software and they wanted more assurance from higher levels than me that it was all safe to go, so . . . I got cross the told them it was all, it had been approved at high level . . . co-ordinated at committee level and approved and was being used elsewhere and they shouldn’t be so silly . . . so they then did come and put it on for me, so it’s now up and running.

GP05 – with role in CCG

[I] installed the software once I had permission from the CCG and that took [from] July/August . . . until December . . . the CCG felt they hadn’t got enough information from the installation guys . . . to be satisfied about the security aspect of the data that’s being transferred . . . the [study] IT engineers [tested] an encryption patch . . . [and] in December 2019, it was approved for installation . . . it’s just checking the security side of things, just make sure we’re not going to get any viruses . . . it’s about data protection; you know, they want to make sure that no patient identifiable data is going to be sent over for the studies.

IT support manager, IT07

Barriers to trial recruitment

Trial recruitment was reduced or slowed by the problems with the TRANSFoRm platform software. Several practices discussed identifying eligible patients but not being able to recruit them to the study because the software did not work properly. In some practices, there was a long delay while staff struggled to get the TRANSFoRm platform to work before clinicians even started trying to recruit patients. Some practices decided to withdraw from the study before any patients had been recruited because of the time taken to try to make the software work:

. . . last month there’s been four we’ve missed and all because the software just does not open up . . . I had someone in front of me on Thursday whose mom was really interested in doing it. I couldn’t even do a decline, and we just couldn’t get it to work and I tried, honestly, I must have spent about 20 minutes, I think, trying to sort it out and I had to move on really.

GP02

We’ve missed recruiting patients to the study because we couldn’t get the software working. We’ve had them in the consulting room agreeing to do the study, but we just couldn’t get the software to work so, therefore . . . we’ve had to abandon it. So there was a possibility of at least seven or eight that we couldn’t get because I’ve gone in and sat with the clinicians as well to try and get the software to work . . . it’s not that we’re not wanting to; it’s not that we’re not trying. It’s the software that’s not working . . .

Data manager, ITA01

I think that’s probably why there’s no recruitment, because the actual software itself wasn’t doing what they needed to do and then if they’re in an appointment they obviously don’t have the time to figure it out so they just move on to the next patient. . . . the doctors . . . they don’t have the knowledge to quickly figure things like that out . . . especially if it’s software that they don’t usually use.

Operations manager, ITA02

[W]e’ve taken a massively long time to recruit four patients . . . we should have done eight in a third of the time – probably would have done if the software worked.

Research co-ordinator, ITA05

Eligible patients were sometimes missed because they were not seen by a clinician who was able to recruit to REST. The processes for channelling eligible patients towards a recruiting clinician differed across practices. This was partly due to differing processes for dealing with same-day appointment requests for acute illness. Some practices had minor illness nurses who would usually see all children with suspected ear infections. Some practices triaged all patients requesting same-day appointments, and those patients might be seen by one or two duty clinicians or any clinician with a free slot. Study recruitment problems arose when these normal processes channelled a potentially eligible patient towards a clinician who was not able to recruit to the study. Administration staff and triaging clinicians responsible for booking patients into appointment slots did not always remember to book potentially eligible patients with the few recruiting clinicians. Where an eligible patient was seen by a non-recruiting clinician, there were various ad hoc arrangements to redirect patients to recruiting clinicians, but these were probably impracticable, and there were no accounts of this happening in practice:

[W]e’ve got minor illness nurses and the minor illness nurses don’t really get involved in research as much . . . the problem is that if someone rings up with a cough or cold, all that kind of stuff, they are being put into the minor illness slot . . . we tried to actually put REST study slots into my clinic so that if anybody rings up with an earache . . . put them in here, but . . . I’ve had those slots up and running and I think it’s not even been used once . . . but we know that these guys are coming through because they have gone and seen the nursing team, yeah?

GP03

. . . reception are made aware that if they have a child with a runny ear to try and book them in with myself or the other GP in question, where possible; where not though, and this has happened . . . a couple of times, that someone else [sees an eligible patient], they send me an instant screen message to say . . . ‘do you want to see them’ . . . and then depending on . . . how my appointments are looking, I might say ‘yes that’s fine’, or I might say . . . ‘could you see patient Y for me instead?’ and then we sort of switch patients, if you see what I mean . . .

GP06

. . . whoever’s doing surgeries will have some appointments added on at the end for same day so we don’t do triage or anything else . . . the nurses will do the same . . . they might see . . . children with coughs and colds . . . the receptionists have an aide memoire to say any child with . . . earache of the right age and they’ve had any discharge at any time that they should be booked in with me.

GP05

We can’t do too much opportunistically, I mean people coming to urgent care, they struggle to get an appointment in the first place . . . our advanced practitioner might not be on the delegation log, but one of my GPs is, but they’re seeing somebody else.

ITA05

Some eligible patients were missed because recruiting clinicians were not available when the patient needed to be seen. Some recruiting clinicians worked part time and were able to protect certain time slots only, which did not necessarily coincide with when patients needed to be seen. A participant from a large, recently merged practice described the challenge of keeping triaging clinicians primed to recruit to the study when they faced long lists and multiple study reminders, and were not even sure of being able to book a patient in with a recruiting clinician:

I think there may have been a couple of kids with discharge who might have seen other GPs. You know, ‘cause I only work part time.

GP08

. . . nothing came in in the morning, and then I had a full-booked surgery in the afternoon . . . somebody came in in the afternoon . . . they came in at 4.30 which is . . . right in the middle of my surgery that was running late anyway . . . nothing’s come in this morning; I had protected time this morning . . .

GP01

. . . now we’ve merged, we’ve got 25 GPs, six partners, but . . . lots are part time . . . so trying to keep it at the forefront of their mind, those images on the computer screen that they have got them every morning, in a way you just get used to them being there, you don’t actually look at them . . . they come and face an absolutely hideous list, phone calls every morning . . . there were 2 days when there wouldn’t have been a slot, if they had remembered, they couldn’t have found a slot, which [. . .] probably make them think ‘oh, well I actually remembered and then I couldn’t book them in’.

ITA06

The number of clinicians in each practice who could recruit to the study was limited, and this contributed to the number of missed recruits. There seemed to be several reasons for the limited number of clinician recruiters per practice. In many practices, only the research lead and perhaps one other clinician completed the training to become a recruiter to minimise the training burden on practice staff. It is likely that the time it took to install the TRANSFoRm platform also contributed to practices installing it on a limited number of computers only:

I could recruit; none of the others did the training on the basis that it was most likely that the [trained] nurses would do it . . . when you’re not recruiting massive numbers, if you have too many people able to recruit, nobody does it very often, so it just takes everybody ages.

GP09

What we can’t do is install it on everybody’s system . . . we’ve got about 20 studies we might be doing at one time so you have to pick and choose your staff of who’s going to be involved.

ITA05

[W]e have it installed on mine and the other GP . . . who has the research experience . . . and the administrative machine; it’s not running on the other clinician’s machine . . . so not everyone’s recruiting, just the sort of people who know about the study.

GP06

The nature of the target patients also presented some barriers to recruitment. Acute infections are seasonal and by the time the TRANSFoRm platform software was working in most practices, the winter season, when these patients are more common, was over. Cases that fitted the REST inclusion criteria were relatively rare. One research nurse described studies evaluating a new medication for acutely unwell children who attend with a parent (and often other siblings) as one of the hardest studies to recruit to:

[I]n the summer, we don’t see that many; in the winter, probably one to two a week . . . as a surgery . . . I’m trying to think, probably about two a week, something like that.

GP02

. . . it started at a time in summer where there wasn’t very much happening [. . .] and I think that’s taken a lot of momentum away . . .

GP03

. . . anything that involves a child that’s unwell, the parents have got other children with them, you know, younger children . . . children’s studies are harder straight off ’cause the parent and the child are involved rather than just an adult and if it’s a child who’s already a bit anxious then that makes it harder; so I think the children’s ones, where its opportunistic recruitment are probably the harder ones we do . . . and if it involves medication, its more complicated straight off. I suppose it is one of the hardest scale of ones that we do in general practice. I’m afraid our current issue is time.

ITA06

Experiences of trial recruitment

When clinicians had successfully recruited, this seemed to be associated with parent interest in the study and consultations with ‘relaxed’ mothers of relatively well children who had time for recruitment. Clinicians reported positive responses to the novel treatment (ciprofloxacin ear drops) and to the study processes:

. . . well, parents were happy that something was being looked at . . . those parents that we’ve seen . . . it’s not . . . the first time they’ve been in; they’re coming in regularly with the same child saying ear pain and discharge . . . And some of them are very reluctant to have antibiotics, but all of them want some help. You know, they all seemed to like the idea of trying topical drops.

Research co-ordinator, ITA05

. . . both of [the recruited children] were . . . it made it easier . . . both children were actually incredibly well, but there was loads of gunk coming out of their ears, erm, and they both had very laid-back mums . . . I said to both of them, you know, ‘there’s this trial going on looking at the different treatments, obviously I will examine your child and if I disagree with what you’re randomised to, you know, we can decide not to go along with it’ . . . I think [the first mum] really liked the follow-up that she had with the trial team . . . all the leaflets with the advice and things like that . . . I think it was helped that they were relaxed mums and they obviously weren’t in a massive hurry and – yeah. It made it easier.

GP08

. . . mum was pretty excited to be part of it so I think that was really quite nice . . . She was randomised to cipro . . . the ear drops . . . She was fine with it, I think the patient information leaflet was very good, you know, so that was excellent.

GP03

Recruited parents described study processes as straightforward. They had a clear understanding of the study purpose and found it easy to provide follow-up data:

I think everything was really well explained and the actual medicine that she got was really easy to use; the questionnaire was really simple; it was really easy to send stuff back. No, it was absolutely fine.

P04

. . . the nurse did explain to me as much as she could but [child] was quite . . . unsettled . . . it was a bit, erm, difficult to kind of grab all the information that the nurse had to give me, but the pack was quite comprehensive . . . it did have quite a lot of information and leaflets about what the study was about so I had the chance to read a bit about it when I got home so, and I had a call from one of the ladies involved in the study as well pretty much the next day . . . which was good because then you kind of get their reassurance of the study itself.

P07

I would have been happy with . . . any of [the treatment options], because they said if you’d have had nothing then if he started to get a . . . higher temperature or got really unwell then they would . . . look at him again . . . if he became really unwell. So they said that, as it was at the moment, he was happy for him to either have antibiotics or to not have it . . . They said at any time I could sort of take him back in and have him checked.

P05

Reasons for parents declining

Of the two parent decliners who participated in an interview, one declined because she wanted her child to have antibiotic ear drop treatment and one because she did not want her child to have antibiotics. In the former case, the child had a history of ear problems and, as a result of this experience, the parent perceived ear drop antibiotics as more effective than oral antibiotics for her son’s ear infections. In the latter case, the parent was aware of the drive to reduce unnecessary antibiotics, was told by the recruiting clinician that he would not normally give antibiotics for her child’s symptoms and she did not want her child to take something that was not needed. Recruiting clinicians described parents declining to participate because they did not want to have to:

[H]e’s had some ongoing problems with his ears . . . about a year ago, he had like a runny ear for quite a long time and he went on an oral antibiotic and it didn’t clear it up. And it carried on for probably about 2 months where . . . when I went back to the doctors and they found in his records, I think from the ENT, that he should have the drops in his ear. And when he did have the drops, it cleared it up within about a week, so when they asked me to do the, you know, be part of the study they said it was a randomised, where they just leave it or have the drops or have the antibiotics. And because I knew that that had been really effective with him last time and I didn’t want it to drag on for a few months like it did last time, I just asked if they would give the drops . . . I think if it had been my daughter, who’s not had any ear problems, you know, or recommendations about what she needed, I definitely would have done the whole study.

PD01

. . . the reason that I didn’t want to participate in the study because if there was no study she wouldn’t have got any medication for it at all. So it did seem a bit odd that . . . if she participated in the study then she’d be given antibiotics. I thought the idea was that you only get antibiotics when you really, really need them . . . The doctor had said that, at this moment, she didn’t need anything . . . I didn’t want her taking something.

PD02

. . . we did start to use the software and go through some of it, sort of, and then I showed them the leaflets about it and mom just started to look more and more doubtful about the whole thing as we went on, so she showed some initial interest and sort of said ‘yeah, OK, maybe I’d be interested in that’ and then as we talked about it and what she might have to do in terms of having a phone call, doing a sort of diary and things like that, she just looked more and more doubtful about it, to the point where you thought ‘no she really doesn’t want to do this’ and I sort of said ‘look, . . . it is up to you, you don’t have to’ and she said ‘no, I think probably not’, so it was a more slow sort of build to her saying ‘no I’m not interested in it’.

GP07

Recommendations

1. Electronic trial platforms should be used to harness the unprecedented opportunities to monitor, measure and test recruitment assumptions, identifying where in the recruitment process, from presentation to consent, the key ‘drop-offs’ occur.

2. All necessary platform preparatory activities and required resources should be clearly defined, taking care not to underestimate.

3. The skills needed to set up a trial platform and to set up a trial are distinct and complementary. Ideally, teams should be co-located to ensure that platform specifications meet individual trial requirements.

4. Platform software needs to be compatible with all practice software systems.

5. Closer integration with EHR providers would prevent incompatible updates (note that this could be obviated if national criteria were agreed or if the trial platform was integral to the EHR).

Recommendation

1. Project teams need to work closely with EHR providers and CCGs from the study outset to agree on the software deployment process and the validation criteria required (note that this could be obviated if national criteria were agreed).

In addition to CCG approval, installation agreement was part of the site agreement, meaning that national- and CCG-level approvals were required before practices could sign the recruitment contract, which included identifying the site personnel involved, machines and training.

Recommendations

1. A pilot installation incapable of being used for recruitment (and, therefore, not a site agreement requirement) should be performed on one computer in each practice, tested, and left to run for a week, before the software is installed on other machines.

2. Where software reinstallation is required, it must be undertaken in a way that does not disrupt the work of the practice.

Recommendations

1. Electronic study platforms require teams dedicated to (1) development and (2) troubleshooting.

2. Careful consideration should be given to who is responsible for troubleshooting. Although it may seem obvious that this would be performed by the trial team (as it involves interacting with sites), it requires awareness of platform function and, therefore, may be better provided by the platform development team.

Recommendation

1. Electronic trial platforms would be best served by a dashboard function to monitor and log platform functionality in real-time, providing real-time alerts and diagnostics for reduced function, and logging functionality across time and space.

Recommendation

1. The format of the final data set to be extracted from the study database should be prespecified to ensure appropriate data format and avoid the submission of linked clinical and personal data.

The National Institute for Health Research Clinical Research Network

1. CRNs could keep logs of which sites have been invited, when and how many times. These could be shared with study teams to populate CONSORT flow diagrams and allow a description of the generalisability of the recruiting sites.

Sponsors

1. Sponsors could consider accepting electronic versions of delegation logs with electronic signatures. These could be designed so that submission of incomplete logs/curricula vitae is not possible.

2. For large distributed trials with many sites, a robust electronic data management system to track documentation could be employed.

Trial management teams

1. When online site training is used, studies could provide training using a website that provides automated reminders and notifies the sponsor and study team when training is complete.

Electronic study platform

National stakeholders, including the Department of Health and Social Care and the National Institute for Health Research

1. Research funders need to formally recognise the potential of electronic study platforms if they wish to put the NHS on the leading edge of pragmatic research globally, allowing the delivery of new, near-real-time generalisable knowledge. This could also provide unprecedented opportunities to monitor, measure and test recruitment assumptions, identifying where in the recruitment process, from presentation to consent, the key ‘drop-offs’ that influence final study sample representativeness occur.

2. NIHR and research funders could consider convening a meeting of national stakeholders to define a strategy for the development, implementation and ongoing management of electronic study platform software.