The Community IntraVenous Antibiotic Study (CIVAS): a mixed-methods evaluation of patient preferences for and cost-effectiveness of different service models for delivering outpatient parenteral antimicrobial therapy

Outpatient parenteral antimicrobial therapy

The delivery of intravenous (i.v.) antibiotics to patients outside a hospital setting was first described on a small scale in the 1970s in North America. By the end of the 1990s, an estimated quarter of a million patients annually were receiving i.v. antibiotics on an outpatient basis as a result of cost savings, patient preference, better i.v. devices, the introduction of antimicrobial agents that needed administration only once or twice a day and the development of dedicated service providers. A wide variety of infections have been treated through this system, in particular skin and soft tissue infections (SSTIs), but also bone and joint infections, bacteraemia, wound infections, pneumonia, complicated urinary tract infection, intra-abdominal infections, device-related infections, endocarditis and central nervous system infections. Although widely accepted as the standard of care in countries such as the USA and Australia, such services are largely limited to patients with appropriate health insurance cover. 1 Clinical efficacy and safety has been addressed in many different clinical areas and using a variety of models of care, largely using retrospective analysis of single-centre experience. 2–5 Some risks of community-based i.v. therapy have been identified, and projects have been initiated to minimise these. 6 The potential for treatment of micro-organisms resistant to antimicrobials and for limiting the spread of health-care-associated infection has been highlighted as an outpatient parenteral antimicrobial therapy (OPAT) benefit. 7 Newer antimicrobial agents have been assessed for their potential for OPAT use, usually because they have long half-lives with less frequent dosing required and may be effective against resistant micro-organisms such as meticillin-resistant Staphylococcus aureus (MRSA). 8,9 However, such agents are much more expensive, so increase the cost of the service.

Clinical practice in the UK

Over the last 10 years, OPAT services have been developed in more areas of the UK, in both the NHS and private sectors, in response to local pressures in combination with health-care staff initiatives. 7 This has led to many service variations using different heath-care professional groups which can be grouped into four main categories:

1. Outpatient attendance at a health-care facility

   1. A variety of NHS hospital departments have set up systems for providing i.v. antibiotics for patients attending on a daily basis, including both specialist and general services. The main disadvantage to this system is the inconvenience to the patient in having to travel, the fact that it is limited to patients who are fit to travel and the cost of transport.

2. Self-administration (SA) of i.v. antibiotics

   1. Particularly where patients require very long or repeated courses of antibiotics, patients or carers have been taught to self-administer the treatment. This system is likely to be cheaper insofar as less professional time is required once the patient has been trained, but there are potential risks to unsupervised administration, including non-compliance.

3. Visiting general nurse (GN) model

   1. There are instances of NHS community nurses (e.g. district nurses) administering i.v. antibiotics; this can be efficient as they can perform other tasks such as wound management at the same visit and with minimal travelling as they are based locally. However, they are likely to be less confident and skilled in i.v. antibiotic management, as this makes up a small percentage of their work and they may have insufficient time to add this to their caseload.

4. Visiting specialist nurse (SN) model

   1. In contrast, specialised visiting nurses have more expertise but may be less efficient as they cover a large geographical area. This is the main model of care in the USA and is generally provided by private specialised companies. In the UK, this model is available through a few providers, both private sector and NHS.

Existing evidence on outpatient parenteral antimicrobial therapy service delivery

There have been evaluations of the staff required to provide OPAT services. However, the conclusions of such studies vary, with the benefits of a nurse-led service10 and the need for infection specialists11 both being suggested. There is a striking lack of prospective studies and only one randomised controlled trial (RCT), which was conducted in New Zealand. 12

Although health economics have been addressed in depth overseas (especially in the USA), there is little detailed analysis in the UK. Most economic evidence comes from studies reporting bed-days saved and simple analyses of cost savings, which are reported to be significant. A comprehensive pharmacoeconomic evaluation of OPAT services has yet to be completed, despite the number of published studies. 1 Chapman et al. 13 did complete a cost-effectiveness analysis of OPAT in a UK setting but this included only one centre and was predominantly a comparison of standard hospital inpatient care with daily attendance at a hospital facility. In addition, owing to a lack of appropriate data, the analysis completed was a cost–consequence analysis rather than a cost–utility analysis and, thus, did not adhere to National Institute for Health and Care Excellence (NICE) guidance. 14

Little has been published on patients’ preferences for different services, although reports of patient satisfaction with services have been cited. Only one study was found to evaluate patient preferences directly in this group, finding that 90% of patients preferred treatment at home to treatment in hospital. 15 However, this study was conducted in Canada, had a small sample (n = 71), compared only two fixed-service models (in hospital vs. SA at home with weekly hospital visits) and used willingness-to-pay to measure preferences.

National policy and initiatives

Outpatient parenteral antimicrobial therapy services have the potential to generate significant cost savings for the NHS and to deliver greater patient satisfaction. They may contribute to the delivery of key health-care strategies and directives such as Equity and Excellence: Liberating the NHS,16 Creating a Patient-led NHS,17 Your Health, Your Care, Your Say18 and Start Smart then Focus. 19 As yet, no national policy in this area exists, although we understand that there have been meetings between members of the British Society for Antimicrobial Chemotherapy (BSAC) OPAT steering group and the Department of Health regarding further service development. Health-care providers mainly use the current system of tariffs provided through NHS England Payment by Results system, which can be interpreted in a number of different ways, thus making the commissioning process complex. For more unusual conditions, the provision of OPAT is part of the recommendations of the NHS England Specialised Services Specifications for Infectious Diseases and Bone and Joint Infection Services.

Following a conference on OPAT in 2009 hosted by BSAC, a UK database has been set up in which centres have the option of sharing their data on, for example, service type, patient numbers and outcomes. A voluntary survey of existing services provided by OPAT group members was carried out at the end of 2011 and presented in summary, focusing largely on clinical issues such as types of infection treated. The BSAC-sponsored OPAT project is supporting the development of such services throughout the UK without favouring any particular model of service design. Various resources have been provided to facilitate this, including the development of practice standards, a preceptorship scheme, regional training days, a model business case including a strengths, weaknesses, opportunities, threats (SWOT) analysis of service models, and software to support a virtual ward round (http://e-opat.com/).

Identification of studies

The world literature from 1993 to March 2015 was reviewed to identify existing research related to the safety, efficacy and cost-effectiveness of community i.v. antibiotic delivery services. Separate searches were run to identify (1) studies of i.v. antibiotics and known models of care and (2) reviews of antibiotic use in cellulitis or cystic fibrosis (to allow for the identification of models of care that were unknown to us and subsequently not considered when identifying terms for search 1). We searched MEDLINE via Ovid (1946 to March week 4, 2015), MEDLINE In-Process & Other Non-Indexed Citations (25 March 2015), EMBASE via Ovid (1947 to 25 March 2015), Cumulative Index to Nursing and Allied Health Literature (CINAHL) via EBSCOhost (1981 to March 2015), International Pharmaceutical Abstracts via Ovid (1970 to March 2015), Cochrane Central Register of Controlled Trials, Wiley, Issue 2 of 12 (February 2015), Cochrane Database of Systematic Reviews, Wiley, Issue 3 of 12 (March 2015), Database of Abstracts of Reviews of Effects, Wiley, Issue 1 of 4 (January 2015), NHS Economic Evaluation Database 2015, Issue 1 of 4 (January), Health Technology Assessment Database, Wiley, Issue 1 of 4 (January 2015), Research Papers in Economics (accessed March 2015), Cost-Effectiveness Analysis (CEA) Registry, Tufts (accessed March 2015), and Health Business Elite, Healthcare Database Search (HDAS) NHS Evidence (1922 to 25 March 2015).

Supplementary searches of Web of Science Conference Proceedings Citation Index – Science, Thomson Reuters (1990 to March 2015), the Health Information Management Consortium via Ovid (1983 to 25 March 2015) and the website of the BSAC (accessed March 2015) were conducted to provide relevant unpublished work. In addition, the reference lists of included studies were reviewed for potentially relevant papers. A sample search strategy and databases searched is detailed in Appendix 1.

Selection of studies

Studies were included if the participants were individuals or groups of adult patients or care providers, and (1) they evaluated the clinical effectiveness or cost-effectiveness of an OPAT model, (2) they described or evaluated patient safety issues associated with OPAT or (3) they considered the acceptability of OPAT from the perspective of the patient receiving treatment or the practitioner delivering care. Any form of i.v. antibiotic drug delivery system (e.g. infusion or bolus) was included. No restrictions on language or study design were applied.

Studies were excluded if they considered the costs related to a model of delivery but did not consider patient benefit alongside these, or if they made reference to costs and benefits but did not report specific cost-effectiveness analysis data [e.g. cost per quality-adjusted life-year (QALY)]. Similarly, studies that made reference to clinical effectiveness without reporting specific patient outcomes were also excluded. Studies that included children or that involved multiple routes of antibiotic delivery were reviewed but excluded if they did not differentiate between outcomes for adult patients or for patients receiving i.v. treatment, and those of other participants. Studies that focused only on the method or process of delivery or on the clinical effectiveness of a single treatment or of one class of antibiotic over another were excluded, as were abstracts only, descriptive or commentary pieces and guidance documents.

Titles and abstracts of all identified studies were screened for eligibility, and full-text versions of papers not excluded at this stage were obtained for detailed review. All abstracts were reviewed by one researcher (EDM) with a random selection (20%) independently screened by a second reviewer. Potentially relevant studies were then independently assessed by two reviewers (EDM with JE, DM, CCM or MT) to determine if they met the inclusion criteria. Differences of opinion were discussed until a consensus was reached.

Data extraction

Data extraction was carried out by one reviewer (EDM) using a standardised pro forma. Data for a sample of studies were extracted independently by a second reviewer in order to validate the items being collected. Extracted data included citation details, study purpose, design, location, duration, population details and clinical characteristics (e.g. reason for antibiotic treatment), models of care [hospital outpatient (HO), SA, GN, SN], topic area (clinical effectiveness, cost-effectiveness, safety, acceptability), type of antibiotic, route of delivery, treatment dose, outcome measures, follow-up and key findings. Assessment of bias was carried out as part of this process.

Assessment of bias

Quality assessment was carried out by one reviewer (EDM). Where possible, studies were assessed using previously developed scoring systems. The Cochrane risk-of-bias assessment tool was used for experimental studies (RCTs, clinical trials, controlled before-and-after studies) and the Newcastle–Ottawa scale was used for cohort and case–control studies.

A method of assessing the strength of evidence of observational studies – developed as part of a previous review on the early diagnosis of cancer20 – was modified for this topic area and applied to relevant studies. The main modification to the assessment system was to account for the fact that, although a study in this area might not use a power calculation and might include a relatively small sample size, this is actually the entire population receiving OPAT treatment. As such, it should not be considered automatically to provide weaker evidence. In this system, papers were evaluated on the basis of ‘population’, ‘ascertainment’ and ‘analysis’ (see Appendix 1). Population relates to the method of determining required levels of participation, with use of a sample size calculation or inclusion of all possible patients/providers rated more highly than selective recruitment. Ascertainment relates to methods of obtaining study data, with use of a rigorous method designed to reduce systematic differences between groups (selection, characteristics, treatment, etc.) rated more highly than other methods. Finally, analysis relates to the use of analytic techniques, with reporting of statistically significant differences (or use of appropriate analytic techniques if qualitative) rated more highly than non-statistical comparisons or descriptive data.

Studies at risk of bias were not excluded from the review, but an appraisal of the strength of existing evidence has been reported and findings interpreted in light of this. Many of the papers included in this review used methodologies that did not lend themselves to the scoring systems outlined above. Many studies included all patients in receipt of OPAT since its establishment at a particular institution, or all patients seen over a specified time period, and simply reported conditions treated and therapies used, along with limited outcomes data. Case series such as these, which were to all intents and purposes audits of service provision that included little or no analytic content, were not subject to quality assessment.

Data synthesis

The main characteristics of included studies and findings relating to clinical effectiveness, cost-effectiveness, patient safety and acceptability and study quality have been summarised in narrative and tabular form. The substantial clinical and methodological heterogeneity precluded pooling of data for meta-analysis. The majority of studies included a varied case-mix and did not differentiate their results between conditions treated. In addition, there was variation across studies in relation to what constituted a successful outcome (cure, improvement, deterioration, etc.) as well as a lack of consistent treatment duration. This meant that it was not possible to pool the results of individual studies to provide estimates of true effect size when using OPAT for different patient groups, or even for comparing OPAT as a whole with inpatient treatment. Substantial clinical [condition treated, duration of treatment and definition of a successful outcome (cure, improvement, deterioration, etc.)] and methodological heterogeneity precluded pooling of data for meta-analysis.

Overview

The search strategy identified 7214 articles, of which 589 (8.2%) met the inclusion criteria for detailed review (Figure 1). We retrieved the full text of an additional 17 papers identified from the reference lists of previous reviews and included studies, giving a total of 606 potentially relevant papers. In a change to the initial protocol, non-English language papers were not assessed for inclusion owing to the overall volume of literature identified and timescale of the review (n = 69; see Appendix 1), and we were unable to obtain one article from the library document service. A total of 128 papers were included in the final analysis. Reasons for exclusion are shown in Table 1.

FIGURE 1.

Flow of studies into the review.

Populations

Three-quarters of studies were carried out in Europe (n = 53; 41%) and North America (n = 45; 35%). Almost two-thirds of the European studies were conducted in the UK. Two studies involved centres in multiple countries,21,22 whereas another was a systematic review of the world literature (Table 2). 23 Most were relatively small in size (mean 476, median 100, range 6–11,427 participants/episodes of care). It was not possible to determine participant numbers in one study24 and in another numbers could be identified only for three of four included groups. 25 The period under study ranged from 6 weeks to 15 years. In general, studies with the largest numbers of subjects (> 1000) either analysed all cases included in an OPAT registry and/or reviewed cases over a more substantial time period. 21,25–34 Most papers were published in the past 10 years (59%).

The most commonly reported reason for treatment was osteomyelitis, followed by endocarditis, SSTI, cellulitis and septic arthritis (Table 3). The majority of studies involved multiple conditions (n = 72, 56%), although six did not specify indications for treatment. These included two qualitative studies and one survey of patient acceptability,35–37 two surveys of practitioner acceptability,38,39 and one secondary analysis of data from an OPAT database. 40

The total number reported is > 128 studies, as many involved multiple reasons for i.v. therapy.

Outcomes and outpatient parenteral antimicrobial therapy models studied

Of the five areas evaluated in this review, the most commonly considered were patient safety (n = 109; 85%) and clinical effectiveness (n = 89; 70%). Just over one-quarter of studies involved some aspect of patient acceptability (n = 37; 29%), but few determined cost-effectiveness (n = 5) or practitioner acceptability (n = 6). Most reported on multiple areas (70%).

Twenty-two studies (17%) either did not indicate the type of OPAT delivery model used or reported that home treatment was used without providing any additional detail. In the remainder of studies, the most frequently reported method across studies was self- (or carer-) administration (n = 66; 52%), followed by visits from a SN (n = 44; 34%), outpatient attendance (n = 35; 27%) and visits from a GN (n = 14; 11%). Just over half of these studies evaluated a single model of OPAT delivery (n = 59; 55%). Other, less common, delivery methods and locations included infusion centres, home infusion or home care companies, Hospital in the Home Units (HHUs), prison and doctor visits (see Appendix 1, Tables 33–37).

Quality assessment

Few studies employed a controlled trial methodology, with two-thirds being case series or observational in nature (Table 4). Many involved retrospective data collection (n = 57; 45%). Of the 12 included RCTs, five reported on subgroup analyses from the main study, and all failed to provide details of the original trial methodology.

Many studies involved a review of medical records and/or the analysis of data from prospectively held OPAT databases (40%); a small number of other studies carried out secondary analysis of national or international OPAT registries. Satisfaction surveys, by either questionnaire or telephone completion, were frequently used, and interviews (face to face, telephone, focus group) and visual analogue scales, although used less frequently, were also well represented. Less common methods of data collection included clinic and ward diaries, direct observation and data from previous studies or from the published literature.

It was often unclear who collected data, and few studies reported on how this was done (e.g. using a standardised pro forma). A significant number provided little or no detail on the methods employed.

Risk of bias within studies

Three of the 14 included trials were assessed as having a low risk of bias,12,41,42 and one was assessed as having a high risk of bias (a pilot in which patients self-selected hospital or home treatment and were recruited consecutively to each arm after this decision was made). 43 In the remaining nine studies, the level of potential bias was unclear. In five cases, it was uncertain whether or not randomisation or controlling had taken place. Four of these studies reported on subgroup analyses from a single open-label trial, and none provided details of the original study methodology. 44–47 The fifth reported on two related trials comparing i.v. with oral treatment for neutropenia in cancer patients, but reported no methodological details of the parent studies. 22,48–51 In the remaining five trials, details of randomisation, allocation concealment and blinding (especially in relation to assessment of outcome measures) were poorly reported. 52

The five case–control studies (three of which were retrospective) rated low for potential bias (median 8/9; range 6–9). Those studies scoring lowest did not provide details on the methods used to determine outcomes. Similarly, four cohort studies (two of which were retrospective) all had low potential for bias (median 8/9; range 7–9).

In the majority of observational studies, the data analysed were derived from reviews of OPAT databases or medical records. Many studies also involved questionnaire surveys. Only five studies included a comparator (inpatient care), with many simply including all OPAT patients over a selected time period. Most studies reported descriptive results only, with no statistical testing of differences.

Twenty of the 128 studies (16%) received full or partial funding from pharmaceutical companies.

Impact on clinical effectiveness

Only 21 of the 89 studies evaluating the clinical effectiveness of OPAT included a comparator, which, with few exceptions, was treatment given on an inpatient basis (see Appendix 1, Tables 32 and 33). Five of the studies did not specify the OPAT model that was being used,29,48,52–54 whereas six others reported combined results for multiple OPAT models. 48,52–55

Synthesis of the findings from these studies indicates that, regardless of the OPAT model used, there is little impact on the duration of i.v. antibiotic treatment in comparison with inpatient treatment12,22,25,41–43,56–58 (see Appendix 1, Table 32). The effect of OPAT on cure rate, however, is less conclusive. When all models are considered, OPAT appears to produce superior results compared with inpatient treatment, a finding that is influenced by the inclusion of positive studies reporting on multiple or unspecified OPAT treatment models. When these studies are removed and specific models are considered individually, outpatient attendance appears to have a lower rate of cure or improvement,22 and SA59 or OPAT by a SN has a higher rate,56,57 whereas OPAT by GN has no impact. 12,41 Results from those studies assessing the impact of treatment via OPAT specifically on lung function in patients with cystic fibrosis were either inconclusive or found no impact. 42,43,58,60

In the remaining studies, which looked at aspects of clinical effectiveness for OPAT only, rates of cure and/or improvement ranged from 61.1% to 100% (mean 89.6%; median 92.4%). When the various OPAT models are considered individually, the highest average cure/improvement rate was seen for the SN model (90.6%), followed by SA (91.3%), the GN model (90.0%) and HO treatment (85.9%). Few studies reported on bacterial eradication, but those that did saw rates of between 57.1% and 100% (mean 86.2%; median 90.0%) (see Appendix 1, Table 33).

Patient safety and adverse events

Only 24 of the 109 studies evaluating OPAT-related safety included a comparator, which in the majority of cases was inpatient treatment (see Appendix 1, Tables 32 and 33). Five studies did not specify the OPAT model that was being used,48,52,55,61,62 and three others reported combined results for multiple models. 29,49,63

Synthesis of the findings from these studies indicates that, regardless of the model used, there is little evidence of impact on either drug-related side effects or number of deaths in OPAT patients in comparison to patients receiving treatment in hospital10,22,25,41–43,56–59,64 (see Appendix 1, Tables 32 and 34). One study looking at outpatient attendance22 did find a higher death rate (1 patient vs. 0 patients), but this was a small study and the overall rate of side effects was lower in OPAT patients (15% vs. 18%). There also appears to be no conclusive evidence of benefit either in relation to hospital readmissions overall, or in relation to those who self-administered therapy,42,59 although there were conflicting results for OPAT provided by nurses, with SNs10,25,57,64 seemingly having superior results to those of GNs. 12,41 Perhaps unsurprisingly, overall there would appear to be more line-related complications in i.v. therapy administered outside hospital.

Across all studies, the most commonly reported adverse events were rash, fever, nausea/vomiting, diarrhoea, allergic reaction or anaphylaxis, phlebitis, leucopenia and line complications (including line infection, occlusion, breakage, and dislodgement).

Cost-effectiveness of outpatient parenteral antimicrobial therapy

Although many of the identified studies reported on the cost of OPAT, only five considered cost-effectiveness (see Appendix 1, Table 35). Three studies applied decision tree models to OPAT provided by SNs, with one also determining the cost-effectiveness of SA. The remaining studies (one literature review, one retrospective observational study) did not specify the OPAT model(s) used.

In two of the three decision tree analyses, i.v. OPAT was found to be more cost-effective than i.v. inpatient therapy. 24,65 In one case, it was also more cost-effective than early discharge with oral therapy and oral outpatient therapy,65 while in the other its dominance was maintained only when the i.v. success rate was > 55%. 24 Conversely, in the third study, i.v. OPAT was found to be less cost-effective than both i.v. to oral switch therapy, and oral treatment both during and after hospitalisation (which was the most cost-effective option). 66 The authors reported the probability estimates used, which were obtained from both published research and institutional data, to be a limitation of their study.

Studies included in the systematic literature review predominantly concluded that home care i.v. antibacterial therapy would lead to significant cost reductions from a societal and third-party payer perspective. 23 In 5 of the 11 studies, inpatient therapy was 2–3 times as expensive as home care therapy. However, there was considerable variation in the ways in which costs were determined and calculated in the individual studies (e.g. incremental costs, costs for selected components only, etc.), and the review itself lacked considerable detail on the methodology used and the criteria for study inclusion and exclusion.

The results of the observational study (based on 435 courses of i.v. antibiotic treatment for respiratory exacerbations in 116 adult patients with cystic fibrosis) indicated that, for both one course and 1 year of treatment, i.v. antibiotic treatment administered mostly in hospital was more effective but more costly than treatment administered mostly at home. 53 This improved clinical effectiveness could be achieved only with the input of considerable additional resources (between £46,000 and £73,000 per patient at 2002 prices). However, when the strictest definition of effectiveness was applied (≤ 0% decline in lung function), hospital treatment was unlikely ever to be cost-effective.

Patient acceptability

Only 4 of the 36 studies considering patients’ acceptability of OPAT involved a comparison of inpatient and outpatient therapy (see Appendix 1, Table 36), two of which involved OPAT delivery by GNs12,41 and two of which involved OPAT delivery by SNs. 25,56 In each case, satisfaction was high, with home treatment seen as being beneficial. One of the two studies involving GN delivery found that only 5% of home group patients would have preferred hospital treatment, compared with 35% of the hospital group who would have preferred home treatment at home (p < 0.001),12 while the other found that patients in the home group were significantly happier with the location of their care than those receiving inpatient therapy (p < 0.001). 41 Similarly, in one study for which patients received home care by SNs, almost all (97%) indicated that they would chose to receive at-home therapy in future and would recommend it to others. The main reasons given for this were quiet and increased home comfort, familiar environment and free choice of activity. However, some patients in the study also reported disadvantages to receiving therapy at home, primarily related to patient and caregiver anxiety. 56

Of the remaining studies that considered acceptability in OPAT patients only, most involved multiple OPAT models (and did not differentiate between them in their findings), or did not specify the model(s) under study. In general, satisfaction with treatment was very high,2,13,27,36,67–83 including when patients had to have frequent attendance at hospital. 5,84 Commonly perceived advantages of OPAT included the ability to resume daily activities,2,27,71,73 feelings of improved self-esteem or greater freedom and control,67,71,73,85,86 and not having to remain in or attend hospital. 2,71,82,85,87 The main disadvantages most commonly related to infusion equipment, and included anxiety about the device and its sterility,35,37,78 the discomfort and limitations imposed by pump devices,36,37 and issues related to storage. 88 Two studies found that younger patients were better able to use infusion devices, and required less support to do so than older patients. 37,89

Two studies determined patients’ willingness to pay to have treatment in their preferred location, and although differences did not reach statistical significance, patients reported that they would pay more for home-based than for hospital-based treatment, including giving up slightly more of their remaining life to ensure this. 15,79

Provider acceptability

Only six studies included some form of assessment of practitioner acceptability, one involving general practitioners (GPs), one involving nurses and four involving infection specialists (see Appendix 1, Table 37). In most cases, professionals saw advantages for patients (or a need for) receiving i.v. antibiotic therapy outside hospital. 39,86,90,91 However, there were also negative perceptions of practitioner involvement. Most GPs saw no advantage to themselves in home treatment, and many thought distance from hospital was an issue for patients. 86 Similarly, nurses perceived that there were challenges in providing this model of care, mainly around the technical nature of the devices used and dealing with patients’ understanding of the technology and its related risks. 39 Finally, many specialists saw logistical and organisational barriers to the use of OPAT relating to a lack of funding, the availability of a dedicated OPAT team, the number of locations involved, leadership, communication and the links between primary and secondary care. 38,90–92 In addition, there were concerns regarding who should assume the cost and/or take clinical responsibility for patients. 90,92

Aim

To explore the types of OPAT services that have been commissioned and provided in the UK and to identify perceived barriers to implementing this type of service.

Although surveys of health-care professionals providing OPAT in the UK were carried out by BSAC in 2011 and 2013, these largely focused on clinical matters such as types of infections treated rather than models of service provision. We set out to look at the service models provided and the issues influencing their development.

Objectives

• To undertake a survey of NHS trusts to identify services in England and their configuration.

• To sample from these NHS trusts to identify participants to interview to explore the barriers and levers to service provision.

Health-care professional survey

A brief electronic survey of infectious disease specialists was conducted using the Bristol Online Survey (BOS) to identify services in England and their configuration.

A letter with a link to the BOS survey was sent to all NHS trusts in England. The letter invited the recipient to complete the survey and to pass the link on to relevant colleagues. When a microbiologist, infection specialist or OPAT team could be identified from the NHS trust website, the letter was sent directly to them. When no details could be obtained, the letter was directed to the hospital main administration office. E-mails were also sent to all infection specialists and microbiologists, where addresses could be identified. The survey was distributed in June 2013 and reminders asking people to complete the survey were sent out in July and August 2013.

The survey was designed to enable us to sample a diverse group of respondents for interview. An initial corpus of questions was developed, from which 20 were selected by the clinical team to give a picture of current practice and to enable us to select a diverse sample. The questionnaire was piloted on five local clinicians to ensure that the questions could be easily understood and that the survey links worked.

The survey asked respondents questions relating to the following.

• Who runs the service and the staffing levels (doctors, nurses, administrators)?

• The involvement of other clinical specialties (e.g. infection specialists, microbiology, pharmacy).

• The size of service and numbers of patients seen per month.

• The involvement of external organisations (e.g. private health-care providers).

• Any future plans (development of service).

• Demographics (where is the service based and what is its coverage), respondent details (job title/position, years in post, years running OPAT service).

The findings from this initial survey were then used to construct a purposive sample using the following criteria: NHS trust type [teaching, foundation trust, district general hospital (DGH)]; geographical area (urban and rural); socioeconomic area [low and high socioeconomic status (SES)]; and diverse ethnicity. Some selection criteria were nested (e.g. hospital type, geographical area) and participant selection ensured that a range of view-points were identified.

These results should be considered in the light of the poor response rate, which could be explained by the timing of this survey. BSAC had just undertaken their survey and there may have been an element of ‘fatigue’ among potential respondents or respondents may have thought that it was the same survey.

Sampling

Of the 35 people responding to the BOS survey, 25 agreed to be interviewed. Our original protocol provided for a purposive sample of up to 30 service leads to be recruited using the sampling frame below in order to collect a wide range of experiences. It was expected that some selection criteria would be nested (e.g. hospital type, geographical area):

• NHS trust type (teaching, foundation trust, district general hospital)

• geographical area (urban and rural)

• socioeconomic area (low and high SES)

• diverse ethnicity.

Interview process

Professional leads were identified from the electronic survey and all who gave permission to be contacted were invited to take part in a telephone interview. All agreed to be interviewed, but four interviews were not conducted owing to difficulties organising interviews, and one interview failed to record owing to a technical failure. For those who agreed to participate, written informed consent was requested along with permission to record the interview.

The interviewers used a semistructured topic guide, developed by the study team from the available literature, to explore current service provision and any facilitators of and barriers to implementation. The interviews were audio-recorded and transcribed verbatim where permission was given. Field notes were taken where permission to record was refused.

Procedure

Four researchers [one psychologist, two applied health researchers (one with a nursing background) and one sociologist] carried out the interviews. Recruitment and set up was delayed owing to the limited time that respondents had available. Interviews were conducted by telephone and recorded with the permission of the interviewee. One participant refused consent to record the interview and notes were made during and after the interview and became part of the corpus of data. Interviewees often had limited time, and interviews varied in length from 30 minutes to 90 minutes. In some cases the interview schedule had to be adapted to fit the time available, so there are cases where some information is missing. Unfortunately, owing to changes in personnel at the start of the study, data analysis was not undertaken until all the interviews were completed, so new emerging themes could not be explored as we would have wished.

Data management and analysis

The data were managed based on principles of information governance at the University of Leeds. The data from the interviews were analysed using a framework approach allowing a structured exploration of the participant’s perspectives and a method to compare and contrast different service types. 95 Data analysis comprised five stages: (1) familiarisation with the data; (2) identifying the thematic framework; (3) indexing; (4) charting; and (5) mapping and interpreting. The process of familiarisation enables the researcher to identify emerging themes or issues in the data. Little is known about why NHS trusts choose to deliver specific OPAT models and so the evidence generated from the systematic review and input from our clinical co-applicants was used to help refine the thematic framework (stage 2). All of the data generated from the interviews were indexed numerically according to the particular theme to which they corresponded (stage 3). Data were then lifted from their original text and placed under subheadings derived from the framework (stage 4). A process of constant comparison was used to examine across themes and cases. This approach was employed to ensure the collection of a large amount of detailed information about the range of services, geographical location, barriers and facilitators (personal and attitudinal), resource issues including staff, budgets and processes for managing and monitoring patients. The interview transcripts were used to identify key information for each service and key themes about the development and implementation of OPAT services in the UK. These interview data were considered by the expert panel and informed the modelling and DCE workstreams.

Service organisation

Outpatient parenteral antimicrobial therapy services were based in a range of departments, with 20% (n = 7) based in acute medicine departments, and 29% (n = 10) based in infectious diseases departments; however, the majority (54%) were spread across a range of departments including microbiology, surgery, orthopaedics, accident and emergency (A&E) and respiratory medicine. Other respondents described their service as ‘informal’ and not located in any specific department. Most services were modest in size, with 48% (n = 13) of services having between one and three whole-time equivalent (WTE) dedicated OPAT nurses, and only 7% (n = 2) having more than five nursing staff. Five services (18%) had nurses who worked on the OPAT service in addition to other roles (e.g. ward nurse). About half of services (48%) had less than one WTE doctor involved in the service, and in most cases the service was consultant led. A small number of services (n = 2, 7%) were led by a microbiologist and a nurse, and one service was covered by a hospital-attached GP.

No one had much administrative support, with 74% (n = 20) reporting less than one WTE administrator working for the service, and 26% reporting that they have no administrative support. Eighteen respondents (two-thirds) had an infection specialist (microbiologist or infectious disease specialist) involved in the service, with another 18% having access to a microbiologist. A total of 15% of respondents had no infection specialist or microbiologist involved.

Eight respondents said that they did not offer an outpatient i.v. service, but went on to provide details of their service. This finding may be explained by the individual completing the survey not having involvement in the services offered by their organisation. Some insights into this have emerged from the qualitative interviews (discussed later). Some of those eight individuals did not class the services provided by their organisation as conforming to an OPAT model. Some described services provided by outlying community hospitals (part of their trust) which were not overseen by the infectious disease specialists. These were not described as official OPAT services by those interviewees.

However, the descriptions provided by those eight respondents appeared broadly similar to those provided by respondents who said that they do offer an OPAT service. Services are based in a range of locations, with two services based in microbiology, two based in acute medicine, one based in infectious diseases, one operating as an ad hoc service with no base and two not running a service. Staffing levels are also similar between services reported by those who said that they ran an outpatient i.v. antibiotic service and those who said that they did not. Half of respondents who did not run an outpatient i.v. antibiotic service had less than one doctor involved in their service; one service had five or more and three said ‘other’. In terms of nurse staffing levels, three respondents had less than one nurse, two had between one and three nurses, one service had five or more nurses and two said ‘other’. A total of 63% (n = 5) of the eight respondents said they had less than one member of administrative staff, with the remaining respondents answering ‘other’ to this question.

Service size

The majority of units that provide an outpatient antibiotic service treated between 1 and 10 short-term and long-term patients per month (44% and 52%, respectively), with only one unit treating 30 or more short-term patients per month. Most units offer more than one model of care (Table 6), with > 80% offering a hospital attendance service and three-quarters offering a district nurse model. Few services use a ‘general nursing’ model for the provision of OPAT.

Of the eight respondents who said they did not offer a defined outpatient i.v. service, four services reported that they treated long-term patients (between 1 and 20 patients per month), two services treated between 1 and 30 short-term patients per month and one service stated that they treated 30 or more short-term patients per month. However, half of the services reported that they did not treat long-term patients or did not record figures. Just over half of respondents said that they either did not treat or did not record the number of short-term patients.

Half of the respondents who did not report a specialised outpatient service said that their patients visited the hospital to have treatment. The other half of services offered a district nurse service at home. SNs were used by three of the eight services, and GN and SA models of care were used by two services. Two respondents said that they did not use any of the services.

Rationale for model of care offered

Respondents reported that they offered an OPAT service largely because the lead clinician has been an advocate for change (70%; n = 19). However, there are often several drivers for change, including patient preference (63%), commissioners’ decisions (26%) and management decision (40%). It seems likely that the decision to start or continue a service is multifaceted, involving decision-making at several levels.

Sample characteristics

Of the health professionals interviewed, four stated that they were from a tertiary service that treated patients from a range of counties; two served areas in the north of England, one served an area in the south of England and one service served patients in the north of England and the Midlands.

Some sites serving largely rural populations provided only a service to those living within a few miles of the hospital (although the distances varied), whereas others offered a limited service (generally a nurse at home model) across a larger area. Eleven respondents reported working in a teaching hospital, most of which were also foundation trusts; the remainder were DGHs. Eight respondents worked in areas with high levels of economic deprivation, and these were also areas of high ethnic diversity. Eight respondents reported that they did not run an outpatient i.v. antibiotic service. The majority of those who said that they did not run an OPAT service reported that some clinical specialties had ad hoc services offering some form of community/hospital-at-home service on a case-by-case basis. One respondent did not elaborate on why there was no outpatient i.v. antibiotic service and another respondent stated that there was no service because there was an intention to use a private company to deliver i.v. antibiotics in the community. (A summary of the key characteristics of each service can be found in Appendix 2, Table 38.) The professions of those interviewed included nurse specialists (3), microbiologists (6), pharmacists (2) and infection specialists including joint qualified (9).

This study includes descriptions of 19 OPAT services in the UK (20 interviews), details of which are provided in Appendix 2, Table 39. Service types are summarised below.

Themes from the analysis

The analysis of the data from interviews with professional leads from OPAT services currently being delivered has provided an in-depth analysis across a cross-section of services in England.

A priori assumptions and emerging themes:

1. variations in resources needed to deliver OPAT

   1. staffing

   2. funding

   3. location and geographical issues influencing delivery

2. facilitators contributing to service delivery

   1. core staff and additional staff needed

   2. conditions treated and pathways for doing this

   3. monitoring effectiveness

   4. evaluation, impact and outcomes

3. barriers to delivering a service

   1. lack of planning

   2. lack of evidence

   3. lack of resources or facilitators

4. relationships within and between people in organisations (e.g. commissioners, providers, partnerships).

Emerging themes that were uncovered through the analysis have included:

• the influence of change management in transforming services

• the role of personal and professionals networks to exchange information

• risks to patients through management and monitoring of services.

Service models

Building the case for support and commissioning the service

Team members and working relationships

Monitoring and review, governance and risk

Evaluation, outcomes and impact

Outcome-based evaluations were few. Around seven services had evaluation plans in place:

Yes we did our last evaluation about 2 years ago and that was a full evaluation looking at patient satisfaction and we also have a look at lines from the point of view of thrombosis and infections and adverse consequences so we have a programme where we review everything on an annual basis so pretty much we take each one of those each month and assess it and basically about 18 months from now we do a full evaluation.

Interview 12

One of the key drivers of implementing this type of service was to save bed-days and to reduce the numbers of patients remaining in hospital for prolonged periods. There are some good cases of services that have saved excess bed-days. At least one service has been able to translate this into changes by reducing the number of beds on wards from 18 to 12. One respondent said that there had been:

an absolutely massive difference on the number of patients that are staying in. So we are reconfiguring wards so there’ll only be 12 beds instead of 18.

Interview 15

In some cases, there was a lack of clarity about how to translate a saving in bed-days into real changes in the system:

I think it is quite complicated so if the patient has now gone home and that bed is empty, the only way you’ll save any money on that bed [is] if you [close] that bed and you no longer staff it. [Closing] random beds scattered throughout the hospital, you’re not really going to make any savings on those, and you’ll only save if you close half a ward or something like that.

Interview 10

There is clearly some knowledge in the system about how to do this and a mechanism to share such learning could be considered.

Most respondents mentioned service evaluation based on feedback from patients, focusing on patient satisfaction and reporting a positive response:

The feedback has been fantastic. Feedback that has helped us to get more funding to be honest, patient satisfaction is extremely high so that’s been good.

Interview 15

It’s a no-brainer for a patient really, they get out of hospital.

Interview 5

Some of the leads alluded to improving outcomes such as returning people back to their usual lives and functioning before they were ill:

The important thing that we found from patients is you know this is not trying to get them out of hospital and save money and although there’s an element to that, the aim is to get them back to the life they had.

Interview 12

Change management

Most services proposed some changes to their current system, although some believed that there was no good reason for change:

It’s primarily configured in the way that it’s configured because no one has decided to configure it in a different way. There just didn’t seem to be the requirement for the sort of configuration that I’ve described that, seemed to work well for the disciplines that were delivering it, so we just left it alone really.

Interview 1

The services with the best levers and agents for change tended to be in urban areas, but there were problems across the board. Many services took an incremental rather than a transformational change approach and did not have a vision of the service for the future. There were often tensions in terms of needing to offer something now (based on a service that evolved over the years) and the need to change based on knowledge of best practice. A few were currently waiting for plans to be approved.

Only one service lead described how they had visited another successful service and adapted the model to suit their own local context:

I spent some time with a service [. . .] when I was trying to set up this service [. . .]. and I tweaked it to kind of suit how we run ours, and that’s how when I set up this service I kind of configured it [based] on theirs.

Interview 2

This indicated some partnership between services. In some areas it could be beneficial for services to work together and share best practice and possibly solve problems, which could occur at a regional level. There is also scope for developing a national commissioning framework that could be adapted at a local level.

There were variations in the perceptions that service leads had about the potential of the organisation to change. In some cases there were reports that senior managers or commissioners were not convinced by the evidence or the scale of the impact and benefit for patients. Some felt that they were deliberately being obstructed.

I don’t really quite understand what all the difficulty is, but there is always historically huge managerial resistance.

Interview 20

Relationships within and between people and organisations

Service leads have developed and evolved services based on their informal and formal networks, which may depend on the size of the trust. Many services developed and evolved based on the motivation and networking abilities of the leads:

It’s a small trust, and we’re pretty well supported by the consultants that use it, so we’re able to contact those fairly easily. I think if it was a bigger trust and you couldn’t get hold of your team then that may be more difficult.

Interview 2

Some service leads did not include a dedicated post such as a pharmacist, but the pharmacy service was still involved in an informal way and may be involved as part of a wider remit:

[. . .] and we are desperately trying to get a nominated pharmacist but that hasn’t happened yet. There is one pharmacist normally in charge of infectious diseases not just OPAT and so if there are problems I ask her but there isn’t regular review or someone to sort of go through the charts and make sure of all the interactions that is really something that we are lacking.

Interview 5

This could indicate a stretched resource or a small number of patients with infections in that area and may be linked to geographical context. Some services such as microbiology and pharmacy are implicit as part of such a service. There were variations in the level of interaction that services had with commissioners, which could be related to the post held by the service lead and their level of seniority. At least two services were jointly commissioned with involvement of partners across CCGs, NHS trusts and community organisations. In some cases, there was tension between providers and commissioners and a lack of trust or engagement between groups.

Service models

Muldoon et al. 91 undertook a survey of OPAT in the Republic of Ireland and reported a participation rate of 10.7%. However, their target participants were slightly different from ours in that they had a broader professional base which included consultant physicians who made use of OPAT for their patients. They report similar results to our findings about the most common clinical conditions treated and some of the problems reported by our respondents. Owing to our modest sample size, our survey results are not generalisable but they do enable us to draw up a representative sampling frame to undertake the qualitative interviews.

The OPAT professional leads interviewed delivered services in a range of settings. Most offered some level of hospital provision, administration and delivery as well as a community delivery option. Community services were delivered by district nurses or private companies. Service models varied from being well organised and well planned with regularly updated business plans to those described by leads as being ad hoc. Many services had evolved and changed incrementally, although there were also examples of transformational change.

Lane et al. 38 suggest from their results that routine infectious disease consultations in an OPAT service can potentially improve antimicrobial stewardship, citing Sharma et al. 11 for their conclusions. Sharma et al. 11 found that when the infectious diseases consultant was involved in patient assessment, there was optimisation of the treatment regimen and 27% of patients did not require OPAT. Defining leadership within teams may contribute to improvements in care.

It appears that the further away from the hospital a patient lived, the less likely they were to be offered a service in which a nurse comes to their home. However, there were some exceptions to this. The logistics of delivering the service with populations over wide geographical areas did cause some specific issues for delivering an OPAT service particularly for patients who required more than two i.v. administrations per day. Lane et al. 38 identified the problem of providing care over a large number of locations as a potential barrier to safe care.

Team members and working relationships

Some services had dedicated OPAT posts, whereas others included OPAT as part of a wider remit; this could depend on a range of factors such as the evolutionary organisation and planning of the service. Although good practice recommendations are in place, services did not always manage to meet the criteria on service structure. 93 Responsibility for care was not always explicit and there were examples of clinical leadership in the form of a designated role being lacking. Leadership was often provided by nursing staff in those situations. The lead role was often bestowed on an individual, particularly in the unplanned type of service. This meant that ownership of the service and perceptions of an individual’s ability to improve or transform the service varied. The additional work could be perceived as an additional management burden on top of usual roles and be interpreted as resistance to further change. Again, the OPAT good practice recommendations have specific recommendations regarding leadership of the service, and there were instances in which these were not met. 93

Organisation, funding and commissioning

Business cases were viewed as important to build the case for support based on existing evidence and local insight. This also may reflect the practice of the hospital senior management team and or relationships with commissioning partners. Several services had to provide annual reports to governance committees to maintain funding levels based on the numbers of referrals.

Some services had received short-term funding with no guarantee of continued investment. Services were often limited by their funding, with expansion being difficult. Funding was a primary reason why services might adapt or evolve over time. Muldoon et al. 91 reported problems obtaining funding to cover the costs of OPAT in both the public and private health systems.

Patient management, monitoring and evaluation

A number of services have monitoring systems in place. Some referred to national standards (e.g. the BSAC surveillance system). 93 In many cases, this was linked to a greater level of scrutiny in terms of the clinical governance of the service. Individual patient results were reviewed by multidisciplinary teams (MDTs) via virtual wards and were then passed on to governance committees and intelligence functions, often as formal reports. This level of monitoring is considered best practice in the literature;98 however, the survey work undertaken by Muldoon et al. 91 and Lane et al. 38 report that responsibility for monitoring patients is patchy at best. Lane et al. 38 noted in their US survey that only 22% of respondents reported having a system in place to pick up adverse events or ‘near misses’. They also found that a dedicated OPAT team treating 16 or more patients per month was more likely to have monitoring systems to review and act on patients’ laboratory results. Conversely, the fewer patients treated, the less likely it was that patients would be cared for by a dedicated OPAT team, which the authors conclude is a barrier to providing safe services. 38

Outcomes

For those who were able to discuss evaluation of their service, the outcome on which they tended to focus was reducing bed occupancy, although some did highlight the need to improve patient satisfaction with care received. There is considerable scope to identify relevant patient outcomes and improve care.

Service models

1. A wide range of infections were treated, from cellulitis to bone and joint infections, diabetic foot infections, respiratory, renal and other infections in a broad range of specialties. This means that OPAT services within a number of organisation could be somewhat fragmented. Examples of effective patient pathways could be shared, with the caveat that they be adapted to local circumstances and not just used as an ‘off the shelf’ solution.

2. Self-administration by patients or family members was uncommon among our participants. In some areas it was restricted through lack of resources or a perception that there is no need for it. Shared examples of where this works well and how this has been achieved could support those areas that have not yet succeeded in providing this for their patients.

Working relationships

1. The role of OPAT champions and of leads appointed to transform the service need some consideration. Commissioning guidance could focus on the type of partnership and input that is needed from stakeholders to drive forward and sustain any changes.

2. Some work may be required to establish how formal partnerships succeed or to audit the effectiveness of existing formal and informal networks. It is possible that this work may have to be prioritised at a regional rather than a local level depending on the population served and the size of the trusts involved.

3. One respondent listed five key issues that drive commissioners and providers to develop a more formalised service. These were:

   1. clinical governance

   2. individualised care that is currently not patient centred

   3. decreasing costs and getting people out of hospital and creating beds

   4. patients’ preferences to be managed at home

   5. meeting national good practice.

Some of the leads expressed an interest in the development of a national commissioning framework to benchmark consistency in commissioning and providing OPAT services. This could be helpful particularly if made available from the BSAC website.

1. There are some services that have established evaluation frameworks and it could be helpful if these were shared.

2. Outcomes for services need to be considered and should include improvements in patient care, cost-effectiveness, professional and patient satisfaction with the services provided. This could be informed by national guidance.

Design

Semistructured interviews and focus groups.

Study settings

The study setting for conducting this research was secondary care, and four hospitals were purposively selected because they offered the following care pathways: HO attendance, nurse at home (GN and SN models) and SA; the centres provided more than one model of care. Two were large teaching hospitals and two were DGHs. Each site also offered significant diversity in its sociodemographic characteristics (Table 7).

Participants

A purposive sampling strategy (see Sampling characteristics) was adopted. Two groups of patients were identified: (1) patients requiring short-term i.v. antimicrobials (n = 15); and (2) patients with deep-seated infections requiring longer-term i.v. antimicrobials (n = 25). The estimated sample size was based on previous studies102 and assumes that those on longer-term antimicrobials will be a more diverse population. It was our intention to capture a detailed and comprehensive range of perspectives. The use of qualitative data to develop DCE experiments is relatively new, so the final number of participants was dependent on theoretical saturation. 103,104

Sampling characteristics

• Short-term and long-term OPAT.

• OPAT via one of the four care pathways.

• Sex.

• Maximum variation in age (≥ 18 years of age).

• Simple (e.g. cellulitis) and complex infections (e.g. bone infection).

• SES.

• Ethnic background.

All participants were invited to take part in a focus group or a qualitative interview (face to face or telephone). Initially, the project intended to conduct five to six focus groups and additional interviews with key informants. However, it proved difficult to recruit participants to focus groups, as working-age patients did not wish to take time off work, so the recruitment plan was revised and only one focus group was completed.

Consent

Participants were approached by NHS research nurses and given a copy of the participant information sheet and given at least 24 hours to decide whether or not to take part. Details of consented participants were then passed to the research team. Ethics approval was sought and obtained from the National Research Ethics Service Committee South West – Frenchay (reference 13/SW/0060).

Procedure

At the start of each session participants were informed of their rights as participants (right to withdraw, confidentiality) and were offered the opportunity to review a copy of the transcript and results. The purpose of the study was explained, and the interview followed the structure below. Interviews took place at the patient’s home or the university. Interviews were audio-recorded, with permission, with one participant refusing to be recorded (notes were taken by the interviewer). Interviews lasted between 30 minutes and 1 hour and 15 minutes.

Topic guide

The interviews and focus group discussions were semistructured and explored patient satisfaction, issues and preferences. The topic guide covered four main questions:

1. What has been your experience of receiving i.v. antibiotics for infections? What were the good and bad points of the care/service you received?

2. What are the most important aspects of i.v. antibiotic services for you?

3. If you were designing a service to provide community antibiotic i.v. services what would it look like?

4. How did the i.v. antibiotic treatment course impact on your everyday life (and that of your friends/family)?

These questions had a list of probes related to each question, to prompt further discussion and gain further insight into pertinent issues.

Data analysis

Interviews were conducted by four team members [MT and SM (psychology), JE (sociology) and, following the departure of JE from the study, CCM (nursing)]. MT, CCM and SM contributed to the analysis of the data. The team met regularly throughout the data collection period to review transcripts to ensure that they were covering the same topics and to identify issues to explore in later interviews. Owing to the departure of JE from the project, data collection and analysis became desynchronised, with data analysis starting after 12 interviews had taken place.

The audio-recordings were transcribed verbatim, anonymised and entered into NVivo version 10 software (QSR International, Warrington, UK) to facilitate data organisation, coding and retrieval. Data were analysed thematically for patterns and themes to understand what patients valued about OPAT services and to explore differences in their experiences using constant comparative methods. Data analysis followed the standard methodology for thematic content analysis, with close reading of the data to identify words that capture thoughts or concepts. Labels/codes were attached to these data and became the initial coding frame. Codes were then sorted into categories based on how they relate to one another and grouped into meaningful clusters. We also drew from aspects of framework analysis, developing matrices95 to map the data and aid categorisation.

Initially, two researchers independently read the transcripts and listened to the first three interviews, from which an initial coding frame was developed. Following this, the wider team examined and agreed the preliminary codes, and a coding index was agreed and applied in NVivo to the remaining transcripts. The research team met fortnightly to discuss the analysis and to examine specific cases for disconfirming perspectives. Before the end of coding it became clear that data saturation had been reached, as no new ideas relevant to constructing the DCE were identified from the last five interviews. 103

Three meetings were held between the qualitative team (MT, CCM, SM), chief investigator (JM), modellers (DM, SH) and two of the PAG members (HG, CT) to discuss the relevance and suitability of the themes produced, the ways in which the themes were linked and candidate attributes.

Copies of the interviews were sent to three participants who asked for a copy, but no changes were requested.

Changes from the protocol

We were unable to recruit enough participants to focus groups so we changed our predominant data collection method to interviews. This provided richer data, but required significantly more resources.

Theme 1: meeting the needs of a diverse population

For many, the benefit of OPAT was not being admitted to hospital. Most did not see their infection as warranting this, and those who had previously been treated as an inpatient saw OPAT as an opportunity to get home earlier. However, being cared for as an outpatient could be challenging:

. . . they [staff] came back and said was I sure this was what I want to do [go home on OPAT], and I was like ‘Yeah, as long as you put in place that I can have someone come out to me, then I’d like to go home.’ I didn’t realise how tiring it would be though. It’s still better than hospital, but I never realised that just making a cuppa could be so tiring!

Female, < 65 years, long term, nurse at home

Older participants, however, often put caveats on when OPAT would be suitable, taking into consideration the severity of infection, the patient’s general health, family circumstances, number of i.v. infusions per day and level of mobility:

It depends on the condition you are in, if you couldn’t move about you would have to wait inside (at home) for somebody to arrive, but only if you have family at home for you, if you could move about, you would be better off coming to hospital.

Male, > 65 years, short term, A&E

The nurse at home model was viewed as more appropriate for the very old or infirm, especially given the challenges of travelling to and parking at hospital. A few participants proposed alternative places where OPAT could be delivered, such as community clinics and GP surgeries and felt that this may be less intimidating for older people than the hospital:

. . . we have a little centre which is mainly all the elders and a lot of them feel, they feel a little bit more scared of the hospitals because some of them are single or widowed so they don’t always have somebody to go with them to the hospital [. . .] a local clinic would be much less stressful for them.

Male, < 65 years, short term, A&E

Many people mentioned that they did not want to be treated in hospital if at all possible, and their reluctance was unrelated to age, length of infection, frequency of infections or type of OPAT received. For some, their fear was linked to the perceived risk of contracting an additional infection, often fuelled by news reports as well as personal experience:

It’s always a worry in hospital, are you going to pick up a staph infection? Things like that.

Female, < 65 years, long term, clinic

Half of participants felt that OPAT services were understaffed, with hospital-based services viewed as particularly affected. None of the patients interviewed felt that this affected the quality of care they received and all were sympathetic to the stress that staff were under, suggesting that more staff and additional facilities were needed to address demand:

I think everyone’s absolutely rushed off their feet in that unit.

Female, < 65 years, short term, A&E

Theme 2: benefits of and barriers to different models of care

A major benefit of OPAT, regardless of the model of care, is the ability to enjoy the comforts of home and not disrupt things such as diet, sleeping patterns and family life, which people felt was hugely important to their recovery and well-being.

Theme 3: effectiveness of treatment

About half of patients believed that i.v. antimicrobials were more effective than oral treatments. Their apparent effectiveness caused patients to question why they had not been offered i.v. earlier, as their infections responded quickly to the change of treatment:

I would have like the full dose, not an escalation policy, i.e. you can only take orally up to a certain level and therefore, when that’s failed, we’ll put the i.v. in. If we know that for many people it’s going to fail then overprescribe. I don’t know what proportion of people that oral antibiotics would work you see.

Male, < 65 years, long term, A&E

Some patients experienced a switch from i.v. to oral treatment and a few worried that their treatment had been changed prematurely, largely because the planned switch was not explained to them. This was noted only for nurse at home and hospital attendance patients. In the case of cellulitis, the time needed to treat the infection is shorter than the time required for the skin to heal, leaving patients uncertain about whether or not the infection was cleared. In these cases, patients commonly said they would have liked to continue i.v. treatment for longer:

He’d said to me that they were going to review after x amount of days, and it was still a little bit red and still swollen. Not as swollen and not as red, and I said ‘look, I’m not happy, I know I’m going to be back here within a fortnight if I don’t have a little bit more’. And he said ‘well go on then!’

Female, < 65 years, short term, clinic

A few patients experienced a subsequent flare-up of their infection, which led them to conclude that their i.v. had ceased prematurely. Unfortunately, as most short-term infection patients were not seen by a health-care professional at the end of treatment, this was picked up only when the patients went back to their GPs, demonstrating an important unmet need of patients:

Well, I went to the GP a week after I finished the i.v. from the hospital, and it (cellulitis) was as bad as ever, so he sends me back to A&E.

Male, > 65 years, short term, clinic

Theme 4: communication

Theme 5: review and aftercare (follow-up)

Review and follow-up at the end of treatment was important to many but not to all patients and their experiences differed. Most long-term patients were reviewed regularly, and, although some patients felt strongly that this review should be undertaken by a doctor, others were happy for the nurse to do this. What was important was being able to contact the OPAT team between appointments as this provided reassurance that they were being cared for as well as if they were in hospital. Being followed up at the end of treatment was particularly important if the patient had not been reviewed face to face during treatment, and a lack of aftercare was a cause for concern:

I’ve got follow up in a month which is nice so they’re keeping an eye on me, I wouldn’t like it if I hadn’t been.

Female, < 65 years, long term, clinic

However, few short-term patients were followed up at the end of treatment. Some were advised to come back to the clinic if they had concerns. Some were content that this was their responsibility, but others would have preferred an appointment and wanted to know whose responsibility it was to organise this. This was of particular concern to patients who had been cared for by a nurse at home, as they were generally not seen again in person by the doctor after the initial diagnosis:

I suppose I was left in the dark as to know what was after the IV, nothing at all. I’d rather if they said ok, make an appointment to see your doctor.

Male, < 65 years, short term, SN

Self-administration patients were all on longer-term i.v. treatment and valued their mid-term and aftercare appointments, as they saw these as important to their care:

There’s a liaison nurse who comes down on a pre-arranged date and time and they’ll usually do (tests) and chat and make sure everything is going ok, that I haven’t got any issues, see how I’m feeling, see if there’s any improvement or effect.

Male, < 65 years, long term, SA

Theme 6: staff expertise

Patients used the term ‘expertise’ to refer to both a staff member’s level of knowledge of infections, i.v. antimicrobials and treatments and their practical experience of delivering OPAT, emphasising both equally. People valued being dealt with by competent staff, and felt that such expertise was built up over time. Ideally, participants wanted to be cared for by people who understood their infection, were aware of complications that could arise and knew what to do in those circumstances. Patients were impressed with staff who knew about their personal circumstances and liked it when they felt that staff ‘went the extra mile’:

The nurses that come out are specialist nurses who are informed about antibiotics and about lots of illnesses that they are treating, because they obviously need to know [. . .] I felt very comfortable that they were very knowledgeable about what I was experiencing and this reassured me about coping at home alone.

Female, < 65 years, short-term, SN

Some patients wanted their care to be overseen by a doctor, because of their extra training and qualifications. Although these patients were happy to be treated by a nurse, they wanted the reassurance of a doctor overseeing their care:

How would you have felt if it had been, for example, a nurse-led service?

Umm, I would have been worried, I think you need a qualified doctor in charge, to do the examination, and make a decision on what to do next, what to prescribe, to continue the treatment.

Male, > 65 years, short term, A&E

Theme 7: impact on family and friends

Outpatient parenteral antimicrobial therapy treatment often had a significant effect on the patient’s family and friends who provided practical and psychological support to the patient. Patients felt that it was important to consider how their treatment had a wider impact, rather than just how it affected them individually. For those attending hospital on a daily basis, a reliance on family and friends to get them to and from the hospital was important, both in terms of time off work and additional travel costs:

I mean it’s a case of them leaving work to take me there [hospital] on a morning because it was at like 10 o’clock my appointment and then to pick me up afterwards. [. . .] the Wednesday, I had to wait about an hour for treatment, so my dad was hanging around, and he had to leave me because he was in a meeting at work.

Male, < 65 years, short term, A&E

Some patients were more concerned about the impact their infection had on others than their own well-being and recovery. Although being at home was viewed as better than being an inpatient, this often meant that patients tried to carry on as normal, including maintaining family roles, for the sake of their family. Some expressed a sense of guilt about the impact their treatment had on their children or spouse, with participants talking about the treats or visits that were missed because they were unwell, or tasks not completed, as they were their responsibility. Patients expressed guilt that their infection had resulted in them not being able to care for family members as they usually would do:

I got discharged, I am a carer for my mum and at that point I couldn’t take care of myself so what they did was, they had to put her in a residential home for about 2 or 3 weeks so until I picked myself up.

Male, > 65 years, long term (focus group), district nurse

Patient experiences of outpatient parenteral antimicrobial therapy

Overall, patients appreciate the care that they receive and felt that services were generally well run and of high quality. While acutely ill, most patients preferred to be cared for in hospital but, once stabilised, most but not all patients preferred to recover at home, a finding echoed by other studies. 2,15,85,105 Not all patients are suitable for OPAT and are generally selected on the basis of their ability to cope in the community, and the availability of suitable antimicrobials, and these caveats were echoed by patients. 35 In line with other studies, patients felt that they benefited from being in their own homes,56,85,86 but this also required them to be more active in their own care, and the willingness of patients to do this needs to be assessed. 35,99

Patients identified a range of health-care experiences as important. One important concept related to the type of staff involved in the service and the skills that they need to deliver good quality care. For some, this meant the active involvement of doctors within the service, owing to the perception that doctors were more knowledgeable or skilled and that health-care decisions should be made by doctors rather than by nurses or patients. Others focused on the expertise and experience of nursing staff. A good service was one in which staff were perceived to be competent and highly skilled. Studies have found that a minority of patients lack the confidence to manage effectively in the community, so understanding what constitutes good-quality care is important if OPAT provision is to be widened. 73,85

Another important concept relates to interpersonal aspects of care, such as respect and empathy. Most patients were provided with good written information about OPAT, but oral communication between patients and staff was more variable. Relationships with staff can take time to develop and one way in which these were achieved was through staff having time to talk patients about their infection and treatment. Poor communication could leave patients without the knowledge and confidence needed to be a competent collaborator in their own care and could affect their perceptions of the service. These findings resonate with the conclusions of a recent review by Entwistle et al. 106 that looked at the aspects of health-care delivery that are most important to patients. The map described by Entwistle et al. 106 links ‘health care delivery to what people are enabled (or not) to feel, be or do’ and suggests that both the structure of health care and social dynamics are important to the patient experience. Our findings lend support this conclusion.

What are the benefits of and barriers to different service configurations?

Each of the care pathways was viewed as having its own strengths and weaknesses, and the importance people attached to different attributes seemed to be linked to the age and health of the patient.

A nurse at home model was perceived to be particularly well suited to older patients, those needing longer courses of i.v. treatment and those with more complex care needs. For many, the one-to-one time with the nurse was viewed as a key benefit and contrasted strongly with their inpatient experience, a finding also noted by Dubois and Santos-Eggimann. 99 However, this benefit could be quickly eroded if the nursing team is too large and the continuity of the care relationship is broken.

Hospital attendance was considered to be most suitable for those who were fitter, younger and required once-daily, short courses (under 1 week) of i.v. treatment, a view also held by those attending clinic with long-term infections, although several patients were receiving long-term i.v. via the clinic. The availability of a doctor on duty provided the reassurance some needed ‘in case anything went wrong’, and a qualitative study by Bamford et al. 35 suggests that some patients do not feel confident about being treated by a nurse at home. However, the evidence is mixed. A large-scale Health Technology Assessment (HTA)-funded study of doctor-led versus nurse-led care in a bronchiectasis clinic found that patients were happy with a nurse-led clinic because it provided better continuity of care. 107 It is therefore important to understand what patients value about the different aspects of the service that they receive.

A potential benefit of hospital attendance was its convenience, but this required the use of timed appointments, and when appointments were not kept this affected the level of patient satisfaction, a finding reported elsewhere. 69 Characteristics of the service (e.g. if understaffed) and the staff themselves (e.g. not being briefed about the patient) contributed to dissatisfaction. Hospital attendance was the only care pathway in which transport is a significant issue. Poor public transport links, a reliance on hospital transport and poor car-parking facilities at hospitals were also key attributes that affected the acceptability of hospital attendance and have been noted previously. 69 One solution that was proposed by patients as an alternative to hospital-based services was to invest in ‘local clinic services’, perhaps based in local medical centres.

Self-administration was the model of care least well represented in our sample, and all patients undertaking this were chronically ill patients who received regular courses of i.v. treatment and thus do not reflect the views of those who experience a one-off course of antibiotics, for example for a deep-seated infection. Patients who had no personal experience of SA voiced most concerns about the risks of SA. Although those using the service found it convenient, there were residual concerns about the safety of SA, and one patient did not rule out inpatient care as a favoured alternative. A study of cystic fibrosis patients reported similar results and recommended that refresher courses be given. 88 SA is generally offered only to patients who are physically and cognitively able to manage its complexity. The findings of our study and the conclusions of Pilling and Walley88 suggest that when patients are taught to self-administer, treatment choices should be revisited over time to determine whether or not people’s circumstances have changed.

Questionnaire design

Pilot study

We conducted a pilot study with 30 patients to gather their feedback on the survey.

Sampling

Participants were recruited through six NHS acute hospital trusts, representing both teaching hospitals and DGHs. Both retrospective and prospective recruitment was allowed to provide the best opportunity to recruit to our sampling frame. Two groups of patients were identified: patients with short-term infections (largely SSTIs) and patients with deeper infections which generally require longer treatment courses (e.g. joint infections). The modelling work, however, identified only small differences between the two groups in terms of preferences (see Results). We aimed to ensure representation on the basis of age, sex, SES (based on work status), antibiotic experience and ethnicity.

Data collection

Recruitment took place between September 2014 and May 2015. Eligible patients were approached by a research nurse and details of consented participants were passed to the research team. The questionnaire was delivered to patients face to face using a laptop or iPad. Almost all participants completed the questionnaire in their own homes. A total of 512 people were approached, almost half (n = 254) of whom consented, and data were collected from 202 participants (20 participants could not be contacted post recruitment, 15 were too ill to participate, 17 refused post consent). Participants had experience of at least one model of care (Table 11). After asking participants about their preferences we asked about their past experiences with OPAT. A total of 94% (n = 191) of participants reported they were quite satisfied or very satisfied with the service they received.

Pilot data

Frequency data were calculated using Microsoft Excel^® 2013 (Microsoft Corporation, Redmond, WA, USA) and Statistical Product and Service Solutions (SPSS; SPSS Inc., Chicago, IL, USA) for analysis. Qualitative feedback in form of written notes and audio-recordings were used to inform revisions to the survey and the DCE.

Main study

Our analysis made use of an advanced discrete choice model, which is a mathematical structure used to explain the influence of explanatory variables (e.g. type of treatment, frequency of treatment, risk) on the choices that patients make between the different treatment options presented to them. The model is based on the notion of utility maximisation, with respondents choosing the option that gives them the greatest utility/smallest disutility. For a detailed overview of choice modelling techniques, see Train. 123

The models explain the preference of a patient in a given choice task (i.e. which of the three models of care is chosen) by estimating the sensitivity that the patient has to the different characteristics of each model. With the expectation of major differences across patients in these sensitivities (e.g. one patient cares more about risk than another) and hence in the resulting preferences for a given model of care in a given choice scenario, our work incorporates three levels of heterogeneity in preferences across patients, namely:

1. differences that can be linked to the sociodemographic and infection characteristics of the participant (age, sex, race, education, employment status, past infections and whether they were short-term or long-term patients)

2. idiosyncratic differences across participants in their preferences that cannot be linked to the characteristics of the participant

3. differences in preferences that can be linked to underlying attitudes, namely attitudes towards hospitals and towards health-care responsibility, where we make use of the methods described by (among others) Abou-Zeid and Ben-Akiva,124 Vij and Walker125 and, with an example of a health application, Klojgaard and Hess. 120

Model estimation produces estimates of the sensitivities/preferences of respondents for the different types of treatment as well as treatment characteristics and patient characteristics (e.g. sociodemographics) in explaining these sensitivities and the underlying patient attitudes. These sensitivities in turn drive the preferences that respondents express between the different services presented in a given choice task. For example, the models produce parameters for risk and for waiting time, and their relative values allow us to understand the relative importance of these two characteristics in explaining the choice between the different models of care. These relative sensitivities vary across individual patients. The actual parameter values are determined in a numerical process that produces estimates that allow the model to best explain the choices observed in the data. Full details on the model structure are given in Appendix 3.

Random utility models such as those used in our work recognise the inability of analysts to capture fully the utility of alternatives and thus acknowledge the presence of a remaining random component of utility. Although initial insights into overall preferences can be obtained by direct questioning with regard to which service a given patient prefers, such approaches fail to recognise that the actual preferences will be influenced by the specific characteristics of the different service delivery methods. In particular, they will be the result of trade-offs, whereby a patient selects the option that provides the best overall combination of characteristics, with the good performance of some characteristics compensating for the poor performance of others.

We chose not to segment the analysis by infection type (short term vs. long term) but instead included this variable as a respondent covariate in the modelling, testing for impacts on the preference for treatment types as well as the valuation of treatment characteristics. This provides greater sample sizes and acknowledges that time to infection resolution is on a continuum. In addition, the modelling work showed only small differences between the two groups in terms of their preferences, and these related solely to appointment times, training and seeing and speaking to someone that the patient knows, and not to the type of treatment or key characteristics such as risk.

Full details on the model structure are given in Appendix 3.

Attitudinal data were analysed using principal component analysis with Varimax with Kaiser normalisation to identify underlying structure within the data (SPSS version 20). Factor loadings > 0.7 were accepted and a scree plot was used to determine the number of factors to include in the analysis. 126 The principal component analysis indicated that a two-factor solution provided a good fit to the data and accounted for 32% of the variance in scores: attitude towards hospitals and attitude towards health care being a doctor’s responsibility. In both cases a higher value means stronger disagreement. These factors were then used as latent variables in the analysis, allowing us to understand the role of attitudes in driving preferences (see Appendix 3 for further information).

Pilot data

Minor revisions were made to the survey questions as a result of the qualitative feedback. In addition, further instructions were provided for respondents about the structure of the DCE, as it was found that participants did not understand that the options would change between each choice set. Two attitudinal items were removed, as all participants agreed with these statements so they did not distinguish between patients. Missing data were minimal. The data from the survey were analysed using a multinomial logit model and this yielded significant models whereby the sign and order of the coefficients on service levels were intuitive and in the expected order (i.e. levels that were worse had higher negative model coefficients).

Main study

This section gives a high level overview of the estimation results, with full details provided in Appendix 3.

We first look at the two underlying attitudes retrieved from the data, which are an attitude towards hospitals and an attitude towards health-care responsibility.

• Participants who present a positive attitude towards hospitals are more likely to be female and non-white, and to live alone, and the impact is stronger the more of these characteristics they have. As this attitude becomes more positive, we also see an increase in the preference for in-hospital treatment in the DCE scenarios.

• Participants who see health care as the responsibility of the doctor are likely to be > 65 years old and non-white. They are also slightly less likely to have a university degree. As this attitude becomes stronger, these participants have a much greater preference for the attendance at hospital or nurse at home models of care compared with SA in the DCE scenarios.

We next look at the headline results in terms of utility scores for the different treatment types and characteristics. The models explain the preference for a given model of care in a given choice scenario as a function of the sensitivities of the patient and the characteristics of the model of care. Each of these characteristics has an influence on the utility of the model of care, and the model of care with the highest utility obtains the highest probability of being chosen. Here, we work with the mean sensitivities at the sample average in terms of sociodemographic characteristics (which have an influence on preferences both directly and through the underlying attitudes) and the random variation in preferences (both directly and through the underlying attitudes).

The results in terms of the impact on utilities (and hence on preferences) are presented in Figure 2 where, for each attribute, one of the levels is normalised to zero for identification. We observe first of all that treatment type matters the most of any characteristic, followed by treatment frequency and risk. Looking in detail at the findings, we note that:

• For treatment type, there is a strong overall preference for the nurse at home model, followed by hospital treatment.

• Respondents prefer a single treatment per day to two treatments, continuous treatment and three treatments per day.

• Having an appointment time is preferred to not having an appointment time.

• A SN is preferred to a doctor or a GN.

• A half-day of training for SA is preferred to a full day of training.

• Patients prefer seeing someone they know.

• Patients prefer speaking to someone they know.

• Patients prefer nurse administered aftercare in hospital to GP aftercare, telephone aftercare and no aftercare.

• For risk, we obtain clear evidence of non-linearity in sensitivities, leading us away from a continuous treatment of risk (which would have implied, e.g., that a 1 in 25 risk would have been valued 2.5 times as highly as a 1 in 10 risk). With only three levels of risk in the data, we then estimated separate valuations for 1 in 25 and 1 in 10, keeping 1 in 6 as the base. Moving away from a continuous specification was also justified by our analysis showing a lack of differences between valuations for 1 in 6 and in 10 in several groups.

FIGURE 2.

Mean preferences (utility scores) at sample average.

With the different sociodemographic interactions (discussed in Appendix 3) used in the model (e.g. how short-term and long-term patients react differently to appointment times), a total of 48 different sociodemographic groups exist in our model, and we note strong variations in the mean preferences relative to SA across these groups, as highlighted in Figure 3. What these data show is that the sociodemographic variables we measured [age, sex, race, education, employment status, number of past infections, type of current infection (long term vs. short term)] do affect the strength of preferences for a particular service, but these effects are modest. There is an overall preference for nurse at home treatment ahead of hospital treatment and ahead of SA, where this ordering applies to 39 out of 48 groups at the mean sensitivities (i.e. before allowing for random variations in preferences). In addition, nurse at home treatment is always preferred to SA at the mean preferences, and only white males < 50 years of age and living alone marginally prefer hospital treatment to nurse at home treatment.

FIGURE 3.

Mean utilities for models of care (relative to SA) in 48 different sociodemographic groups.

Some groups of patients prefer SA to hospital treatment, namely white respondents aged between 50 and 65 years of age who do not live alone (male or female), or are female and live alone, or are male, live alone and have a university degree. Importantly, the gaps between preferences for the different treatment types differ across groups.

Of course, owing to random heterogeneity, in sensitivities across individual patients, some probability of reversal of preferences exists. Looking at the largest group (age > 65 years, male, white, not living alone, no university degree), the mean utilities for hospital treatment and nurse at home (with SA as the base) are:

• hospital treatment utility score: 1.11

• nurse at home utility score: 4.50.

This would mean that the nurse at home model has the highest probability of being chosen, ahead of hospital treatment and SA. However, the large standard deviations caused by random variations in preferences give the following 95% confidence intervals (CIs; as used here referring to 95% of values falling within these intervals, and not suggesting an insignificant estimate, but random variation in preferences):

• HO treatment: 95% CI –12.17 to 14.40

• nurse at home treatment: 95% CI –7.48 to 16.48.

We now see that there is clearly a possibility, even with a small probability, of SA being the most preferred option for some patients. An important component in our work is the incorporation of underlying attitudes. As described earlier, participants who present a positive attitude towards hospitals are more likely to choose hospital treatment, whereas those who see health care as the responsibility of the doctor have a much greater preference for attendance at hospital or nurse at home models of care, than for SA. Here, we see that the attitude towards responsibility of health care accounts for 65% of the heterogeneity for the preference for hospital treatment, and 88% for the preference for nurse at home treatment. The attitude towards hospitals, however, accounts for only 6.9% of the heterogeneity for the preference for hospital treatment, and 4.3% for the preference for nurse at home treatment.

For the remaining attributes, we now look only at those in which differences exist across sociodemographic segments (which is not the case for aftercare and frequency of treatment), given that the overall results are included in Figure 2.

Figure 4 shows a very large difference between long-term and short-term patients in their preference for having an appointment time (with no appointment time as the base), with the latter having a much stronger preference for an appointment time.

FIGURE 4.

Appointment time (utility scores; no appointment time as base).

Figure 5 shows that, although patients not living alone have a strong preference for SNs giving the treatment ahead of GNs and doctors, for those living alone, there is a preference for doctors and SNs (where the two are not significantly different) over GNs.

FIGURE 5.

Who gives the treatment? (Utility scores; doctor as base.)

More differences arise across groups in terms of the preferences for a half-day of training for self-administration (with 1 full day as the base). Figure 6 shows that, although long-term patients aged < 50 years who do not live alone have a very strong preference for a half-day of training, short-term patients > 50 years of age and living alone have a strong preference for 1 full day of training. Other groups are in between these two extremes.

FIGURE 6.

Preference for half-day training (utility score; 1 full day as base).

For communication in person, all patient segments prefer seeing someone they know, but Figure 7 shows that this is more important to younger patients and long-term patients. More important differences arise with regard to communication over the telephone, and Figure 8 suggests that, in particular, older patients living alone have a preference for speaking with someone who they do not know.

FIGURE 7.

Communication in person (utility scores; base = unknown person).

FIGURE 8.

Communication over telephone (utility scores; base = unknown).

Finally, Figure 9 highlights the sensitivity to risk (e.g. of an adverse event). We see that patients aged < 65 years and not working fail to distinguish between the two higher levels of risk, where the difference in sensitivities between these two levels is stronger for older patients and for those in employment. The difference between the two lowest levels of risk (1 in 10 and 1 in 25) is not significant for those aged > 65 years and in employment, so the positive shift in Figure 9 can be ignored. We chose not to model risk as a continuous variable, as it appeared to be non-linear. Thus, we were not able to identify the marginal rates of substitution between this and other attribute levels.

FIGURE 9.

Risk (utility scores; 1 in 25 as base).

Markov models

Model horizon, structure and assumptions

The model structure (Figure 10) was informed by a rapid review of published infection decision models and by discussions with patients and clinicians. Cohort Markov models were constructed with daily cycles until the cohort was healed (or switched to oral antimicrobials) or died. The patient cohort receive one of the OPAT services and every cycle were either healed (or switched to oral treatment), experienced a severe adverse reaction to the treatment (e.g. anaphylactic shock), contracted Clostridium difficile infection (CDI) or contracted a severe secondary-line infection (e.g. Staphylococcus aureus). If patients experienced any of these negative health events then they were subject to a mortality risk. In addition to the more severe negative events, those who were not healed were exposed to a daily risk of a mild adverse event (such as rash, nausea, vomiting, dizziness, fever and minor line events). These events and the associated costs are incorporated within the ‘not healed’ health state and are not associated with a quality-of-life decrement as they are both mild and transient in nature. However, a sensitivity analysis did employ a utility decrement for these events. It was assumed that they do not increase time to heal.

FIGURE 10.

Markov model structure.

All patients began in the ‘not healed’ state and moved between the health states according to pre-specified probabilities. The cohort of patients flow through the model pathway accruing costs and quality-of-life penalties or ‘decrements’ until they are healed or die or until the model has ended. A sensitivity analysis explored the impact of some patients ‘relapsing’ and needing to receive IVA again after they had been switched to oral antimicrobials. Even though the costs and benefits of IVA treatments may register after the infection period (e.g. those who die as a result of an adverse event will lose future QALYs), we decided to limit the model horizon to the infection period. This was mainly due to the expectation that very few deaths would occur and, hence, the additional complexity and computation time required to model a longer time horizon would not be worthwhile. The time horizons chosen for the short-term (3 months) and long-term (12 months) models reflected the expectation that a vast majority of infections would have resolved within these periods. It was assumed that the period of time spent on oral antimicrobials and their effectiveness was the same across service models and unrelated to their IVA treatment. Given this, it was assumed there were no differential resource use or quality-of-life effects of oral treatment and that modelling these outcomes was not warranted.

The transition probabilities, costs and utility parameter values required for the models were taken from a number of sources including a systematic review134 and expert clinical opinion. In addition, hospital records of a group of patients (n = 503) who had recently received OPAT care were gathered and used to generate parameter values (Table 14 provides sample characteristics).

TABLE 14 - Hospital record data: sample characteristics

Characteristic	Short term (N = 223)	Long term (N = 280)
Female, n (%)	90 (40.36)	133 (47.84)
Age (years), mean (SD); range	52.85 (16.36); 18–89	52.59 (18.25); 18–94
Ethnicity, n (%)
White	195 (87.44)	257 (91.79)
Asian	20 (8.97)	16 (5.71)
Black	2 (0.9)	3 (1.07)
Mixed ethnicity	6 (2.69)	4 (1.43)
Type of service received, n (%)
HO	154 (69.37)	63 (22.58)
GN	18 (8.11)	68 (24.37)
SN	36 (16.22)	34 (12.19)
SA	0 (0.00)	68 (24.37)
Combination	14 (6.31)	46 (16.49)
Type of infection, n (%)
Cellulitis/SSTI	196 (87.89)	44 (15.71)
Cystic fibrosis	0 (0.00)	44 (15.71)
Respiratory	8 (3.59)	37 (13.21)
Bone and joint	0 (0.00)	73 (26.07)
Cardiovascular	1 (0.45)	11 (3.93)
Urinary tract	3 (1.35)	7 (2.5)
Intra-abdominal	2 (0.9)	7 (2.5)
Other	13 (5.83)	57 (20.36)
Number with infection in past 6 months	55 (24.66)	154 (55.0)
Number with complex infection	0 (0.00)	75 (26.79)
C-reactive protein level	87.28 (93.01)	80.46 (99.52)
White blood cell count	10.11 (3.55)	10.84 (5.12)
Days to heal, mean (SD); range	5.77 (5.35);1–55	28.54 (20.64); 4–119
Side effects, n (%)
Rash	3 (1.35)	5 (1.79)
Nausea/vomiting	3 (1.35)	12 (4.29)
Dizziness	1 (0.45)	2 (0.71)
Fever	2 (0.90)	1 (0.36)
Diarrhoea	5 (2.24)	4 (1.43)
Infection of i.v. access device	1 (0.45)	2 (0.71)
Blocked i.v. device	9 (4.04)	13 (4.64)
Other	23 (10.31)	52 (18.57)

Time to heal was calculated by type of infection and after controlling for a number of factors including markers of infection severity (e.g. C-reactive protein levels and white cell blood count) and patient characteristics (e.g. whether or not they had experienced previous infections in the past 6 months). We applied a negative binomial regression model, as the variable of interest (days to heal) was a non-negative count variable with a wide dispersion (there were several outlier patients with relatively long heal times). We used the margins command in Stata (StataCorp LP, College Station, TX, USA) to produce predictions (and associated standard errors) holding all other covariates at their means. A number of the severity markers were entered into the model as categories: white blood cells as ‘normal’, ‘mild’, ‘severe’ or ‘not known’; and C-reactive protein levels as 0–40 mg/l (normal or mild), 40–200 mg/l (active) and > 200 mg/l (severe).

The final model for the adjustments was:

⁽¹⁾ Y ^ = α + β CR 0 … CR 2 X CR 0 … CR 2 + β WB 0 … 2 X WB 0 … 2 + β S 0 … 4 X S 0 … 4 + β C X C + β Pr ⁡ X Pr ⁡ ,

where Ŷ = time to heal in days; β_{CR0. . .2} = dummies for C-reactive protein level groups; β_{WB0. . .2} = dummies for white blood cell groups; β_S0. . .4 = dummies for OPAT service type; β_C = complex infection and β_Pr = previous infection in past 6 months.

The adjusted time-to-heal figures for the short and long term are included in Table 15 along with their standard deviations. The cost of resource use was calculated on a daily basis for all services combined using the same variables and the β on days to heal being the adjusted value employed in the model.

TABLE 15 - Parameter values: effectiveness and risks

Parameter	Mean	SD	Distribution	Source
Short-term model effectiveness (days to heal)
HO	4.73	0.24	Gamma	Adjusted hospital record data
GN	7.36	1.00	Gamma	Adjusted hospital record data
SN	6.33	0.65	Gamma	Adjusted hospital record data
Long-term model effectiveness (days to heal)
HO	27.21	2.30	Gamma	Adjusted hospital record data
GN	31.16	2.65	Gamma	Adjusted hospital record data
SN	25.46	3.02	Gamma	Adjusted hospital record data
SA	28.20	2.52	Gamma	Adjusted hospital record data
Daily risk of an anaphylactic shock	0.00005	0.00099	Beta	Matthews et al.29
Anaphylactic shock
Anaphylactic shock mortality risk
HO	0.067	0.04480	Beta	Hopf et al.143
GN	0.13	0.04480	Beta	Assumed double HO risk
SN	0.13	0.04480	Beta	Assumed double HO risk
SA	0.27	0.04480	Beta	Assumed double GN/SN risk
Risk of CDI
HO	0.000105	0.000023	Gamma	Hourly risk × 4100
GN	0.0000087	0.000004	Gamma	Assumed third HO risk and 1-hour contact
SN	0.0000087	0.000004	Gamma	Assumed third HO risk and 1-hour contact
Self-administered OPAT	0.00	N/A	Fixed	Assumed no risk
Time to heal from CDI (days)	16	0.40	Log normal	Forster et al.144
Daily CDI mortality risk	0.00040	0.00004	Beta	Wiegand et al.145
Risk of a line infection that leads to a S. aureus infection	0.00052	0.01580	Gamma	Barr et al.40
Time to heal from S. aureus	17	0.32	Log normal	Forster et al.144
Daily S. aureus mortality risk	0.0092	0.03346	Beta	Thwaites et al.146
Short-term model: mild adverse events
HO	0.020	0.006	Gamma	Hospital record data
GN	0.046	0.032	Gamma	Hospital record data
SN	0.054	0.025	Gamma	Hospital record data
Long-term model: mild adverse events
HO	0.008	0.003	Gamma	Hospital record data
GN	0.007	0.003	Gamma	Hospital record data
SN	0.009	0.004	Gamma	Hospital record data
SA	0.033	0.009	Gamma	Hospital record data
Probability of infection relapse
HO	0.0	N/A	Beta	Assumed 5% lower than SN
GN	0.0	N/A	Beta	Assumed same as SN
SN	0.0	N/A	Beta	Lillie et al.147
SA	0.0	N/A	Beta	Assumed same as SN

Parameter values

Simulation models

Simulation modelling techniques such as discrete event simulation (DES) afford the opportunity to ‘replicate’ an OPAT service in a community and estimate the number of resources (nurses and rooms/beds) required to cover a specific population. DES is an individual-level modelling technique that originated in the operational research field but is increasingly being used in a health context. The core concepts of a DES model are ‘events’, ‘entities’, ‘attributes’, ‘activities’, ‘queues’ and ‘resources’. The ‘entities’ are a representation of individuals or patients who move through different ‘activities’ in a pre-defined pathway. The entities possess certain ‘attributes’ (or history) such as whether or not they have experienced a particular adverse event which can be modified when performing certain ‘activities’ in combination with different ‘resources’. Those modifications influence the entities’ future movements through the pre-defined pathway. ‘Queues’ can be formed when entities are about to pass through the different activities and this is determined by the number of entities, number of resources and time taken to undertake the activity. In contrast to a Markov model, time is measured as a continuum but evaluated when an events occurs. In the context of an OPAT service, the DES model not only allows patient history to be considered (via attributes), but also the operational aspects of an OPAT service as nurses and/or beds can be represented by ‘resources’ and home or outpatient visits as ‘activities’.

Simulation models have been used previously in health to optimise the use of resources and inform decision-making. The Mayo Clinic’s Centre for Health Care Delivery (Rochester MN, USA) used a DES model to predict the minimum number of beds needed to meet patient demand and the clinic’s quality standard of care. 159 Elsewhere, Troy and Rosenberg160 estimated the required number of beds for an intensive care unit in Canada, and Cipriano et al. 161 evaluated different strategies to reduce waiting time in a total joint replacement unit in Canada. The aim of the current analysis was to replicate the operation of OPAT services to estimate the amount of resources required to run OPAT services in England and Wales taking into account cost-effectiveness and size and type (e.g. rural vs. urban) of population. As the DES technique is an individual-level analysis, instead of a cohort of patients, individuals are followed through a defined pathway in the model. However, the population, model structure and evaluated services are otherwise the same as that used for the Markov modelling. Any divergence in methods is described below.

Model horizon, structure, assumptions and perspective

As with the Markov model, patients enter the simulation after being referred to the OPAT service: HO, GN, SN or SA. Although the simulation model also used daily cycles, it ran for the same period for both short- and long-term infections (12 months). This was longer than the short-term Markov model as the simulation model may run for longer as it is better at capturing the real length of health events. The first step of the simulation assigns patients to one of the four services to commence treatment. The initial treatment for the patients on the GN service would require a home visit of a GN, whereas for the SN and SA services a home visit by a SN to commence treatment or training was necessary. The HO service required a private room in the clinic of the hospital for the service to be delivered.

Although GN and SN service patients ‘wait’ for the nurse visit to deliver their IVA, HO patients wait for an available bed and SA patients administer their own IVA. The duration of the wait is dependent on the availability of resources (nurses and rooms). This availability was given by the number of nurses available, the time required to administer the i.v. service (including travel time) and the working hours (clinic and nurses). As such, if the availability of nurses is limited owing to lack of personnel or high number of patients to be treated, the size of the ‘wait’ will increase. We included a sensitivity analysis in which departments subcontracted a private nurse service to cover any unmet demand (at higher cost) to reduce or prevent delays in care.

As the resource and safety elements are crucial to the functioning and viability of the services, these components were explored further by adding a delayed treatment state to the model. Patients who were not able to be seen within 24 hours for their IVA travelled to this state, as their missed treatment would result in an outpatient visit and a risk of admission to hospital owing to a worsening of their infection. Modelling this eventuality allows us to determine the minimum number of resources needed to avoid delayed cases that can lead to further costs and hospital admissions. Figure 11 shows the structure of the model.

FIGURE 11.

Simulation model structure. Screenshot from Simul8^© [Simul8 Corporation (Boston, MA, USA) www.simul8.com (accessed 8 November 2016)].

Two different perspectives were used to estimate the costs of the services. The first refers to the HPSS in line with the Markov model analysis. These are based on hourly costs and, therefore, nurse visits (general or specialist) are costed based on the time they spend with the patient. If no patient is seen, the cost of this resource is zero. There is also no limit to the resources available. The second costing perspective is that of the commissioner of services. Here, the cost of a nurse is given by their annual salary irrespective of the number of patients seen in any given time frame. Thus, hiring an additional nurse will represent an increase in the costs equivalent to their annual salary but increase the resources available to treat patients. The rest of the costs, such as hospital beds, outpatient service, etc., are costed from the HPSS.

Value of information analysis

Given that the PSA characterises the level of uncertainty in the analysis, it is possible to cost this formally based on the probability of making a ‘wrong’ decision regarding service provision choice using the value of information framework. 142 We estimated the expected value of perfect information (EVPI) for the population based on the Monte Carlo simulated NMB, identifying in each case the net benefit loss by basing the decision on the average net benefit rather than that based on perfect knowledge (each simulation). This represents the individual value of perfect information and is multiplied by the population of interest over the relevant decision period to gain the population EVPI. The higher the population EVPI, the greater the cost of uncertainty and the greater the value of additional research. Should the EVPI exceed the expected research cost for reducing uncertainty then investment in research is recommended. 163

We assumed that the population that could benefit from the services to be 21,000 patients with short-term infections. This figure is based on the assumption that 70,000 patients per year are hospitalised with SSTIs164 and approximately one-third of these could be eligible for IVA. 137 Although not all of these would be suitable for OPAT (e.g. owing to impaired functioning), this figure is likely to be an underestimate, as it does not include other types of infection such as respiratory and urinary tract infections. To explore this issue we have run the analysis assuming 100% could receive OPAT (n = 21,000) and a scenario in which only 50% are eligible for OPAT (n = 10,500). The long-term infection population group was estimated based on the assumption of 1% prevalence of infections at elective orthopaedic surgical sites. 130 This translates to 2000 joint-related infections in addition to 5000 osteomyelitis-related infections. The total of 7000 was scaled up to include other long-term infections such as endocarditis using the proportions from a UK cohort study. 29

The expected value of partial perfect information (EVPPI) was calculated using a recently developed non-parametric regression method tool. 165 Using the simulated parameter values and associated NMB estimates from the Monte Carlo simulation, the EVPPI represents the monetary value of uncertainty emanating from each of the parameters used. This is particularly useful in identifying which parameters are driving the EVPI and which might be the target of efforts to reduce uncertainty in the decision and future research investments, for example whether it relates to effectiveness, risks or quality-of-life estimates. The expected value of sample information (EVSI) represents the expected value of increased research sample size. EVSI helps determine the optimal sample size for a study based on the marginal benefit from an additional study participant compared with the marginal cost of enrolling them. The optimal point occurs when the marginal benefit is equal to the marginal cost. 12 This analysis was conducted if there was sufficient value of additional research and information was required on the research design.

Markov model: short-term infections

Sensitivity analyses: probabilistic

Table 21 includes the probabilistic mean estimates from the Monte Carlo simulation. These, and the cost-effectiveness plane which plots the simulated ICERs, corroborate the deterministic results (Figures 12 and 13). These indicate that, even with uncertainty introduced into the results, GN is more cost-effective than HO, and SN is more cost-effective than both. The CEAC (Figure 14) indicates that SN would be the cost-effective option for all values of willingness-to-pay thresholds and at £20,000 has a 78% chance of being cost-effective. The INMB distribution plot shows that, compared with the optimal strategy (SN), there is some overlap in the simulated INMB results with GN, but very few of the HO simulations exceed 0.

TABLE 21 - Base-case probabilistic cost-effectiveness: short-term infections

Mean	Service			Comparison	ICER	INMB	Interpretation
	HO	GN	SN
Costs (£)	998	851	778	HO vs. GN	£32,634	–£57	GN cost-effective
QALYs	0.174	0.169	0.171	HO vs. SN	£80,351	–£166	SN cost-effective
				GN vs. SN	SN dominates	–£109	SN cost-effective

FIGURE 12.

Cost-effectiveness plane: short-term infections.

FIGURE 13.

Incremental net monetary benefit distributions vs. SN: short-term infections.

FIGURE 14.

CEAC: short-term infections.

Value of information

The population EVPI is shown in Figure 15. The EVPI is low (£302,353, where λ = £20,000) even with the assumption that 100% of patients requiring IVA would be eligible for OPAT. This reflects the low decision uncertainty in short-term infections given that, in the base-case model, SN clearly offers the best value for money. The EVPPI for individual parameters shown in Figure 16 suggest that the greatest proportion of EVPI is driven by the respective effectiveness parameters (days to heal) in the GN and SN services. Other risks and utility values did register a positive value but their EVPI was small. Over the course of 1 year (and even up to 5 years) it is unlikely that the cost of decision uncertainty characterised here would warrant significant further research investment in the form of a clinical trial. Given the low EVPI values, EVSI was not calculated.

FIGURE 15.

Expected value of perfect information: short-term infections.

FIGURE 16.

Partial EVPI for single parameters: short-term infection. A. shock, anaphylactic shock; P, probability.

Markov model: long-term infections

Sensitivity analyses: probabilistic

Table 25 includes the probabilistic mean estimates from the Monte Carlo simulation. This mirrors the deterministic values and decision, with SA being the most cost-effective service. Figures 17 and 18 are the cost-effectiveness planes for all services versus HO and for SA versus SN, respectively. Figure 19 is the CEAC for long-term infections and indicates slightly more decisional uncertainty than the short-term infection model. Where the ceiling ratio is £20,000, SA has a 70% chance of being cost-effective versus a 30% chance for SN. At £30,000 this becomes 62% and 37%, respectively, and when the ceiling ratio increases to £61,000, SN becomes more likely to be cost-effective. GN and HO do not become the optimal strategy at any point across the ceiling ratio range used (up to £100,000). This is further illustrated by the distribution plot of INMB in Figure 20 where only the tail of the distributions for GN and HM cross the INMB = 0 line (vs. SA).

TABLE 25 - Long term: base-case probabilistic cost-effectiveness

Mean	Service				Comparison	ICER	INMB	Interpretation
	HO	GN	SN	SA
Costs (£)	4635	3081	2564	2112	HO vs. GN	£130,359	–£1315	GN cost-effective
QALYs	0.646	0.634	0.650	0.641	HO vs. SN	SN dominates	–£2152	SN cost-effective
					HO vs. SA	£482,422	–£2419	SA cost-effective
					GN vs. SN	SN dominates	–£836	SN cost-effective
					GN vs. SA	SA dominates	–£1103	SA cost-effective
					SN vs. SA	£48,845	–£267	SA cost-effective

FIGURE 17.

Cost-effectiveness plane vs. HO: long-term infections.

FIGURE 18.

Cost-effectiveness plane vs. SN: long-term infections.

FIGURE 19.

Cost-effectiveness acceptability curve: long-term infections.

FIGURE 20.

Incremental net monetary benefit distributions vs. SA: long-term infections.

Value of information

The population EVPI is illustrated across the range of ceiling ratios in Figure 21. Over 1 year and where λ = £20,000, the EVPI is £1,078,204. Figure 22 indicates that a large majority of this value is associated with the time to heal values from the SN and SA parameters. There is little value in reducing the uncertainty around the GN effectiveness parameter. The only other factor to register a positive EVPI figure was the probability of S. aureus infection. The value here is higher than that in the short term but still relatively low and, given this, EVSI was not calculated.

FIGURE 21.

Expected value of perfect information: long-term infections.

FIGURE 22.

Partial EVPI for single parameters: long-term infections.

Health and Personal Social Services perspective: comparison with Markov

Tables 26 and 27 include the expected costs and benefits of the OPAT services generated by the simulation model for short- and long-term infections, respectively. These allow a comparison with the Markov model results which are largely in agreement. The decision in the short- and long-term infection groups is unchanged, with SN and SA, respectively, being clearly optimal.

The results of the simulation corroborate what the Markov model estimated, providing certainty to the model results. The CEAC (Figure 23) shows that the probability of SN and SA being cost-effective is > 65%, which is similar to that estimated by the Markov model (> 70%).

FIGURE 23.

Cost-effectiveness acceptability curve: (a) short- and (b) long-term infection simulation model.

Service combinations (Table 28) introduce combined services into the cost-effectiveness analyses. None of the combined services was cost-effective.

Commissioning perspective

There was a clear, positive relationship between the number of delays in treatment experienced, costs and time to heal (Figure 24). This is corroborated by Figure 25, which plots the relationship between costs, delays in patient IVA administrations and NMB when different levels of resources (nurse numbers) are available in the long-term SN service. This is presented in two scenarios: (1) an urban setting in which we might expect nurse visit times to be shorter owing to reduced travel times; and (2) a rural setting in which patients may be more dispersed and nurse travel and visit times are increased. For a given patient population (in this case, n = 304), there appears to be a non-trivial number of treatment delays. In the urban setting adding further nurses reduces the number of delays significantly without adding significantly to the costs. However, NMB is relatively unchanged. In the rural setting the delays are much higher and the benefit (reduced costs and increased NMB) of additional nurses is marked.

FIGURE 24.

Impact of delays on time to heal and costs.

FIGURE 25.

Relationship between delays, resources and NMB. (a) Urban setting and (b) rural setting.

Tables 29 and 30 include the cost-effectiveness results from the commissioning perspective. Here, the costs of resource use are not calculated on an hourly basis but as sunken costs relating to the employment of nurses on a salaried annual basis. In short-term infections, fewer nurses lead to more treatment delays, but this does not translate into greater costs and reduced QALYs. This is the case when the patient population is 304 and 150. In long-term infections the nursing resources do have an impact on costs and QALYs when nurses drop from 4 to 2. Furthermore, when the visits require 2 hours rather than 1 hour, there is a noticeable drop in patient QALYs as a result of treatment delays.

An estimation of the number of patients that can be seen per nurse was made assuming a long-term infection in an urban setting (1 hour’s treatment including travel time) using the SN service. As shown in Figure 26, 92.50 patients can be treated per nurse in a year before reaching saturation. At this point costs will start to increase, while QALYs will decrease.

FIGURE 26.

Number of patients per nurses before saturation.

Patient involvement

The research team met three times with members of the PAG during the development of both the Markov and simulation models. At these meetings we presented the graphics for the preliminary models to the group and provided an oral overview of them and what we were aiming to do. Their input and that of the clinical experts informed the development of the patient pathway described in the models. The patients also provided key insights into the important health events that might occur, on the health-care services that might be used and into the impact on their quality of life that infections had.

Limitations

Decision-analytic modelling is useful when the information required to make a decision is not available or requires synthesis; it can also help to inform research priorities. Notwithstanding this, the usefulness of modelling is a function of the quality of data available to populate the models that are constructed. In this economic evaluation we were constrained to some extent by the available data. There was a paucity of useful comparative UK data on the effectiveness and safety of the OPAT services. Furthermore, the concept of effectiveness in OPAT services is attenuated, as, quite often, fixed treatment courses are prescribed. We used a hospital record data set to derive our measure of ‘effectiveness’ (time to heal). The data set permitted adjustment for patient heterogeneity between services and did indicate differences in time to heal (or switch to oral antimicrobials). The differential between services on this factor may reflect effectiveness or the impact of adverse events and compliance, which may indirectly affect treatment time. Although there are non-trivial chances of experiencing mild adverse events in the course of IVA receipt, these are often self-limiting, transient and have minimal cost implications or impact on patient quality of life. The more severe events that are possible (e.g. severe line infection, anaphylactic shock or CDI) are relatively rare and, in the case of CDI, becoming rarer; thus, their impact at a cohort level is relatively minor. Given these issues, it is debatable whether or not better-quality data would have influenced the results to a great degree.

Antimicrobial stewardship is currently a key concern but we chose not to model antimicrobial resistance. We believe that the differential rate of resistance between the service models would have been negligible and did not warrant the additional modelling resources required or additional layer of complexity in the models. We also acknowledge that other variations on OPAT service models are available, for example delivery by a GP at the local clinic. However, it was not practicable to evaluate all of these. Outcomes data for all possible service configurations would be scarce and we chose to focus on the most commonly used in current practice.

Further research

The value of additional research was formally assessed using the outputs from simulations of the Markov decision models. These values (EVPI) represent in monetary terms the level of uncertainty in the estimates of cost-effectiveness; if these exceed costs we might expect to incur undertaking activities to reduce the uncertainty (e.g. conducting a large RCT) then we may proceed to invest in further research. If they are lower than the research investment then we should make the decision with the current available evidence. In the short-term model the EVPI was relatively small, given the low levels of uncertainty around the decision (SN had a 78% chance of being optimal). The EVPI figure over 1 year is £302,353 and, assuming that a clinical trial to reduce uncertainty would cost in the order of £1M–£3M, is outweighed by the research costs. Given this, additional research of this nature does not appear to be warranted.

The EVPI for long-term infections was higher (£1,078,204), mainly because there was greater uncertainty (present in the decision between SN and SA), and approached the cost of a trial. If we consider that the decision problem may be over the next 5 years then the EVPI increases significantly (although not sufficiently in the short-term group) to a value that exceeds the research costs. However, in reality, further trial-based research may take at least a further 4 or 5 years to report. By this time it is debatable whether or not the research question would still be relevant, as new treatments and modalities become available or current practice is challenged. 167 To illustrate this, the Oral Versus Intravenous Antibiotics for bone and joint infection (OVIVA) trial currently being conducted in Oxford is comparing IVA with oral antimicrobials in bone and joint infections. Should this research find that oral treatments are a suitable alternative to IVA, research investment in establishing evidence for OPAT services in this group will potentially be obsolete.

Should future trials of OPAT be considered, it is likely that they will be based on equivalence and safety. However, it is debatable whether or not the sample sizes sufficient to capture risks of the more severe events would be achievable. This is further complicated by the fact that departments often need to offer more than one type of service because some OPAT services are not suitable for all. Although SA is the most cost-effective option for long-term infections, it would not be possible to provide only this service because it would not be appropriate for those who were not able to self-administer (for cognitive or functional reasons). Furthermore, the expert panel also appeared to agree that a clinical trial may not be the most useful avenue for further research but that the establishment and analysis of large cohort data may be more fruitful. We are aware that BSAC are currently undertaking such an initiative but that, to date, the results have been mixed with variable uptake. Given the results and opinion of the consensus panel, the value of information analysis did not extend to include the EVSI that could inform trial design.

In terms of future methodological pursuits, the current research has highlighted the disconnect between macro- and micro-level decision-making. The real world usefulness of cohort-model-based estimates of cost-effectiveness conducted from the perspective of the health system is questionable when local-level factors are brought to bear. The conclusion we might make from the short-term infection modelling is that SN should be adopted wholesale. However, if combinations of services are required (to include HO), patient preferences are considered important and local resources accounted for, decisions made based on this information may not be optimal. Future research is required to explore the impact of local decision-making factors on estimates of cost-effectiveness and ways in which to consolidate local and wider NHS perspectives. Further analyses might also explore the whole commissioning picture whereby both short- and long-term service decisions are made concurrently.

Changes to protocol

Recruitment to the DCE was initially very slow so rather than risk getting insufficient participants for the health economic evaluation we continued to sample after the initial 200 data sets. As the profile of the DCE patients would then mean oversampling on some models (e.g. hospital attendance), a decision was taken to continue recruiting patients to provide anonymised data. We chose not to conduct the EVSI as the EVPI figures suggested that further research was not warranted.

Panel composition

The panel comprised an independent chair (pharmacist), a CCG representative, an OPAT nurse, a microbiologist, two pharmacists, an infection specialist clinician, a patient, a statistician, a clinical triallist and a health economist. Three members of the panel sit on the BSAC committee. Three patients from our PAG were invited to attend. Two were unable to attend on the day owing to illness and the third attended the first half of the event but was unwell and could not stay for the discussion group. In addition to the independent panel members, six members of the research team were present and were available to answer questions during the group discussions.

Question 1: What is likely to represent an optimal service model for delivery of antimicrobial therapy for the two patient groups (long- and short-term i.v. antibiotic patients)?

Members of the expert panel drew on their own clinical expertise and experience as well as the evidence presented from CIVAS when considering these questions. To facilitate their discussion, three inter-related questions were posed.

Question 2: where does patient choice fit?

Question 3: Is there sufficient evidence upon which to base a clinical trial comparing service models?

Following a long discussion, it was agreed that NIHR would not fund a randomised clinical trial comparing two or more models of care. The rationale for this view was:

• There is no obvious comparator group.

• It was difficult to decide what the outcome measure would look like. Quality of life and readmission to hospital seemed the most plausible, but readmission is not common, which would lead to an unfeasibly large trial.

• The OPAT population is too heterogeneous (e.g. cellulitis vs. bone infection), and a complex cellulitis may also require longer treatment than a simple cellulitis.

• We do not have a clear picture of what current service provision in England looks like. This makes it impossible to determine whether or not there are sufficient ‘clean’ sites to introduce a new model to test in any future trial. It was suggested that BSAC data should be used to help answer this question. Another source of information would be local commissioners in each CCG.

• It was agreed that it is unlikely that any CCG would pay for the set up costs of any new services to test the effectiveness of ‘pure’ models (e.g. pure nurse-led community service vs. doctor-run, hospital-led service).

• There was a discussion about whether or not a hub and spoke model could be used (cluster trial) so that an area could offer both services, but areas would be large and it was agreed that this was unlikely to be feasible.

A final but important stumbling block to any trial was that if the premise from the first discussion holds true, namely that any service should be accessible to as many people as possible, then comparing HO with community-based treatment becomes an invalid comparison. The data presented by the research team indicate that there is heterogeneity in patient preferences, and the group felt that offering choice is important. It was agreed that it is more important to offer forward-thinking models of care that meet the needs of patients, which is why such different models of care have developed across the country.

Finally, there is an acknowledged lack of equipoise in the community (most services are developed and run by enthusiasts, who set up what they believe to be the best model). The group discussed the possibility of other study designs.

Question 4: what should the future research priorities for outpatient parenteral antimicrobial therapy be?

• It was agreed that replicating the DCE study in other areas would be useful. There were some concerns within the expert panel members around the lack of ethnic diversity in the development of the questionnaire. If the robustness of the measure could be determined by input from further work with patients from a range of ethnic groups, this would be beneficial.

• The inclusion of attitudinal data questions in the DCE was of particular interest to commissioners and there was a desire to have this included in a commissioning toolkit to inform commissioning decisions and also to use for quality monitoring, along with other measures.

• It might be useful to consider the cost-effectiveness of OPAT services, but also to consider the costs of ‘injectables’ (e.g. blood products) more widely.

• One possible area of research would be to consider patients’ willingness to pay. It was proposed that repeating the DCE with a question about willingness to pay would be interesting. However, this may dominate the model, so it would be important to have a split sample to test what impact this has on the data. However, from a commissioning perspective it was viewed as unlikely that patients would be asked to pay for OPAT as it saves the NHS money. We could look at willingness to pay (patient) to travel to a health-care setting to receive care versus care at home. For example, having a service at hospital versus at a general practice/medical centre. Would paying for transport influence patient choices? Would being able to park (e.g. at GP surgery) overcome any reluctance that patients have?

• Is there a role for a GP-facilitated model of OPAT (not involving GPs, but using their venue with a district nurse/SN model of care)?

• Is there a role for the third sector (voluntary services) and private companies?

• Who has overall responsibility for patients’ infections? It was agreed that the governance of OPAT services was an important issue. Although the referring specialist could maintain control, is this clinically effective? Would it be better for the patient to be ‘managed’ by an infection specialist?