Evaluating Long-term Outcomes of NHS Stop Smoking Services (ELONS): a prospective cohort study

Article history

The research reported in this issue of the journal was funded by the HTA programme as project number 09/161/01. The contractual start date was in October 2010. The draft report began editorial review in November 2014 and was accepted for publication in May 2015. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The HTA editors and publisher have tried to ensure the accuracy of the authors’ report and would like to thank the reviewers for their constructive comments on the draft document. However, they do not accept liability for damages or losses arising from material published in this report.

Declared competing interests of authors

Dr Shahab has received grants and honoraria from Pfizer, a manufacturer of smoking cessation products. In the last 3 years, Professor Aveyard has received a consultancy fee for 1 day of consultancy with Pfizer, a manufacturer of smoking cessation products. Professor Coleman was paid an honorarium and travel expenses for speaking at Paris Smoking Cessation Practitioners’ Conference in January 2014. He was also reimbursed for attending two expert meetings hosted by Pierre Fabre Laboratories (PFL, France), a company that manufactures nicotine replacement therapy (2008 and 2012). Dr McRobbie has received research grants, honoraria and travel expenses from Pfizer and Johnson & Johnson, manufacturers of smoking cessation products. Dr McEwen is a trustee and board member for Action on Smoking and Health (ASH), but received no financial reimbursement for this. He has received travel funding, honorariums and consultancy payments from manufacturers of smoking cessation products (Pfizer Ltd, Novartis UK and GlaxoSmithKline Consumer Healthcare Ltd) and hospitality from North 51 that provide online and database services. He also has a shared patent on a novel nicotine device but has received no payment for, or relating to, this patent.

Copyright statement

© Queen’s Printer and Controller of HMSO 2015. This work was produced by Dobbie et al. under the terms of a commissioning contract issued by the Secretary of State for Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK.

Early research on smoking cessation in the UK

The first evidence of clear links between tobacco smoking and ill-health emerged in the 1950s when Doll and Hill published the first paper showing that smoking caused lung cancer. 8 This, and other evidence, led to the production of two important reports – one in the UK and one in the USA – on smoking and health. In the UK the Royal College of Physicians report of 1962 outlined the need for a comprehensive approach to reduce smoking rates. 9 This described a series of needed policy measures including tobacco taxes, restricting advertising and availability, education on the risks of smoking and, importantly, support for smokers wanting to quit. In the end, it would take almost 50 years for all the recommendations in Smoking and Health to be implemented, but in the meantime a range of important studies were conducted that provided better evidence on smoking as an addiction, rather than a ‘habit’, and consequently how it should be treated. 10 A number of early studies on nicotine and NRT were conducted by Professor Michael Russell and colleagues at the Addiction Research Unit at the University of London. These contributed to the licensing of the first pharmacological treatment for smoking cessation in 1981 – nicotine gum. From that period onwards a number of important research and policy developments on smoking cessation took place in the UK. Members of our team summarised these in an earlier study and we reproduce these here, outlined in Table 1. 10

NHS Stop Smoking Services

The NHS Stop Smoking Services (SSSs) were established following the publication of the 1998 White Paper, Smoking Kills. 15 Initially piloted in deprived areas of England in 1999, they were rolled-out across the UK from 2000. The services were developed on the basis of national guidance issued by the Department of Health (DH) that built on a review of the evidence of the effectiveness of smoking cessation interventions published in the journal Thorax. 16 This evidence emphasised the efficacy of intensive behavioural support (in groups or one to one) plus pharmacotherapy for smoking cessation. Services were established by primary care trusts (PCTs) and operated primarily in primary care settings delivering behavioural support and providing access to NRT and bupropion (Zyban^®, GlaxoSmithKline). From 2000–4, a national evaluation of the services in England was conducted by members of our team. The evaluation found that the services were effective in supporting smokers to quit in the short term (at 4 weeks)17 and the longer term (at 1 year),18 and were reaching smokers from more deprived groups. 19 A subsequent analysis by our team also found that they were making a contribution to reducing inequalities in health caused by smoking. 20 However, since these studies were conducted, NHS SSSs have continued to evolve, attempted to adapt to meet local needs and encountered various financial and structural challenges. National developments, including the National Institute for Health and Care Excellence (NICE)’s guidance on services (published in 2008),21 have influenced what is available and smoking cessation medications have also diversified, with new NRT products and the medication varenicline (Champix^®, Pfizer) providing additional options for smokers trying to stop.

One element that has changed in the past decade is the type and variety of behavioural support options available to smokers using the services. In particular, a heavier reliance on one-to-one support options in a far wider range of settings. The existing evidence base from systematic reviews of trials would suggest that there is no significant difference in outcomes for smokers receiving group or one-to-one behavioural support. In a review of 50 randomised controlled trials22 and another review of 23 studies since January 2000,23 a group format for a behavioural intervention was not found to be significantly different from an individual format; yet real-world evidence may be different from trials in this respect. Findings from our earlier studies with NHS SSSs suggest that group interventions may be more effective in practice24 but that smokers, given a choice, will choose one-to-one support. Issues of perceived preference may have led to the dominance of one-to-one support in today’s services although new forms of group counselling (such as ‘rolling groups’ where clients can join without having to wait for a new group to start) have emerged.

Another element that has evolved is who provides support to stop smoking. Before the services existed, this was primarily doctors and nurses in primary care or hospitals with a few examples of specialist clinics. Today, a wider variety of professionals are involved in delivering support. In some cases this is described as different ‘levels’ of support. We describe these levels here.

• Level 1 practitioners are typically primary care professionals who deliver ‘brief interventions’, which usually involves opportunistic advice, discussion, awareness raising and/or referral to SSSs.

• Level 2 practitioners (known elsewhere as community practitioners) do not work exclusively as stop smoking specialists but instead work in other health and social care roles and deliver stop smoking support as one of a number of job tasks. The majority of level 2 service provision takes place in general practitioner (GP) practices (usually delivered by nurses and health-care assistants) and pharmacies (delivered by practitioners with a variety of posts from pharmacists to pharmacy assistants). Generally one-to-one support is offered.

• Level 3 practitioners’ sole remit is to deliver smoking cessation working as smoking cessation ‘specialists’. They offer a wider variety of behavioural support which can include: open and closed groups, one to ones, drop-in services, text and telephone support,25 and work in various locations such as community centres, GP practices and workplaces.

For clarity, throughout this report we will differentiate between level 2 and level 3 service providers by using the terms ‘level 2′ and ‘specialist’ practitioners. Data on level 1 support are not collected by the SSS and is not part of the study.

A growing proportion of clients now receive support from level 2 practitioners. Large differences have now been identified between the quit rates achieved by services in different locations, which may be partly because of the quality of behavioural support delivered by the practitioner who offers it. 26–28

The organisational context within which SSSs operate has also changed since our earlier studies. Some of these changes preceded the Evaluating Long-term Outcomes of NHS Stop Smoking Services (ELONS) study (e.g. reorganisation of PCTs in England) but possibly the most significant change occurred in April 2013, just after the fieldwork was completed, when commissioning of local SSSs moved from NHS to local authority control. This resulted in change to the funding and delivery structure of local SSSs, with some areas either reducing funding for the specialist service in favour of a greater reliance on level 2 providers or tendering out previously in-house services which has sometimes led to SSSs being run by private and voluntary sector companies. This structural change poses challenges for the services and preparation for the move also, to some extent, affected study recruitment, which is discussed further in Chapter 4.

As a result of all these changes, current research is needed to examine the longer-term efficacy of the different methods employed by NHS SSSs to deliver support to smokers trying to stop. Current evidence is also required to explore the effectiveness of services with different groups of smokers, particularly those from more disadvantaged groups. This report sets out findings from a study designed to examine these issues.

Structure of this report

This is a detailed report of a complex study with a number of different elements. Chapter 2 presents an overview of the research including aims and objectives, study design and settings. It was also describes the ethical approval and local research permissions process.

Chapters 3 and 4 present the methods and main findings of the secondary analysis element of the ELONS study respectively. Chapters 5 and 6 describe the methods and analysis (see Chapter 5) and the results and key findings (see Chapter 6) for the second (and main) part of our study. This was a prospective cohort study of smokers using the services who were followed up in the short (4 weeks post quit date) and longer term (at 1 year).

The subsequent three chapters present findings from elements that were added to the ELONS prospective study. For ease and clarity, they are presented as standalone chapters including the methods and results for each. The focus of Chapter 7 is an exploration of client satisfaction with the support received from their local SSS. Chapter 8 examines the relationship between well-being and smoking status of ELONS study participants. Chapter 9 outlines findings from a study of longer-term NRT use that was made possible by additional funding from the UK Centre for Tobacco Control Studies [now the UK Centre for Tobacco and Alcohol Studies (UKCTAS) – www.ukctas.ac.uk]. The remaining two chapters include the discussion (see Chapter 10) with conclusions presented in Chapter 11.

Aim and objectives

The principal aim of the study was to explore the factors that determine longer-term abstinence from smoking following intervention by SSSs in England.

The study objectives were to:

• examine the effectiveness of SSSs by PCT and intervention type using routine data

• explore the reach of services by identifying what proportion of the local population set a quit date with services using routine data

• describe the factors that determine longer-term abstinence from smoking or relapse to smoking among clients who set a quit date with services in a sample of PCTs in England

• examine the relationship between client characteristics [in particular socioeconomic status (SES), age, gender, disability and ethnicity], adherence to treatment, intervention type received and longer-term abstinence

• create an evidence base to guide delivery of interventions by SSSs so that these interventions will have maximal effect on smoking cessation and population health.

Overview of study design

The ELONS study was an observational study with two main stages:

• secondary analysis of routine data collected by SSSs

• a prospective cohort study of SSS clients with three additional elements (which are presented as standalone chapters on the report):

  • a client satisfaction survey (CSS)

  • a well-being survey

  • a study of longer-term NRT use.

The overall aim of the prospective element of the ELONS study was to collect long-term follow-up data at 52 weeks. A subsidiary aim was to collect detailed information about clients and the treatment they received in a consistent manner. Data were collected at three key stages: baseline (i.e. when a client registered with the SSS), 4-week follow-up and 52 weeks post quit date. The 52-week follow-up data were collected by a research company, TNS BMRB (Taylor Nelson Sofres, British Market Research Bureau), that had worked with the academic team on a previous study. 29 Table 2 describes the data collected at each time point and also summarises the data collected for the three additional elements (client satisfaction, well-being and long-term NRT). Further detail of the data collected at each stage and data collection tools can be found in Appendix 1. A flow chart depicting sample sizes and sampling frames is also available (Figure 1).

FIGURE 1.

Flow diagram for the ELONS prospective study. a, n refers to quit attempts at the SSS from March/April 2012 to March 2013; b, more detail is available in Chapters 5 and 6; c, providers’ employees provide smoking cessation support rather than specialists employed directly by SSSs; d, this is an overestimate because level 2 providers who did recruit did not usually recruit for the level 2 recruitment period at each site; and e, CSS.

Definitions of smoking cessation

Smoking cessation, or ‘quitting’, was defined in the following ways for this study.

Setting

The setting for the study was SSSs in England. For the secondary analysis, routine data from 49 services were obtained. For the prospective study, nine services were involved. These were Bristol, County Durham and Darlington, Hull and East Riding, Leicestershire County and Rutland, North and North East Lincolnshire, Northamptonshire, Oldham, Rotherham, and South East Essex. The target population was clients of these services. The health technologies that the ELONS study assessed were the treatment provided by SSSs – a combination of behavioural support and medication intended to increase the chance of a successful quit attempt. A key focus was to evaluate outcomes for smokers who received one of the five main forms of behavioural support (also called intervention types) provided by the services:

• closed groups (scheduled group sessions of around 1–2 hours dependent on the number of clients, normally delivered once per week and facilitated by a specialist practitioner

• open/rolling groups (drop-in groups run at a variety of times of day and in a variety of locations where clients can attend without an appointment and for as many weeks as they wish)

• one-to-one drop-in (a clinic hosted at a regular time where clients can attend without an appointment for one-to-one support)

• one-to-one sessions with a specialist practitioner (scheduled one-to-one sessions with a practitioner whose main role it is to help people stop smoking)

• one-to-one sessions with a level 2 practitioner.

Ethics and local permissions

Ethical approval for the ELONS study was obtained from NHS Lothian (South East Scotland Research Ethics Committee) in June 2011. During the recruitment phase of the prospective study a total of four substantial amendments were made for alternations to the study protocol and consent process. In addition to ethical approval, we also obtained local research and development (R&D) approval for each SSS. For three SSSs, single R&D office approvals were required but for the remainder two sets of approvals were required owing to the location of the SSSs and the NHS trusts involved. All the necessary local approvals were obtained before data collection began. It was also a requirement that the research team obtain NHS research passports and, for one site, it was necessary to obtain Caldicott Guardian approval.

Public involvement

Members of the public were involved in the ELONS study in a number of ways. Within the prospective study, the public were research participants and the study was designed to provide feedback on an important public service for the benefit of future users. However, more direct forms of involvement were also included in the research.

First, the study principal investigator, Professor Bauld, serves as the public engagement lead for UKCTAS, a UK Clinical Research Collaboration Centre for Public Health Excellence that covers 13 universities (the co-investigators for the study are also UKCTAS members). In this role she convenes a panel of continuing smokers and recent quitters who meet in Bath. The original design for the ELONS study was discussed with the panel at meetings in 2010 and again in 2011, and feedback sought. Initial results were also circulated to panel members at a meeting in 2013. One of the panel members, Robert Graham, was asked to join the study steering group as a lay adviser and did so. Although he was not able to attend all steering group meetings, he maintained contact with Professor Bauld and, having used NHS SSSs himself in the past, provided very useful input.

In addition to panel contributions and Mr Graham’s involvement, we had the participant information leaflets and consent forms for the study reviewed by a patient representative. This was organised through the Primary Care Research Network (PCRN) Patient and Public Involvement Manager, and we are grateful to them for facilitating this process.

We also prepared the ELONS study newsletters for all the participating services in the prospective study. There were three of these during the study. These were intended to be available to service clients as well as staff and we sent these to each site for distribution.

Finally, tailored feedback from the study was prepared for each study site and for one site an event was held that involved a varied audience and presentations on study findings. Dissemination at a number of academic and practitioner conferences is currently under way.

Rationale for secondary analysis

All SSSs in England are required to submit routine monitoring data to the Health and Social Care Information Centre. 31 These data include basic information on client characteristics and the types of treatment received. They also include self-reported smoking status at 4 weeks post quit date. These monitoring data are supplied as aggregated summary returns and so cannot be used to analyse which factors are associated with individual-level smoking cessation outcomes.

Increasingly, however, SSS outsource management of these data to private companies. The most commonly used is QuitManager (North 51, Nottingham, UK). 7,27 QuitManager provides a framework for SSSs to collect the minimum data set required by the DH, plus any additional client and service information thought to be useful locally. The research team includes the Director of the National Centre for Smoking Cessation and Training (NCSCT), who has good links with North 51 and has been able to access their data for previous research and training activities. We were therefore able to obtain QuitManager data for the ELONS study in an anonymised form. This was important as by having access to these data made it possible to examine individual-level information not available from the Health and Social Care Information Centre returns. Data were available from 49 services in England. These data were analysed to fulfil objectives 1 and 2, to examine the effectiveness and reach of services. Results from this first stage also informed the selection of SSSs for the prospective cohort study.

The secondary analysis had two aims:

• to explore the reach of services by identifying the proportion of the local population who set a quit date with services using routine data

• to examine the effectiveness of NHS SSSs by PCT and intervention type using routine data.

Sample

QuitManager32 is an online database for recording information on NHS SSS clients, including sociodemographic and treatment characteristics, in accordance with the DH’s standard SSS monitoring requirements. Collection of data starts at baseline (first contact with the SSS) and should be updated at each contact point. The data are collected by the stop smoking practitioner and entered onto QuitManager in ‘real time’ (via the computer software) or recorded on a paper form and entered at a later date. PCTs may elect not to ask clients questions, and clients themselves may not answer every question, leading to missing data. Within PCTs clients whose data are collected on paper forms may answer a slightly different set of questions to those whose data are entered electronically.

At 4 weeks post quit date clients are asked whether or not they have quit smoking for the past 14 days (self-report quit) and clients may also perform a CO breath test. Clients who have self-reported as quit and have CO readings of < 10 p.p.m. are said to be CO validated as quit. The DH guidelines suggest 85% of clients should be CO validated. 33

More than 60 out of the ≈ 150 English PCTs use QuitManager and 49 of these gave permission to the NCSCT for their data to be used for research purposes. Thus, data from these 49 PCTs were included in the analysis.

Data on clients who set quit dates from July to December 2010 were downloaded from QuitManager in January 2012 and data on clients with quit dates between January and June 2011 were downloaded in July 2011. In total there were 202,804 client records included in the database. Each record is a ‘treatment episode’ where the client sets a quit data and receives treatment. It is possible for clients who relapse to have more than one treatment episode in a year. 33

Measures

Analytic approach

The analyses were carried out using SPSS [SPSS version 19.0 (SPSS Inc., Armonk, NY, USA) and PASW version 18.0.3 (SPSS Inc., Chicago, IL, USA)]; Microsoft Excel^® 2010 (Microsoft Corporation, Redmond, WA, USA); Stata (StataCorp LP, College Station, TX, USA) and MlwiN version (MLwiN, Centre for Multilevel Modelling, Bristol, UK) as follows.

See Table 3 for overall quit rates and validation rates for all clients and then for clients where their age, gender and practitioner were identified. Uptake (distribution), quit rates and CO-validation rates for subgroups of these clients with various client and quit attempt-related characteristics are presented. Chi-squared was used to assess significance.

Second, a descriptive table (see Table 4) provides further information about the PCTs. Each row represents a different PCT. It includes the number of practitioners within each PCT, the number of clients per PCT (throughout the year) and the average number of clients per practitioner (clients/practitioner) in addition to estimates of the smoking rate, population, clients aged over 16 years and reach of services of each PCT.

The analysis included a large number of predictors and three measures of SES, so in order to assess potential multicollinearity in multivariable models, the variance inflation factor (vif) and corresponding tolerance were calculated from a logistic regression using interaction expansion (part of the Stata xi suite). In this xi suite analysis, PCT was entered as a cluster variable and practitioner was entered as an explanatory variable as only one cluster variable is allowed.

Multilevel modelling (see Tables 5 and 6) allows PCT and practitioner to be treated as separate levels in the analysis – otherwise known as random effects. Thus there were three levels in the models: client (level 1), practitioner (level 2) and PCT (level 3). Other independent variables were entered as fixed effects. Practitioner types were entered as a level 2 fixed effects, IMD quintiles of the PCT entered as level 3 fixed effects and all other fixed effects were at client level. MLwiN second order penalised quasi-likelihood iterative generalised least squares (IGLS) estimation was used for the analysis.

Odds ratios (ORs) are reported for self-report quit, CO-validated quit and CO validation. Some changes were made to the predictors for the multilevel analysis. So that practitioner should completely nest within a PCT, PCTs that shared practitioners were merged. There were 63 practitioners who worked in more than one PCT. The IMD score was then recoded for the new merged PCTs so that it was the average of both PCTs taking into account the number of clients contributed by each PCT.

In preliminary analysis (not shown) MLwiN IGLS multilevel models were compared with Stata Survey Suite models. Anomalous results were explored further using SPSS crosstabulations, Stata xt and MLwiN Markov chain Monte Carlo. Full models did not converge in either Stata xt or MLwiN Markov chain Monte Carlo. However, the results that were produced tended to support IGLS results; thus only MLwiN IGLS models are presented here.

Ideally we would look at how much variance of the higher levels was explained by the fixed variables; however, when a binary outcome is modelled and there is more than one higher level, it is not possible to calculate because the level 1 variance is always constrained to be 3.29. 37 Thus the impact observed on the higher-level variances from adding a fixed-effect variable to the model may be because of the overall variance changing to allow for the addition of the variable since the level 1 variance cannot change, rather than because the new variable in the model has explained the higher-level variance. 38,39 It is not currently possible to accurately estimate how much of the change is caused by real changes in variance and how much is caused by the constrained level 1 variance (Yang M, University of Nottingham, 2012, personal communication).

The higher-level variance is shown later in the report (see Table 7). The first and second columns show higher-level variance in the CO-validated modelling. The first column displays PCT-level variance and the second column displays practitioner-level variance. The next two columns show variance for the self-report modelling and the other columns show higher-level variance for the CO-validation modelling with or without exclusions of extreme PCTs (see Table 4). The first row displays variance in null models with no fixed effects entered. The subsequent rows display variance when fixed effects are entered one at a time so there is only one fixed effect in a model. The final row displays variance for the multivariable model when all fixed effects are entered. PCTs with significantly below and above average residuals were identified.

Residuals for the PCTs were calculated and graphed using caterpillar plots.

For measures of impact by PCT, see Table 8. Each row represents a different PCT. Throughput (numbers setting a quit date per 100,000 population), CO-validated and self-report quit rates for all unique clients age 16 years or over, impact (numbers of 4-week quitters per 100,000 population) calculated using CO-validated quits only, all self-report quits and overall (average of previous two impacts) are shown and the results are also presented in a graphical format. Summary measures of unique patients, quit rates, throughput and impact, and correlations between them are provided (see Table 9).

Uptake

Table 3 shows which groups of clients were more or less common among service users and the service options that they were more or less likely to use.

Reach of services by primary care trust

Estimated smoking rates varied from 15% to 41%. PCT populations varied from 80,000 to nearly 900,000. Most services saw between 5% and 10% of their population of smokers between July 2010 and June 2011. Note that smoking estimates were accompanied by confidence intervals (CIs) of approximately 6% thus these should be treated with caution.

Cessation

Multivariable results

Multivariable modelling in Stata indicated the extent of multicollinearity. All vifs were below 2.54 for self-reported and CO-validated models so there was no significant multicollinearity in the data (concerns are raised if a vif is ≥ 10). Multilevel multivariable ORs for client variables are presented in Table 5 and ORs of quit attempt-related variables are presented in Table 6.

TABLE 5 - Multilevel multivariable ORs (95% CI) of CO-validated quit, self-report quit and CO validation: client characteristic fixed effects

Three PCTs with particularly high or low CO-validation rates were excluded.

Outcome	Self-report quit, OR (95% CI) (n = 177,291)	CO-verified quit, OR (95% CI) (n = 177,291)	CO validation, OR (95% CI) (n = 86,512)	CO validation, OR (95% CI) (n = 80,002)
Exclusions	–	–	–	Extreme PCTsa
Null model (constant)	0.34 (0.27 to 0.43)	0.13 (0.09 to 0.19)	1.58 (0.77 to 3.25)	1.69 (0.98 to 2.90)
Age (years) at quit date quartiles
10–30	1	1	1	1
31–42	1.44 (1.40 to 1.48)	1.53 (1.48 to 1.58)	1.29 (1.22 to 1.35)	1.34 (1.27 to 1.41)
43–53	1.54 (1.49 to 1.59)	1.73 (1.67 to 1.79)	1.48 (1.40 to 1.57)	1.60 (1.51 to 1.69)
54–100	2.02 (1.96 to 2.09)	2.28 (2.20 to 2.37)	1.69 (1.60 to 1.80)	1.87 (1.76 to 1.99)
Gender
Female (not pregnant)	1	1	1	1
Female (pregnant)	1.29 (1.20 to 1.38)	1.16 (1.07 to 1.26)	0.93 (0.82 to 1.05)	0.87 (0.77 to 1.00)
Male	1.07 (1.04 to 1.09)	1.04 (1.02 to 1.07)	0.97 (0.94 to 1.01)	0.97 (0.93 to 1.00)
Ethnicity
Black, Asian, mixed, other	1	1	1	1
White	1.02 (0.98 to 1.07)	1.04 (0.99 to 1.09)	1.05 (0.97 to 1.13)	1.05 (0.97 to 1.14)
Unknown	1.02 (0.95 to 1.09)	0.96 (0.89 to 1.03)	0.90 (0.80 to 1.02)	0.89 (0.79 to 1.01)
NSSEC
Routine and manual	1	1	1	1
Intermediate	1.14 (1.10 to 1.19)	1.09 (1.04 to 1.14)	0.96 (0.89 to 1.03)	0.96 (0.89 to 1.03)
Managerial and professional	1.19 (1.15 to 1.24)	1.14 (1.10 to 1.18)	1.02 (0.97 to 1.08)	1.02 (0.96 to 1.09)
Retired or home carer	1.07 (1.03 to 1.11)	1.06 (1.02 to 1.11)	1.02 (0.96 to 1.09)	1.02 (0.96 to 1.09)
Sick or disabled and unable to work	0.70 (0.67 to 0.73)	0.70 (0.67 to 0.74)	0.85 (0.78 to 0.93)	0.83 (0.76 to 0.90)
Always/long-term unemployed	0.75 (0.72 to 0.78)	0.76 (0.73 to 0.79)	0.96 (0.90 to 1.02)	0.94 (0.88 to 1.01)
In prison	1.15 (1.00 to 1.32)	1.98 (1.71 to 2.29)	3.82 (2.92 to 4.98)	4.84 (3.61 to 6.47)
Other	0.84 (0.81 to 0.87)	0.82 (0.78 to 0.85)	0.91 (0.85 to 0.97)	0.89 (0.83 to 0.96)
Prescription payment
Exempt	1	1	1	1
Pays	1.24 (1.21 to 1.27)	1.18 (1.14 to 1.21)	0.98 (0.93 to 1.02)	0.97 (0.93 to 1.02)
Unknown	1.08 (1.03 to 1.13)	0.92 (0.88 to 0.97)	0.84 (0.78 to 0.90)	0.80 (0.74 to 0.86)
Average IMD score 2010 (PCT)
Ranked 1–30 (disadvantaged)	1	1	1	1
Ranked 31–60	1.20 (0.85 to 1.69)	1.58 (0.91 to 2.77)	3.61 (1.14 to 11.47)	1.67 (0.72 to 3.86)
Ranked 61–90	1.29 (0.88 to 1.89)	1.34 (0.72 to 2.51)	0.61 (0.17 to 2.19)	0.60 (0.25 to 1.45)
Ranked 91–120	1.25 (0.92 to 1.70)	1.43 (0.88 to 2.35)	1.34 (0.49 to 3.67)	1.19 (0.59 to 2.42)
Ranked ≥ 121 (affluent)	1.36 (0.96 to 1.91)	1.61 (0.92 to 2.81)	1.06 (0.34 to 3.33)	1.01 (0.45 to 2.22)

TABLE 6 - Multilevel multivariable ORs (95% CI) of CO-validated quit, self-report quit and CO validation: quit attempt-related fixed effects

Three PCTs with particularly high or low CO-validation rates were excluded.

Outcome	Self-report quit, OR (95% CI) (n = 177,291)	CO-verified quit, OR (95% CI) (n = 177,291)	CO validation, OR (95% CI) (n = 86,512)	CO validation, OR (95% CI) (n = 80,002)
Exclusions	–	–	–	Extreme PCTsa
Quit date month
July 2010	1	1	1	1
August 2010	1.04 (0.99 to 1.09)	1.04 (0.99 to 1.10)	1.05 (0.96 to 1.15)	1.05 (0.96 to 1.15)
September 2010	1.14 (1.08 to 1.20)	1.11 (1.05 to 1.17)	1.02 (0.93 to 1.11)	1.02 (0.93 to 1.11)
October 2010	1.10 (1.05 to 1.16)	1.08 (1.02 to 1.14)	1.01 (0.92 to 1.09)	1.00 (0.91 to 1.09)
November 2010	1.02 (0.97 to 1.07)	0.88 (0.84 to 0.93)	0.82 (0.75 to 0.89)	0.77 (0.71 to 0.85)
January 2011	1.33 (1.27 to 1.39)	1.34 (1.28 to 1.41)	1.11 (1.02 to 1.19)	1.11 (1.03 to 1.21)
February 2011	1.17 (1.12 to 1.22)	1.19 (1.14 to 1.25)	1.06 (0.98 to 1.15)	1.06 (0.97 to 1.15)
March 2011	1.02 (0.98 to 1.07)	1.04 (0.99 to 1.09)	0.98 (0.90 to 1.06)	0.96 (0.88 to 1.05)
April 2011	0.96 (0.91 to 1.01)	1.02 (0.97 to 1.08)	1.08 (0.99 to 1.17)	1.08 (0.99 to 1.18)
May 2011	0.85 (0.81 to 0.89)	0.97 (0.91 to 1.02)	1.08 (0.99 to 1.19)	1.10 (1.00 to 1.21)
Episode
Episode 1	1	1	1	1
Episode 2	0.97 (0.94 to 0.99)	0.97 (0.94 to 0.99)	1.00 (0.96 to 1.05)	1.00 (0.95 to 1.05)
Episode 3	1.00 (0.96 to 1.04)	0.98 (0.94 to 1.03)	1.00 (0.94 to 1.07)	0.99 (0.93 to 1.07)
Episode 4 or more	1.04 (1.00 to 1.08)	1.04 (1.00 to 1.09)	1.05 (0.98 to 1.13)	1.04 (0.96 to 1.13)
Medication
Single NRT	1	1	1	1
Combination NRT	1.88 (1.83 to 1.94)	2.06 (2.00 to 2.13)	1.55 (1.47 to 1.65)	1.66 (1.56 to 1.76)
Bupropion only	1.79 (1.58 to 2.03)	1.48 (1.29 to 1.69)	0.97 (0.79 to 1.20)	0.96 (0.77 to 1.18)
Varenicline only	2.57 (2.49 to 2.65)	2.31 (2.24 to 2.40)	1.31 (1.23 to 1.39)	1.37 (1.29 to 1.45)
Mixed NRT/bupropion/varenicline	1.46 (1.37 to 1.57)	1.64 (1.53 to 1.76)	1.46 (1.30 to 1.65)	1.55 (1.37 to 1.76)
No medication or missing	1.13 (1.08 to 1.18)	0.80 (0.76 to 0.84)	0.74 (0.68 to 0.80)	0.68 (0.63 to 0.74)
Intervention type
One to one	1	1	1	1
Drop-in clinic	0.96 (0.91 to 1.01)	1.02 (0.96 to 1.07)	1.01 (0.93 to 1.10)	0.97 (0.89 to 1.06)
Open (rolling) group	1.15 (1.04 to 1.27)	1.28 (1.15 to 1.41)	1.39 (1.18 to 1.64)	1.45 (1.23 to 1.73)
Closed group	1.12 (1.00 to 1.26)	1.11 (0.99 to 1.24)	1.15 (0.96 to 1.37)	1.16 (0.96 to 1.39)
Other or missing	1.12 (1.03 to 1.21)	0.62 (0.57 to 0.68)	0.46 (0.40 to 0.51)	0.41 (0.36 to 0.46)
SSS practitioner
Specialist	1	1	1	1
GP	0.46 (0.38 to 0.55)	0.53 (0.42 to 0.66)	1.15 (0.83 to 1.61)	1.43 (0.97 to 2.10)
Nurse	0.58 (0.52 to 0.64)	0.70 (0.62 to 0.78)	1.11 (0.95 to 1.29)	1.19 (0.99 to 1.43)
Health-care assistant	0.69 (0.60 to 0.78)	0.86 (0.74 to 0.99)	1.29 (1.05 to 1.58)	1.42 (1.11 to 1.80)
Pharmacy	0.67 (0.61 to 0.73)	0.91 (0.82 to 1.02)	1.80 (1.55 to 2.10)	2.20 (1.84 to 2.64)
Other or unknown	0.78 (0.72 to 0.84)	0.88 (0.80 to 0.97)	1.25 (1.10 to 1.41)	1.34 (1.16 to 1.55)

The results for CO-validated quit and self-report quit are mostly similar so they are discussed simultaneously with differences alluded to. Generally unknown and other classifications were associated with less chance of quitting than reference groups.

Demography

The relationship between client-related characteristics and CO-validated or self-report quits were mostly similar. As age increased so did the chances of quitting. Clients aged over 53 years were twice as likely to quit as those age 30 years or under. Men and pregnant women were more likely to quit than non-pregnant women. In the bivariable results pregnant women had the lowest quit rate. Further exploration (not shown) suggested that the change in the multivariable results was attributed to pregnant women being less likely to take medication. Ethnicity was not significantly related to quitting.

Socioeconomic status

Compared with clients with routine and manual occupations, clients with intermediate, managerial and professional occupations, the retired, home carers and prisoners were more likely to quit. Those who paid for prescriptions were more likely to quit than those who were exempt. PCT deprivation was not significantly related to quitting.

Stop Smoking Service related

September, January and February appeared to be particularly successful months for setting a quit date with a SSS. Lower chances of quitting in May and November might partly be because of late quit information not being included in the database owing to cut off points at the end of December and June. However, differences between these months and the other months were not always significant, suggesting this had a minimal effect. Clients who had used the service once before were less likely to quit than those who had never used the service before and those who had used the service four or more times were more likely to quit. Taking any medication other than single NRT increased the chances of quitting. The highest ORs were for clients who took varenicline alone. These people were more than twice as likely to self-report and CO validate as quit.

For intervention type, the highest chances of quitting were among clients who took part in open/rolling groups. Closed-group clients were significantly more likely to self-report as quit than those who undertook one-to-one counselling but the difference did not quite reach significance for CO validation. Further analysis (not shown) suggested this was caused by a number of factors such as the practitioner, SES and medication.

Among the practitioner types, clients of specialist advisers were more likely to quit than clients of other (and unspecified) practitioner types. This difference was significant for all categories except for CO validation among practitioners who held pharmacy posts. ORs were surprisingly low for GPs given bivariable results. Further analysis (not shown) suggested this was because the majority of GPs worked in one PCT (Warwickshire) where quit rates were similar to other practitioner types, but in most other PCTs, with more than 20 clients seen by GPs, GPs were achieving fewer CO-validated quits than other practitioner types.

Primary care trust and practitioner

Figure 2 is a caterpillar plot of PCT residuals for the multilevel multivariable model of self-report quit. PCT CIs that do not overlap 0.0 represent significant departures from the average. The lowest chances of self-reported quitting were found in the PCT ranked 1.

FIGURE 2.

Residual caterpillar plot of PCTs for multivariable multilevel model of self-report quits.

Figure 3 is a caterpillar plot of PCT residuals for CO-validated quit. PCT CIs that do not overlap 0.0 represent significant departures from the average. Further exploration (not shown) suggested that excluding clients from the lowest ranked PCT made little difference to the results.

FIGURE 3.

Residual caterpillar plot of PCTs for multivariable multilevel model of CO-validated quits.

There is little that can be said with certainty about the higher-level variance (Table 7), as it is not possible to tell how much of the change in variance between models is owing to fixed effects in the model and how much is the result of level one variance being constrained to 3.29, leading to inflated or deflated higher level variance.

TABLE 7 - Primary care trust-level and practitioner-level variance for models with no fixed effects, one fixed effect and all fixed effects

MCMC, Markov chain Monte Carlo.

Models	CO validated		Self-report		CO validation
	PCT variance (95% CI)	Practitioner variance (95% CI)	PCT variance (95% CI)	Practitioner variance (95% CI)	PCT variance (95% CI)	Practitioner variance (95% CI)
No fixed effects
Null model (constant)	0.36 (0.20 to 0.53)	0.54 (0.51 to 0.58)	0.14 (0.07 to 0.20)	0.41 (0.38 to 0.43)	1.83 (1.03 to 2.63)	1.32 (1.23 to 1.42)
One fixed effect
Quit date month	0.36 (0.20 to 0.52)	0.54 (0.51 to 0.58)	0.14 (0.07 to 0.20)	0.41 (0.38 to 0.43)	1.83 (1.02 to 2.64)	1.33 (1.23 to 1.42)
Age (years) at quit date quartiles	0.38 (0.21 to 0.55)	0.55 (0.51 to 0.58)	0.14 (0.08 to 0.21)	0.40 (0.38 to 0.43)	1.84 (1.03 to 2.65)	1.35 (1.25 to 1.44)
Gender	0.36 (0.20 to 0.53)	0.54 (0.51 to 0.57)	0.14 (0.07 to 0.20)	0.41 (0.38 to 0.43)	1.82 (1.01 to 2.63)	1.32 (1.23 to 1.41)
Ethnicity	0.37 (0.20 to 0.53)	0.54 (0.51 to 0.57)	0.13 (0.07 to 0.20)	0.40 (0.38 to 0.43)	1.83 (1.02 to 2.64)	1.32 (1.23 to 1.42)
NSSEC	0.36 (0.20 to 0.51)	0.52 (0.49 to 0.56)	0.12 (0.07 to 0.18)	0.39 (0.36 to 0.41)	1.83 (1.03 to 2.63)	1.31 (1.22 to 1.40)
Prescription payment	0.36 (0.20 to 0.52)	0.53 (0.50 to 0.57)	0.13 (0.07 to 0.20)	0.40 (0.37 to 0.43)	1.82 (1.02 to 2.63)	1.30 (1.21 to 1.39)
IMD 2010 (PCT)	0.30 (0.17 to 0.44)	0.53 (0.49 to 0.56)	0.11 (0.06 to 0.16)	0.40 (0.38 to 0.43)	1.40 (0.78 to 2.02)	0.74 (0.68 to 0.80)
Episode	0.37 (0.20 to 0.53)	0.54 (0.51 to 0.58)	0.14 (0.07 to 0.20)	0.40 (0.38 to 0.43)	1.83 (1.02 to 2.63)	1.32 (1.23 to 1.42)
Medication	0.38 (0.21 to 0.55)	0.53 (0.50 to 0.57)	0.15 (0.08 to 0.23)	0.40 (0.37 to 0.43)	1.84 (1.02 to 2.65)	1.29 (1.19 to 1.38)
Intervention type	0.35 (0.20 to 0.51)	0.53 (0.50 to 0.56)	0.13 (0.07 to 0.19)	0.40 (0.37 to 0.42)	1.80 (1.00 to 2.60)	1.29 (1.20 to 1.38)
SSS practitioner	0.37 (0.20 to 0.53)	0.53 (0.50 to 0.57)	0.13 (0.07 to 0.19)	0.38 (0.35 to 0.40)	1.91 (1.07 to 2.75)	1.29 (1.20 to 1.38)
All fixed effects
Multivariable model	0.34 (0.19 to 0.50)	0.49 (0.46 to 0.52)	0.13 (0.07 to 0.18)	0.36 (0.33 to 0.38)	1.47 (0.82 to 2.11)	0.71 (0.65 to 0.76)
Multivariable MCMC model	0.43 (0.22 to 0.65)	0.53 (0.49 to 0.57)	–	–	–	–
Multivariable model without extreme PCTs	–	–	–	–	0.64 (0.34 to 0.94)	1.02 (0.94 to 1.10)

We can safely say that there was significant variance between practitioners and between PCTs even after all fixed effects had been included for both self-report and CO-validated quit. It is probably also safe to say that there was less variance between PCTs and between practitioners for self-report quits than CO-validated quits.

Carbon monoxide validation

The CO-validation rate (self-reports that were CO validated) of the database was 73.9% and was 74.3% among clients with gender, age and practitioner identified (see Table 3). The DH has recommended that SSSs should aim to CO validate 85% of clients. 33 Client groups where CO validation of 85% was achieved were prisoners and clients enrolled in open groups. Of the 49 PCTs, 13 achieved CO-validation rates of 85% or more.

Multivariable results

Odds ratios for IMD at PCT level and multilevel variance changes suggested that some PCTs were having a disproportionate effect on the results, as an intermediate quintile was most associated with quitting (see Table 4) and including this variable in the model was associated with a major reduction in PCT-level variance (see Table 7). Two PCTs, with very high CO-validation rates and one PCT with a particularly low validation rate were consequently excluded. Both models are shown (see Table 6). Multivariable modelling in Stata provided vifs and tols. All were below 3.20, so there was no significant multicollinearity in the data.

Demography

As age increased, CO validation was more likely. Non-pregnant women were more likely to be CO validated than men and pregnant women. Ethnicity had no effect on CO-validation rates.

Socioeconomic status

Prisoners were much more likely to be CO validated than other groups, approximately four times more likely than routine and manual workers. Eligibility for paying for prescriptions and PCT deprivation (once extreme PCTs were excluded) did not significantly affect CO-validation rates.

Stop Smoking Service related

Self-report quitters who set quit dates in January were most likely to be CO validated and November quitters were least likely. Clients who took varenicline, combination NRT or a mixture of medications were more likely to be CO validated than those who took single NRT. Self-report quitters who enrolled in open groups were more likely to be CO validated. CO-validation rates were lowest among specialist practitioners. Health-care assistants, pharmacy employees and unclassified practitioners were significantly more likely to CO validate clients than specialist practitioners (see Table 6).

Primary care trust and practitioner

Residuals suggest (Figure 4) that, after taking other demographic- and service-related factors into account, there was significant variation in PCT validation rates.

FIGURE 4.

Caterpillar plot of residuals for multivariable multilevel model predicting CO validation (model excluding extreme PCTs).

There is little that can be said with certainty about the higher-level variance, as it is not possible to tell how much of the change in variance between models (see Table 7) is owing to fixed effects in the model and how much is the result of level one variance being constrained to 3.29 in binary outcome models.

The variance between PCTs appeared to halve when PCT deprivation was entered into the model. The fixed effects suggested that PCTs that were fairly, but not the most, disadvantaged were three times more likely to be CO validated. This suggested that idiosyncrasies were responsible rather than an effect of deprivation. When extreme PCTs were excluded the variance between PCTs halved and there was no longer significant variation between practitioners.

Impact

In general, the services had a positive impact over and above what would be expected from quitting spontaneously or with medication only. When the impact was calculated using all self-report quitters, two services were estimated to have a negative impact as did six services when only CO-validated quitters were included. However, only one service was judged to have a negative impact when both CO-validated quitting and all self-report quitting were taken into account. At the other end of the scale, two PCTs were estimated to have independently added over 400 ex-smokers per 100,000 population between mid-2010 and mid-2011 (Table 8).

To represent quit rates, throughput and impact on the same graph (Figure 5), different scales were used. Thus, a point of 100 on the y-axis represents 100 extra ex-smokers per 100,000 population (impact), 1000 clients setting quit dates per 100,000 population (throughput) or a quit rate of 10%. The graph suggests that impact is more strongly related to quit rates than throughput. Two London PCTs displayed high quit rates but low throughput whereas three others displayed particularly high throughput but low CO-validated quit rates.

FIGURE 5.

Impact, throughput and quit rates by PCT.

Summary statistics for unique clients (as opposed to client records) and impact in general were also calculated. There were 171,830 unique clients (note that unique clients could not be identified for three PCTs) and 157,479 clients where quit rates could be calculated (i.e. did not quit in December or June). The CO-validated quit rate of unique clients was 34.4%, the self-report quit rate was 49.8% and the CO-validation rate for clients who self-reported as quit was 73.3%. There were 169,909 unique clients aged ≥ 16 years altogether. The CO-validated quit rate for these clients was 34.5% and the self-report quit rate was 50.0%.

Impact rate summaries and correlations between impact rates throughput and quit rates were calculated for 40 PCTs for adults aged ≥ 16 years. The distribution of client records and unique clients among the PCTs were skewed, and quit rates and impact measures were normally distributed.

The median number of client records per PCT was 3713 (interquartile range 2101–5136) and the median number of unique clients was 3385 (interquartile range 2169–4603). There were 92.0% (interquartile range 89.7–94.7) of client records that referred to unique clients.

The mean quit rates for unique clients aged ≥ 16 years in the PCTs used for calculations were 36% (CO validated) and 50% (self-report) (Table 9). On average 1487 unique clients set a quit date per 100,000 population and 184 ex smokers per 100,000 population were added. If only CO-validated quits were taken into account there were 152 extra ex-smokers and if all self-report quits were included there were 216 extra ex-smokers.

Overall, impact was significantly positively associated with CO-validated quitting, self-report quitting and impact, taking into account either only CO-validated quits or taking into account all self-report quits. Throughput was not significantly associated with quitting or impact, taking into account CO-validated quits but it was significantly associated with impact taking into account all self-reported quits and overall impact. Thus, PCTs with a better throughput did have a higher increase in the number of self-reported smokers in the population but such PCTs were not CO validating sufficiently to be sure that the number of ex-smokers is increasing with a higher number of clients.

Socioeconomic status

Results presented in the published multivariable analysis26 showed that affluent smokers were more likely to be abstinent from smoking at 4 weeks post quit date than disadvantaged smokers as measured by either eligibility for free prescriptions or by occupational group (as measured by the NSSEC), for example adjusted odds ratio (aOR) 1.38 (95% CI 1.35 to 1.42) for clients who paid for prescriptions compared with those eligible for free prescriptions. In total, almost 80% of the service clients received one-to-one counselling but open group forms of behavioural therapy were more successful [main effect aOR 1.26 (95% CI 1.12 to 1.41)] except among some of the most disadvantaged clients (prisons and long-term unemployed). Closed groups were little deployed and they were not significantly more successful than one-to-one behavioural therapy after controls. Practitioner type did make a difference for some clients, with all but the most affluent less likely to be successful if they had been treated by a nurse compared with other types of practitioners, including smoking cessation specialists [main effect aOR 0.73 (95% CI 0.65 to 0.83)]. More details are found in the published paper. 26

Summary of key points

• The estimated number of clients treated by SSSs from mid-2010 to mid-2011 was 5–10% of their smoking population.

• The self-reported and CO-validated quit rates were 49% and 34% respectively at 4 weeks post quit date.

• Smokers attempting to stop with NRT and minimal behavioural support have previously been estimated to have quit rates of 25% (for CO-validated quits) and 35% (for self-reported quits) at 4 weeks, so SSS interventions need to show rates higher than these and they should aim for rates of at least 50% for self-report and at least 35% for CO validated. 33,42

• The lower CO-validation rate was likely to be the result of the CO-validation rate of self-report quits being below recommended levels (74% compared with the recommended 85%). 33 This was, however, slightly better than that reported for all PCTs in 2009/10 (69%). 33

• The self-reported quit rate for all English SSSs from April 2010 to March 2011 was also 49%, with 70% of these CO validated43 compared with 74% in the current sample. Overall, 787,527 people set a quit date43 compared with the 202,084 client records in the QuitManager database used here. Thus, the North 51 PCTs that allowed their data for research purposes comprised roughly one-quarter of the total and have similar self-report quit and CO-validation rates. It must be noted that the self-report quit rate used was not as narrow as the DH quit rate regarding time of data collection but was more stringent in that it was an intention-to-treat analysis.

• Highest quit rates were found among older people, men and clients with higher SES. January was the month with the highest number of quit dates set and the most successful quitters.

• Affluent smokers were more likely to be abstinent from smoking at 4 weeks post quit date than disadvantaged smokers.

• Varenicline and combination NRT were both used frequently and increased the chances of quitting compared with a single NRT product.

• The majority (79%) of clients received one-to-one behavioural support. This type of support was significantly less successful than open rolling groups [aOR open groups 1.28 (95% CI 1.15 to 1.41) compared with one to one].

• Clients who saw specialist practitioners had higher quit rates than those who saw other types of practitioners.

• As a result of SSS treatment, the estimated number of ex-smokers per 100,000 population was 184 from mid-2010 to mid-2011.

Rationale for the second element: prospective cohort study

Stop Smoking Services in England are not required to routinely collect data on longer-term cessation outcomes. Only outcomes 4 weeks after a client’s quit date are collected and reported. Furthermore, the routine data collected by SSSs are limited in scope, and the quality and quantity of data collected varies between service providers. Consequently, there was a need for a prospective cohort study to collect long-term (12-month) follow-up data and more detailed and consistent information on client and service characteristics. Data from the prospective study were used to fulfil study objectives 3, 4 and 5 of the ELONS study, to explore the relationship between client characteristics, treatment characteristics and longer-term abstinence from smoking.

Recruitment

Stop Smoking Service clients were eligible to be recruited to the ELONS study if they were aged 16 years or over and were not pregnant. In order to compare behavioural support types, initial power calculations suggested that we needed data from quit attempts made by 5000 individuals, with at least 370 quit attempts within each behavioural support type with 90% power and an average quit rate of 15%. This quit rate was based on the previous evaluation of long-term outcomes of the English services. 18

Recruitment to the ELONS study was a multistage process:

• Secondary analysis of QuitManager data was conducted (see Chapter 3 and 4). The PCTs became the sampling frame of services. To achieve statistical power we initially approached services on the basis that they offered clients a range of behavioural support types. In order for our PCTs to be representative we also approached services with a range of short-term (4 week) quit rates and geographies (urban and rural areas, affluent and disadvantaged areas, and include areas from most regions of England).

• Once services were identified within this sampling frame, a member of the research team made contact with the service commissioner and service manager to invite them to take part in the study.

• After the service manager had agreed to participate, the research team obtained permission from the local PCT R&D department, and service support costs used to reimburse SSSs for staff time spent on the ELONS study were negotiated.

• While this process was under way, the team liaised with services to decide whether to include only specialists or additionally include level 2 providers.

• If it was decided that level 2 providers could be included, they needed to be approached individually (as each is a separate contractor, i.e. a pharmacy) to determine if they would take part.

• Once participation of a service (and if relevant its linked level 2 practitioners) was confirmed, a study briefing was delivered to smoking cessation practitioners to enable them to recruit clients to the ELONS study.

• After these elements were in place, stop smoking advisors recruited clients to the study. Inclusion criteria were all clients who were over 16 years old, were not pregnant and had set a quit date.

The recruitment process was aided considerably by the involvement of the PCRN staff, who helped us make initial contact with SSS managers and colleagues in R&D, promoted the study to level 2 providers, finalised service support costs and delivered study briefings. Two service managers (Northamptonshire and Rotherham) did not want their practitioners to be burdened with recruitment to the ELONS study or collecting the extra monitoring data. In these sites, PCRN-funded staff recruited clients, and collected client and treatment characteristics on paper forms. However, as PCRNs are organised by area and each have their own budget, the level of support available did vary across the nine ELONS study sites.

Recruitment challenges

Recruitment was a particular challenge for the prospective study. The initial hurdle was that the data collected in QuitManager did not consistently differentiate between behavioural support types. The main ambiguity was between one-to-one specialist practitioner support and level 2 support and, to a lesser extent, between one to one and drop-in. To overcome this, to some extent, the likely behavioural support range was checked with site managers when they were first contacted.

The second hurdle was securing the participation of SSSs as study sites. If services agreed to take part, this would involve SSS practitioners consenting clients to the study and collecting additional client and treatment data. This could be perceived as onerous in the context of limited appointment times and with local targets to meet. In addition, when we approached SSSs they were preparing for their busiest time of the year (post Christmas/New Year and No Smoking Day on 14 March. Moreover, local service funding was contracting in the face of NHS reforms and the move of public health from the NHS to Local Authorities in England (see Chapter 1). This led to some staff being made redundant or having their employment grade changed. Despite this, only three PCTs that the research team approached declined to participate in the study. The main reasons for declining were that they perceived available service support costs to be inadequate, it was an onerous consent process and there were factors related to service reorganisation.

Eight services were initially recruited to the study. From the numbers of clients attending these services recorded in the data set used for secondary analysis, it was thought that the target of 5000 participants (as set out in the original study proposal) would be achievable. However, the process of site recruitment took longer than expected (it was common for this process to take months rather than weeks): site managers needed to engage with colleagues and senior staff to explain what participation would involve before they could agree to take part; R&D permissions, liaison with local PCRN and service support cost calculation also were needed before a site could be confirmed as taking part and start recruitment. As a result of this, client recruitment started later than planned, after the busy New Year period had ended. In addition, the recruitment start date was staggered by site.

The third recruitment challenge was engaging with level 2 providers. As discussed previously, SSSs are delivered via specialist providers (their remit is purely smoking cessation) and level 2 providers such as GP practices, pharmacies and dentists (also called community providers) whose staff deliver smoking cessation in conjunction with their other responsibilities. Level 2 provision has grown over the past decade44 and thus it was important to involve these providers as well as specialists. Level 2 providers have different management arrangements to the specialist service, so each level 2 provider needed to be contacted separately. We therefore had to implement a recruitment strategy, which is summarised below (note there was some deviation between sites depending on the extent of SSS management commitment to the study, PCRN involvement and service support costs):

1. we identified which sites had level 2 service provision

2. we agreed with SSS managers whether or not we could approach their level 2 providers

3. where permission was granted, we publicised the study and letters were sent to all level 2 providers (via the SSS) to describe the evaluation and invite them to opt in by completing an ‘expression of interest form’ and returning in a reply paid envelope

4. to increase the number of providers, telephone calls were made by the research team and PCRN staff to practice managers and pharmacy managers/pharmacists from GP practices and pharmacies who had been active in the previous 6 months

5. a news article was added to the newsfeed page of QuitManager for each site, with instructions on how to register interest as a recruitment site

6. the research team and PCRN staff briefed the level 2 providers through lunchtime and early evening briefing events (with refreshment and buffet to encourage attendance) or individual face-to-face and telephone briefings for those who were unable to attend events.

Six sites recruited level 2 service providers. The number of individual providers recruited varied between 4 and 23.

The fourth challenge was obtaining informed consent. In order to secure PCRN help and to be able to reimburse SSSs for their time using service support costs, we asked the NHS Ethics Committee to classify the ELONS study as a research study rather than an audit or evaluation. This classification differed from that taken in ethical review processes for our previous national study of SSSs, conducted between 2001 and 2004. 18 In the earlier study, the National Institute for Health Research (NIHR) and PCRNs, etc., did not exist, there were no service support costs available and ethical requirements were arguably not as onerous. At that time we were permitted to merely add consent to take part in the research as a single question on existing service data collection sheets, but not for this study. For the ELONS study, the Research Ethics Committee required a consent form for each client with a section relating to each element of the study. Clients were asked to consent to each part. This procedure and the length of the consent form was undoubtedly a barrier to participation and SSSs reported that it resulted in serious time implications, which was the fifth recruitment barrier.

One of our study sites served as a within-study pilot and we tried out our recruitment methods there. This was Country Durham and Darlington, where we were able to estimate that the process of recruiting a client, seeking informed consent, asking the additional monitoring questions and collecting saliva samples added an additional 10–15 minutes to the appointment. This meant that the ELONS study added additional time to each consultation, which affected the number of people that the service could see. From a service manager perspective, this was of concern because they had local targets to meet (in terms of the number of clients they see and successful ‘quits’ at 4-week follow-up) with ongoing funding dependent on these targets being met. Some sites found this easier to manage than others, depending on how they delivered their service. Situations where there were particular problems included:

• Groups where the consent process and collection of baseline data had to be done for all group members. With limited staff, this could take up the majority of a session.

• One-to-one sessions where slots were doubled/triple booked because of a high did-not-attend rate in order to keep service busy. This meant the additional time required to recruit clients to the ELONS study was limited.

A sixth recruitment challenge was ‘buy-in’ from both the SSS practitioner and clients. The ELONS study mostly relied on practitioners to recruit clients and some were more confident and committed to this than others. This is an issue in any research study that recruits in routine practice. Some practitioners were genuinely interested in, and could see the value of, research and so made extra efforts to encourage their clients to consent to participate. Others were not so enthusiastic, or had more limited time. Client ‘buy-in’ was also an important issue for recruitment. Some practitioners, particularly pharmacy employees, reported that clients simply did not want to take part. Practitioner feedback suggests that the main reasons were that clients did not have time to go through the recruitment process (particularly for working people who scheduled appointments) or did not want to be contacted for follow-up (often older people).

A further challenge was failure to set quit dates. In the previous English longer-term evaluation,18 clients were included in a study only if they ‘set a quit date’ (i.e. gave the SSS a particular day when they were going to attempt to stop smoking). In some of the services, clients were recruited to the study when they registered with the SSS, which was sometimes before they set a quit date. Some of these clients never did set a quit date. Comparing the QuitManager prospective study extracts with the saliva samples database for the long-term NRT use study revealed further clients who had been initially recruited to the ELONS study (as they had provided saliva samples) but had been dropped from QuitManager owing to lack of progress with their quit attempt.

The last challenge was footfall. Several sites reported a general decrease in the number of people using their service during the study period. Analysis of the number of quit dates set over the recruitment period for the ELONS study (March 2012–March 2013) suggests that this was a reality. The number of quit dates set in SSSs in England for 2011/12 was 816,444 and in 2012/13 this has decreased by 11% to 724,247. 31 The decline continued after recruitment was completed, and much or at least some of this has been attributed to more smokers using electronic cigarettes (e-cigarettes) for cutting down or quitting rather than more established methods.

Strategies to address recruitment challenges

By late July 2012 it became clear that the client recruitment rate was considerably below the level that was expected from the previous English long-term evaluation, where up to 80% of clients in the two study areas had been recruited. 18 Various strategies were implemented to improve recruitment:

• We negotiated further assistance from the PCRN/Clinical Research Unit and we were able to boost the number of level 2 providers.

• A substantial amendment was submitted to the ethics committee requesting to simplify the consent form from six points, each of which the client was asked to initial, to three tick boxes. This request was approved (we initially asked for a move away from written to verbal consent which was rejected).

• In order to support and motivate sites, we encouraged practitioners who were recruiting well to share their approach with their peers who were performing less well. We also supplied fortnightly updates to site leads with a breakdown of recruits by practitioner name so that they could monitor performance and discuss with staff when appropriate. Practitioners were sent a newsletter and either a mug or a pen with the ELONS study logo to both thank them and act as a reminder to keep recruiting.

• A further two sites were approached to take part in the ELONS study. One of these, Hull and East Riding, agreed to take part.

• Clients who had agreed to be recruited to the ELONS study but where there was no record of quit date set were included in the analysis.

• We postponed the end of recruitment from the end of November 2013 to the end of January 2014 where sites were willing.

Sample

Despite efforts to boost recruitment, the challenges described above meant that our initial target baseline sample of 5000 (as set out in our original research proposal) was too ambitious. Therefore, in consultation with the chairperson of the NIHR Health Technology Assessment (HTA) Board and statistical practitioner we revised our sample size calculation to a more realistic target sample of 3000, which allowed us to maintain a sufficient level of power to detect meaningful effect sizes between the intervention groups (90% to 80% power).

The NIHR HTA agreed the revised sample size of 3000 and the study was also granted a 7-month extension to allow us to successfully recruit this number of participants.

The measures taken and the timing of measures are presented in Chapter 2 and Table 2.

Prospective study analysis

For the prospective study, seven stages of analysis were undertaken. These were:

• preliminary analysis of short-term quit rates using unweighted data

• comparison with the ‘all cases’ data set to develop weights

• short- and longer-term unweighted and weighted quit rates

• multivariable and multilevel logistic regression modelling of CO-validated abstinence at 4 weeks

• multivariable logistic regression modelling of CO-validated abstinence at 52 weeks

• adherence to treatment

• comparison of longer-term quit rates with those in other evaluations.

Analysis of short-term outcomes using unweighted data

This section describes the characteristics of the ELONS study clients, the types of support they received from services and how these were associated with stopping smoking in the short term (4 weeks after their quit date). Note: this material represents an early stage of work, using unweighted data, in order to inform further refining of the variables and multivariable analysis. Thus, these are not the main findings of our study.

Data were available from 3075 clients; there were, however, six participants that were excluded from this preliminary analysis because they were pregnant, leaving 3069 cases. Overall, at 4 weeks post quit date just over a half of study participants reported that they had quit smoking (56.6%). Most of these clients had their report of abstinence confirmed by CO validation – the 4-week CO-validated quit rate was 44%.

Note that chi-squared tests test only the significance of the greatest difference between categories: they cannot differentiate intermediate categories. For several variables, particularly those that were collected only for the ELONS study clients, it was the other/unknown category that was the least likely to quit. This was because clients who did not complete the ELONS study monitoring form often were those also who failed to engage properly with the service and many did not set a quit date. For these variables we therefore do not know whether or not the categories of interest differ significantly at this stage but we can assess if quit rates are in the expected direction.

Comparing the ELONS study and ‘all cases’ data sets

The final ELONS study sample consisted of 3057 SSS clients, once pregnant clients and clients whose advisor was unclear had been excluded. For the reasons described in Chapter 5, these participants represented a small proportion of those who accessed SSSs in the study areas during the recruitment period. The sample is therefore not representative of all service clients in those areas. This is important as the ELONS study was designed to try and draw some overall conclusions about the longer-term effectiveness of the support to stop smoking offered by services in England. As a first stage in the analysis, therefore, we conducted comparisons between the ELONS study data and that available for all service clients, drawn from the routine (QuitManager) data available for each site.

The ‘all cases’ database contained quit attempts with quit dates that took place at the nine study sites from March 2012 to March 2013 (the months where any ELONS study client set a quit date; n = 71,800). Quit attempts where the client was pregnant, under the age of 16 years or were a prisoner who attended the specialist service were excluded because these clients were not recruited to the ELONS study (n = 65,972). Additionally, to be consistent with the final ELONS study sample, quit attempts where the practitioner was unknown were also excluded (n = 65,937). Thus, 5% of qualifying quit attempts that were made in the nine study sites were included in the ELONS study. However, because the recruitment period for each site was variable, if we just look at periods when the specialists and level 2 providers were actually recruiting, 9% of quit attempts were included in the ELONS study. Recruitment rates were higher for the specialist service (14%) than the level 2 providers (2%) at least partly because not all level 2 providers took part and those that did recruited for different periods.

The ‘all cases’ database was used to develop weights in order to calculate quit rates generalisable to the nine services that took part in the ELONS prospective study. The frequencies of the variables included and response rates are provided in Table 19. The response rate was the number of ELONS study clients divided by the number of all cases clients multiplied by 100.

Clustering by service also had to be taken into account when producing final quit rates. This was because different locations offered different behavioural support types and some sites we were not able to recruit clients accessing some behavioural support types. The overall profiles of the ELONS study clients behavioural support types, however, strongly reflected ‘all cases’ behavioural support types. This is illustrated in Figure 6, which compares the behavioural support in each site the ‘all cases’ data set with the behavioural support types of clients recruited to the ELONS prospective study.

FIGURE 6.

Distribution of behavioural support by location in the routine monitoring data and clients recruited to the prospective study.

Short- and longer-term quit rates

The main results for the ELONS study are the short- and longer-term cessation rates achieved by participants in the prospective study. The CO-validated raw quit rate at 4 weeks was 44.1% but after weighting this reduced marginally to 41.2% (95% CI 34.9% to 47.9%). The raw quit rate in the longer term, at 1 year, was 9.3% but after weighting this reduced marginally to 7.7% (95% CI 6.6% to 9.0%). The weighted results are shown in Table 20. These also include the proportion of people who had relapsed to smoking at 4 and 52 weeks post quit date, and also those lost to follow-up. Participants who were lost to follow-up (22.6% at 4 weeks and 45.2% at 52 weeks) were assumed to be smoking, consistent with an intention-to-treat analysis.

We also examined differences in short- and longer-term cessation outcomes by key variables of interest, including client and service characteristics (Table 21). Quit rates in the short and longer term did vary between groups of clients with particular characteristics, and the type of support provided by the SSSs.

Adherence to treatment

It is likely that an important determinant of successful quit attempts is the extent to which service clients continue to attend sessions and also use stop smoking medication. One general term we can apply to describe these two issues (attendance and medication use) is ‘adherence’. Owing to concerns about reverse causality we could not include adherence in multilevel modelling. Instead, we tried to examine this issue in some depth. The two measures of adherence collected were number of sessions of behavioural support and number of occasions that medication was recorded.

As Table 23 shows, drop-off among the ELONS study clients from attending sessions was steady: a little over 10% of the sample did not return following each session. In addition, about half the sample were recorded as only taking medication on one occasion.

Adherence was much more strongly associated with cessation at 4 weeks than 52 weeks. This provides some evidence that reverse causality does exist: clients who had not quit at 4 weeks did not continue to attend sessions after 4 weeks. At 52 weeks differences were smaller but still significant for both number of sessions and medication taken.

Adherence to medication was explored further by taking into consideration the type of medication taken (Table 24). The majority of clients who took either only single NRT or only combination NRT were recorded only taking medication on one or two occasions. Similar proportions were recorded taking varenicline on one or two occasions and three or more occasions. Thus those who took varenicline were more likely to adhere to taking their medication.

The 4-week quit rates of clients who took all types of medication for three or more occasions were significantly higher than those who took medication on one or two occasions. Differences are marked: fewer than one-quarter of clients quit if medication was recorded only once or twice, whereas approximately two-thirds or more of clients quit if they took medication three or more times (irrespective of medication type).

Number of occasions on which NRT was recorded did not have a significant impact on quit rates but clients who took varenicline on three or more occasions were significantly more likely to quit than clients who took varenicline on one or two occasions. Quit rates of clients who took varenicline on three or more occasions were also significantly higher than the quit rates of clients who took combination NRT on one to two occasions or clients who took single NRT on three or more occasions.

If we consider what type of medication clients took at any point (Table 25), two-fifths of clients took varenicline and one-fifth took either only single or combination NRT. Clients who at any point took more than one form of NRT on one occasion (thus combination NRT) were significantly less likely to quit at 4 weeks. This is not consistent with evidence on the effectiveness of combination NRT and raises some questions about the recording of combination therapy use in the study. There were no significant differences at 52 weeks.

Clients who adhered to medication protocols (i.e. took medication for several weeks) have more chances to change medication. Thus, we also looked at medication recorded in week 1 because then there was no chance of dropping out through failure to quit. There were no significant differences in quitting detected at 4 or 52 weeks.

Comparisons between the ELONS study and other studies

How do the short- and longer-term results from the ELONS study compare with previous studies? Some limited comparisons can be made, although these come with a range of caveats (discussed further in Chapter 10) owing to the observational nature of the data. There are two previous pieces of research conducted by members of our team that employed very similar approaches. The first of these was the previous ‘national evaluation’ of SSSs in England that was conducted between 2001 and 2004,17,18 already mentioned in Chapter 1. This involved two areas of England – Nottingham and North Cumbria. The second study was conducted in Glasgow between 2005 and 2007 and compared a pharmacy-based SSS with another service providing closed-group behavioural support. 28 Both reported abstinence rates at 1 year.

As Table 27 shows, the previous English evaluation in Nottingham and North Cumbria had a higher CO-validated quit rate at 52 weeks (14.6%) than the ELONS study and a lower lost to follow-up rate (37.5%). The difference in lost to follow-up occurred at all stages (4-week follow-up, 52-week telephone and 52-week validation) but was highest for the 52-week telephone interview. The Glasgow evaluation had a lower CO-validated quit rate. This was primarily because of a much higher loss to follow-up at 4 weeks. These comparisons are discussed in more detail in Chapter 10.

Summary of key points

• The prospective study collected data from over 3000 SSS clients at 4 and 52 weeks after their quit date. Only a small proportion (9%) of all service clients in the nine study areas were recruited. To correct for this, weights were applied, drawing on an ‘all cases’ database created from routine data in the study areas.

• Recruitment rates were higher for the specialist service (14%) than the level 2 providers (2%) at least partly because not all level 2 providers took part and those that did recruited for different periods.

• In terms of smoking cessation in the short term, the raw quit rate at 4 weeks was 44.1%. With weighting this reduced marginally to 41.2%. In comparison, the quit rate for England from April 2012 to March 2013 was 37%. 31

• For smoking cessation in the longer term, the raw quit rate at 1 year was 9.3% but after weighting this reduced to 7.7%.

• Predictors of smoking abstinence at 4 weeks were:

  • attending an open group

  • attending SSS in the New Year

  • being older

  • being more affluent

  • having a lower dependence on tobacco

  • being determined to quit

  • having a higher well-being score

  • having support from a spouse or partner

  • having a social network not populated with smokers.

• Predictors of abstinence at 52 weeks were largely similar and included:

  • attending group behavioural support or receiving one-to-one support from a specialist practitioner

  • taking varenicline

  • attending SSS in the New Year

  • being older

  • being more affluent

  • having a lower dependence on tobacco

  • having a higher well-being score

  • having support from a spouse or partner

  • having a social network not populated with smokers.

• Taking stop smoking medication and attending support sessions (described here as ‘adherence’) was significantly associated with smoking cessation – more so at 4 weeks than at 1 year.

• Limited comparisons with previous evaluations of SSSs in England are possible. Quit rates for 4 and 52 weeks from the ELONS prospective study were lower than those identified in the previous national evaluation in England, but higher than a recent study in Glasgow that examined closed-group and pharmacy-based services.

Rationale

A questionnaire to measure client satisfaction with SSSs was developed by members of the research team in 2008 as part of the DH’s ‘gold standard’ monitoring and evaluation guidance for SSSs. 50 The questions were designed to assess the overall level of client satisfaction with SSSs, with additional questions to look at specific elements of the service, for example, appointment times, convenience of venue, type of support received, willingness to recommend the service to others and smoking status. A copy of the questionnaire is appended (see Appendix 1).

Stop Smoking Services are under no obligation to use a CSS, which means that there was no consistency of use among the nine ELONS study sites. Some administered the survey but had no resource to analyse or make use of the findings to improve their service; others asked some questions but not all. Thus, despite not being directly relevant to research objectives, the research team thought that by offering to administer, analyse and write up the CSS and feed findings back to sites, this might encourage sign-up from the selected SSSs. This, in turn, would boost the study sample and contribute to answering the research objectives. In addition, the CSS generated useful data, highlighting the value of the service provided to smokers trying to quit.

Recruitment, sample and analysis

The research team posted a self-complete questionnaire to all clients 4-weeks after their quit date was set (regardless of the outcome of their quit attempt). To encourage response, two reminder mailings were sent and a prize draw of £200 of high street vouchers was offered.

There were 1006 questionnaires received and the final data set had 996 cases (one was removed because of missing data and nine discarded because they had been completed twice). This means that one-third (33%) of the ELONS study participants completed this survey. Questionnaire data were entered into Excel and then transferred to SPSS (version 19.0) for analysis. Respondents were also given the opportunity to write additional comments about the service they received. These have been analysed thematically and placed in the relevant sections, with quotes to illustrate where relevant. As a result of missing data, the number of clients responding to each question will vary.

Findings

Client satisfaction survey findings and the ELONS prospective study data set

Satisfaction levels by three key variables from the ELONS study data set (CO-validated quitting, location and behavioural support type) are explored in this section. CO-validated abstinence from smoking (in the short term) was chosen in order to explore whether or not any of the satisfaction variables might point to facilitators for cessation. Location was chosen because some issues with satisfaction may be study site specific and so may be relevant for only some sites. Finally, behavioural support was chosen as one of the aims of the ELONS study in order to assess the effectiveness of various forms of behavioural support and establish whether or not the support that clients receive affects their satisfaction.

Some sites and behavioural support categories did not support analysis owing to insufficient respondent numbers and were merged into ‘other’ categories. Note that different sites offered different forms of behavioural support and recruitment was higher for some support types than others. The percentage of respondents from each location for each support type is tabulated (Table 32). Nearly two-thirds (63%) of GP practice respondents were from site 1 and 96% of open-group respondents were from site 6.

Response categories for the 5-point Likert scales were merged to enable analysis. Chi-squared tests were conducted between CO-validated quitting and all the satisfaction variables. If there was a significant difference between quitters and non-quitters (p < 0.05) then chi-squared tests were conducted for location and behavioural support types. Results are presented where p < 0.05.