Development and evaluation of an intervention providing insight into the tobacco industry to prevent smoking uptake: a mixed-methods study

Uptake and ever smoking

In the past 10 years there has been a steady decline in the number of students aged 11–15 years in England who report having tried smoking at least once, from 39% in 2004 to 18% in 2014. 9 The proportion of students who have ever smoked increases with age; in 2014, 4% of 11-year-old children reported ever smoking, increasing to 35% of 15 year olds. 9 In 2014, 3% of 11- to 15-year-old students smoked regularly (at least one cigarette a week), a decrease from 9% in 2004. 9 The proportion of children smoking regularly also increases with age, reaching 8% among 15 year olds. 9

Susceptibility to smoking

It has been suggested that initiation of smoking among adolescents is preceded by a shift in attitudes when young people begin to entertain the possibility that they might try a cigarette and no longer hold a strong cognitive commitment not to smoke. 10 This cognitive shift has been described as the development of a susceptibility to smoking. 11 There are no national data on susceptibility to smoking in England, although in one local study 27.2% of 11- to 15-year-old students were deemed to be susceptible. 12 In 2014, 26% of students aged 11–15 years nationally thought that it was OK to try smoking to see what it was like, and 10% thought that it was OK to smoke once a week. 9 Both figures have declined steadily over time, from 48% and 25%, respectively, in 2003. 9

Effectiveness of existing interventions

Existing studies have tested a variety of interventions, differing in theoretical approach, design, intensity and mode of delivery and utilising different outcome measures and follow-up periods in evaluation. Therefore, given the substantial heterogeneity between studies, evidence from systematic reviews and meta-analyses of this literature is difficult to interpret. Although there is no definitive evidence regarding interventions that do not work and should thus be avoided, there is also no clear evidence for what may be effective.

There is no conclusive evidence that any one theoretical approach or conceptual framework is superior to others in preventing smoking uptake. 15 Interventions that involve providing young people with information about the prevalence and consequences of smoking, enhancing their social competence (e.g. teaching skills to increase self-esteem and cope with stress) and addressing social influences (e.g. increasing awareness of peer and media influence and teaching refusal skills) have all been tested with inconsistent results in terms of their effects on smoking uptake. 14

In the UK the A Stop Smoking In Schools Trial (ASSIST) programme trains students in Year 8 (aged 12–13 years) as peer supporters who are able to intervene in their friendship groups to encourage non-smoking. A randomised controlled trial (RCT) of ASSIST, the results of which were published in 2002, found a significant reduction in reported recent smoking (in the past 2 weeks) up to 2 years later. 16 In the only other UK study, carried out in the late 1990s, a RCT in 52 schools in the West Midlands tested the effectiveness of a 1-year-long programme in which students in Year 9 (aged 13–14 years) received three classroom-based and three computer-based sessions to encourage them to refrain from or to stop smoking. 17 This study found no significant effect of the intervention on the proportion of adolescents smoking one or more cigarettes per week 12 months later.

There is good evidence that ‘booster’ sessions are useful in the months and years after the delivery of a main intervention to strengthen and maintain its effectiveness, and, therefore, NICE recommend their use. 15 Booster components that have been trialled in previous smoking-prevention interventions have included classroom-based sessions,18 tailored letters,19 telephone calls,20 videos21 and magazines. 22

Some young people may experiment with smoking intermittently over many years before they identify and report themselves as a smoker, whereas others show signs of dependence very early in their smoking career. 23 For this reason it is recommended that school-based interventions contain elements of both smoking prevention and cessation and that a school smoke-free policy should include efforts to promote local NHS stop smoking services to both students and staff. 15

What constitutes a cost-effective intervention?

There is limited evidence for the cost-effectiveness of smoking-prevention interventions in UK schools; studies carried out elsewhere are also of limited relevance here given differences in their populations and health-care and education systems. 24 ASSIST was judged to reduce adolescent smoking at a modest cost; the incremental cost of the intervention per student not smoking at 2-year follow-up was £1500 [95% confidence interval (CI) £669 to £9947]. 25

Who should deliver an intervention?

Existing studies have utilised a range of people to deliver anti-smoking interventions to young people, including teachers, school nurses, project staff external to the school and students themselves. However, there is no consistent evidence regarding which of these providers gives the best results, and it is impossible to compare across studies given the probable confounding by other factors such as the content of the intervention itself. A Canadian study carried out in the 1990s found that teachers and school nurses were equally effective in delivering a classroom-based intervention to students in Grades 6 to 8 (ages 11–14 years) and there was no differential effect based on whether staff were formally trained in organised workshops or were provided with self-preparation materials. 26 The ASSIST study trained students as peer supporters and, as noted earlier, found a significant impact on student smoking up to 2 years later. 16 This positive effect was not restricted to the students trained as peer educators but was also extended to the non-trained students with whom the peer supporters were intervening. The ASSIST intervention was most effective in schools in the Welsh valleys, and the authors attribute this to the existence of clearly defined, close-knit communities in which the trained students were able to diffuse new behavioural norms. This suggests that the use of peer educators may be less effective in more poorly defined communities. Peer educators may also be counter-productive among students who already smoke. Other studies in the USA and Europe not using ASSIST have identified a ‘boomerang effect’, whereby over the course of a study smoking prevalence increased more in those exposed to anti-smoking interventions than in controls who did not receive the intervention. 18,27

At what age should you intervene with young people to prevent smoking uptake?

Age is a strong predictor of smoking behaviour, and existing studies have intervened across the age spectrum from the earliest years of primary school28,29 to the later years of secondary school. 30 The majority of interventions have been aimed at young people between the ages of 11 and 14 years; ASSIST intervened with students in Year 8 (aged 12–13 years). 16 There is no conclusive evidence on the age at which it is best to start delivering school-based interventions to prevent young people from smoking, but Years 7 and 8, the years in which smoking experimentation and the prevalence of regular smoking begin to increase dramatically,9 are arguably a logical point at which to intervene.

How do other student and school characteristics influence the effectiveness of an intervention?

There is conflicting evidence over whether or not school-based interventions have a differential impact according to gender and, indeed, the direction in which this effect might operate. 15 Similarly, students’ ethnicities and social groups may or may not influence the effectiveness of an intervention. The ASSIST intervention was most effective in the Welsh Valleys, a deprived area, and, in particular, was effective in reducing smoking only among girls. 31 In contrast, an intervention with 13-year olds in the Netherlands was significant only among young people whose parents had high levels of education and who worked full time, and particularly among boys with high levels of parental education. 21 Previous work by the authors of this report32,33 has shown that young people are more likely to begin smoking if they attend a school at which smoking prevalence among senior students is high; therefore, school smoking prevalence may influence the effectiveness of an intervention. Students who are smokers are also more likely to be absent from school and less likely to engage fully with an intervention than non-smokers. 17

What do schools and teachers want?

An intervention must be acceptable to teachers if it is to be delivered as intended and have the hoped-for consequences. At present, the National Curriculum for Personal, Social, Health and Economic Education (PSHE) in England, although non-statutory, suggests that schools teach facts and laws about tobacco use and misuse, and the personal and social consequences of smoking for the smoker themselves as well as for others. 34,35 However, a wide variety of approaches to teaching this material is seen across schools and, although the Office for Standards in Education (Ofsted) concludes that the quality of PSHE delivery is generally improving, there are variations in coverage and quality. 36 In many secondary schools PSHE is taught by form tutors, although many of these may lack the training, skills and experience to deliver content effectively; only 22% of PSHE teachers have any relevant training, and half of secondary schools have no staff with continuing professional development accreditation in the subject. 37 Therefore, an intervention needs to be accessible to teaching staff with no subject-specific training. In addition, schools face an increasing number of demands on their time and, thus, any intervention must be suitable for implementation with little preparation. A process evaluation of the ASSIST intervention suggested that teachers are receptive to new ideas and ways of teaching the difficult topic of smoking prevention and cessation, but that this must fit in with a school’s existing ethos and organisation. 38 The inevitable added workload for teachers involved in trialling a new intervention means that it should arguably be delivered before students reach Year 9 when they have often begun work towards external General Certificate of Secondary Education examinations (summative school-leaving exams taken at the age of 16 years) and when teachers may be occupied with Standard Assessment Tests (progress assessments completed during Year 9) teacher assessments. 38

How can young people’s families be engaged to help prevent young people from smoking?

The NICE recognises that no one intervention alone will succeed in preventing the uptake of smoking among young people, but that a wider approach tackling individual, family, community and societal influences is needed. 15 The strong association between parental and child smoking9 suggests that the addition of a family component to a school-based intervention might help both to encourage young people not to smoke but also to encourage cessation among any parents who smoke themselves. A recent Cochrane review found there to be moderate-quality evidence from RCTs that family-based interventions can have a positive effect on preventing young people from starting to smoke. 39 The strongest evidence was for intensive interventions, delivered independently of a school-based intervention, which encouraged authoritative parenting (defined as showing an interest in and care for the adolescent, often with rule setting), although the limited numbers of studies and participants made conclusions difficult to draw for other types of interventions. 39

A promising literature is developing around the effectiveness of interventions to encourage parents to talk with their adolescent children about smoking. 40 Various approaches to prompt discussions between parents and children have been trialled, including the provision of pamphlets and quizzes,41 booklets,42 newsletters43 and postcards44 mailed to parents, homework activities for the child to complete with their parents,44 videos43 and financial incentives. 43 These have demonstrated some positive effects on the degree to which children report talking to their families about smoking, although their effectiveness in reducing smoking uptake remains unclear. As expected, interventions prompting communication may be most effective when the parent is a non-smoker. 40 Most of these studies were carried out in rural, middle-class areas of the USA and their relevance to the UK is not clear.

To our knowledge, no work has been undertaken with secondary school-aged children themselves to understand their views on whether or not, and how, their families should and could be engaged to support them to remain non-smokers. Focus groups with primary school-aged children after they had received a smoke-free homes intervention suggested that they were confident in talking to their parents about the issue, although the children themselves were not involved in designing the intervention. 45

What does existing smoking-prevention education look like in the UK?

To the best of our knowledge, there are no existing publications summarising the state of existing smoking-prevention education in the UK. A systematic-style literature and internet search conducted by a University of Nottingham medical student during the course of this study identified 19 smoking education resources available for use in the UK; these varied in content and delivery style, and, although they all received positive feedback from teachers and students, formal evaluation of effectiveness was lacking. A survey of East Midlands secondary schools also conducted as part of this project suggested that schools dedicated little time to smoking education (especially with older students) and concentrated on teaching about the health effects of smoking, and that a large proportion of staff leading sessions had not received any training in delivering PSHE. Further details on this work are included in Appendix 1.

Study design and participant recruitment

Focus groups with students in Year 7 and interviews with teachers were used to explore the acceptability and effectiveness of Operation Smoke Storm and to provide feedback to inform refinements and extensions. Separate focus groups for boys and girls were planned, as research suggests that smoking has become increasingly gendered, with the UK having one of the most pronounced gender differences in regular smoking behaviour in Europe (with girls having the higher prevalence). 53 Qualitative research with 15- and 16-year-old smokers in Scotland concluded that differences in smoking behaviours and attitudes according to gender could be attributed to individuals’ social worlds and relationships, interests, meanings attached to smoking and the role smoking had in dealing with the everyday experiences of being a girl or boy in her or his teens. 54 Therefore, gender-specific groups were used both to encourage honest discussion and to allow us to explore any potential differences according to gender. We aimed to run four focus groups in each school, two for each gender, with between 8 and 12 students in each, in line with recommendations for focus group size. 55,56 We also attempted to recruit all of the teachers who delivered the package in each school and the two PSHE teaching leads. However, recruitment remained flexible with the option to cease if recruitment targets proved difficult to meet but data saturation had been reached.

Parents of all Year 7 students in the two schools were sent a letter by the research team notifying them about the Operation Smoke Storm sessions and the associated evaluation. Parents were asked to return an opt-out slip if they did not wish their child to complete a questionnaire or participate in a focus group. Teachers explained the purpose of the focus groups to students after the first Operation Smoke Storm session. Those who wished to take part provided their name, gender and class on a piece of paper to the teacher. Students were told that up to 12 participants would be selected at random if more people volunteered than were needed. This sampling technique ensured that adequate numbers would be interviewed even if some individuals chose to withdraw or failed to attend, and would help individuals to feel sufficiently comfortable to contribute equally. In School 1 the research team randomly selected students from a list of volunteers’ names provided by the Head of PSHE; students from School 2 were randomly selected by the PSHE teaching lead.

All teachers who delivered Operation Smoke Storm, and the PSHE teaching lead from each school, were invited via e-mail to take part in a one-to-one semistructured interview (face to face or via telephone) after they had delivered all their sessions, with repeat invitations sent as necessary.

Focus group and interview procedures

Semistructured focus group guides (for students) and interview guides (for teachers) were developed to explore the acceptability and perceived effectiveness of Operation Smoke Storm (see Appendix 4). Given the limited time available for the focus groups and interviews, our choice of topics was driven by a need to gather responses to key questions in order to inform the refinement of the Year 7 sessions and development of the booster and family components. Focus groups considered students’ views of the Operation Smoke Storm sessions and their awareness of, and attitudes towards, the tobacco industry. Students’ were asked for their opinions on what a booster component and family component might look like and what they would be likely to enjoy and engage with the most. Examples of existing resources were used to stimulate discussions around these topics. As a potential booster activity, students were asked to consider the acceptability and potential effectiveness of playing a game (either paper-based, such as a board game, or an electronic game on a computer, tablet or mobile telephone) or producing their own short film, with the example of Cut Films57 being shown to students in their focus groups. The acceptability and potential effectiveness of printed materials and online resources were also discussed.

Interviews with teachers focused on the design, suitability and usability of Operation Smoke Storm, the extent to which it was an ‘off-the-shelf’ resource, how well delivery integrated with the school’s existing timetable and curriculum, and their suggestions and preferences for the content and format of the booster and family components. The interview guide for the PSHE teaching leads also sought information about how smoking education was usually taught in the school.

Before the start of the focus groups and interviews, it was explained to participants that their responses would not identify them and would be confidential, and that they had the right to withdraw at any point if they wished to. Following this, written informed consent was obtained. Focus groups took place in the schools, during school time. They were facilitated by JT or AT, supported by MB, LS or LJ, and lasted 35 minutes on average (range 27–50 minutes). Face-to-face interviews took place at the respective schools (conducted by JT or AT), and telephone interviews were conducted from a private room (by AT), at a time that was convenient for teachers. Interviews lasted 34 minutes on average (range 24–50 minutes). The focus groups and interviews were digitally audio-recorded to aid transcription.

Data analysis

Qualitative data were approached using a realist paradigm. As such, data were viewed as telling us about, in a fairly straightforward way, what the participants thought about Operation Smoke Storm, which subsequently helped to inform the development of the resource. Given that this was the first time that Operation Smoke Storm had been evaluated, this approach was deemed appropriate. An external specialist transcription company transcribed the digital audio recordings generated from the student focus groups and teachers’ interviews clean verbatim. Transcripts were then checked for accuracy (by JT and AT) and any personal identifiers were removed. Each was assigned a unique code that identified the school (1 or 2) and focus group (male or female) or teacher number. These data were analysed using the framework approach58,59 to examine emergent themes. To aid familiarisation, data from the first four focus groups and four teacher interviews were initially read several times by JT, AT and MB who independently summarised these data [using Microsoft Excel^® 2010 (Microsoft Corporation, Redmond, WA, USA)] and identified initial codes, themes and subthemes. This stage also allowed the researchers to identify any contradictory cases or any within- or between-group differences (according to school and gender). As the codes identified from both the focus groups and teacher interviews were similar (apart from an additional teacher theme about preparation to deliver the resource), data were analysed together. Initial codes, themes and subthemes were discussed between the researchers in order to reach consensus on an initial analytical framework. This framework was then applied and refined following analysis of the remaining transcripts by JT and AT. JT and AT met at regular intervals to discuss their respective coding decisions and continued to refine the framework as required. Data were then indexed according to the final framework using NVivo (QSR International, Warrington, UK) version 10 software and the transcripts were charted into matrices according to each theme to facilitate synthesis and interpretation. These charts were rich in terms of detailing the data that we had, which enabled us to map, interpret and report findings in a succinct manner. Data presented reflect the overall views of the participants from both schools and extracts from the transcripts are included to illustrate the main findings.

Resource flexibility

The overall impression from teachers was that more flexibility in the delivery of the resource was needed, primarily to accommodate different lesson lengths and needs of a class. In particular, some teachers felt that they needed to find more time to allow the student groups to discuss their opinions and to prepare properly for their presentations. To increase flexibility, a menu bar, back button and the ability to stop the countdown timer were added to allow swifter navigation and to enable the teacher to skip to sections of priority if they were not able to deliver the whole content in the time available.

Guiding students

Learning objectives were added in audio format at the beginning of each session and the accompanying teacher notes were modified to remind teachers to highlight these. Teachers were provided with written answers to all questions to which they could refer if students could not remember answers. In line with requests from teachers, an initial page was inserted in the teacher booklet providing background information about smoking and setting the scene for teachers about the importance of preventing youth smoking uptake. An overview of Operation Smoke Storm was also provided, as well as information on how to use the new navigation menu bar and suggested timings for shorter lessons. Also included in the teacher booklet was advice on the questions that students might ask and how they could respond if students got upset or were worried by any of the resource content.

Teachers had suggested that students needed more guidance on the group presentation that was prepared and delivered in the final session. The student handbook was amended to make it clear that the content of the presentation was more important than the format of delivery, and that the content should be based on their learning from Operation Smoke Storm. The score sheet students used to mark each other’s presentations was amended to match the assessment criteria, with the following specific question being added: ‘How well do you think the group’s presentation will convince people of your age not to start smoking?’.

Enhancing student engagement

After much deliberation the decision was taken to overdub the expletive (‘shit’) with the word ‘rubbish’ and similarly to replace it in the student handbook (in quotation marks to signify that this was a substitute for the original word). It was too costly to remove words such as ‘impotence’ and ‘menstruation’ from the video recordings, although the activities in the student workbooks were changed so that the students were not required to recall this information.

For a full description of the final Year 7 lessons, see Appendix 2.

Description of the final booster session

The booster component was designed to create classroom discussion and debate about tobacco industry practices that may be targeting young people. Specifically, it focuses on tobacco marketing strategies from the perspectives of a tobacco industry executive and marketing company, as well as a health campaigner, seen through the eyes of a teenager and reported directly to camera in the form of a social media blog.

The session begins with a video recap of the Year 7 Operation Smoke Storm lessons to remind students of what they did the previous year and to orientate teachers who had not delivered the Year 7 lessons.

This is followed by an introduction to a new character, Kiara, who explains on her online blog that she is going on work experience with a marketing company. The storyline follows Kiara into an encounter with a tobacco industry executive in a meeting in which she is asked to sit in. In this meeting the tobacco industry executive discusses with the marketing company ways in which they might improve the company’s public image. This meeting prompts Kiara to investigate more about tobacco and the way it is advertised to young people, including speaking to a health campaigner to find out his perspective.

Students are asked a series of questions relating to the tobacco industry at key moments in the storyline. They are prompted to write their answers in an individual workbook and then to discuss their views with the rest of the class.

There are also two further activities, which teachers can choose to complete if they have sufficient time. The first asks students to write a slogan for a billboard poster, advertising a fake cigarette brand, targeting young people. The purpose of the activity is for students to consider how tobacco companies may portray smoking to young people and raise their awareness of being targeted.

The second activity asks students to write a Tweet (Twitter, Inc., San Francisco, CA, USA; www.twitter.com) about the tobacco industry, sharing their opinion with peers. The aim of this activity is for students to consider what they have learnt about the tobacco industry and their personal thoughts and feelings about this.

For a full description of the booster session see Appendix 2.

Description of the final family component

The final booklet comprised 10 pages designed to stimulate discussions about smoking between parents and students at home. The booklet contains a series of informative and interactive activities, each of which covers a different smoking-related issue pertinent to young people. The booklet is intended to be given to students after their first Operation Smoke Storm lesson in Year 7 for them to take home and share with their parents. An introductory page acts as a reminder for the students of their mission as a secret agent, as well as an overview of Operation Smoke Storm for parents.

The first activity, Involve Your Family, is a repeat of quiz questions that students completed in class, ascertaining knowledge about areas such as the chemicals in cigarettes and the health effects of smoking. Students are encouraged to ask their family members to answer the questions to ascertain their level of knowledge and the subsequent level of seniority at which they might be recruited to a job in a tobacco company.

The second activity, Know the Industry, contains new information about the marketing practices that the tobacco industry has employed in the past and at present. The activity prompts students to think about how they might be targeted by the tobacco industry, to consider their feelings and to consider their families’ feelings about it.

The third activity, Supporting Others, asks students to give advice to other young people in various scenarios relating to smoking in which they might find themselves. It encourages students to discuss with parents or friends what they might say in these situations. Signposts to cessation support services were also included in the booklet.

A prototype of the family booklet was reviewed by two public groups (see Public and patient involvement) and their feedback was taken into account when finalising the resource. For a full description of the family booklet, see Appendix 2.

National Children’s Bureau

The NCB is a charity that aims to improve the lives of children and young people. 60 As part of its work the NCB runs a Young Research Advisors group, a group of 12- to 21-year-old young people who consult and collaborate on research projects relevant to children and young people. Members of the research team (AT and LS) led a meeting of 17 Young Research Advisors (aged 12–18 years) in June 2014. The aim of this meeting was twofold: to gain an insight into young people’s perceptions and levels of understanding of tobacco marketing in order to inform the level of detail to include in the booster and family intervention components, and to gather young people’s opinions on the acceptability of the family booklet to inform the refinement of the resource before piloting it in schools.

In the warm-up activity, each Young Research Advisor introduced themselves and was asked to give one reason why a young person might try a cigarette to encourage their thinking about the issues surrounding smoking. They were then split into three groups, facilitated by either AT, LS or a member of NCB staff, for the two main activities.

Nottingham Smokers’ Panel

The Nottingham Smokers’ Panel is a volunteer group of adults who are either active or former smokers, recruited from the local area. This panel meets face to face every 6 months to discuss and give opinions on different aspects of the research being undertaken by the UK Centre for Tobacco and Alcohol Studies (UKCTAS). The first half of the June 2014 meeting was devoted to exploring issues around preventing the uptake of smoking in young people. The aims of the session were to receive feedback from the group about the proposed take-home family booklet and to ascertain levels of understanding of the ways in which tobacco is marketed.

The session comprised, first, asking each member of the group to recall the age at which he or she had started smoking, the circumstances in which they had started smoking and their feelings around this. JT then presented evidence to highlight the age of uptake of and susceptibility to smoking in England and the importance of trying to intervene prior to this to prevent smoking experimentation. The rationale for Operation Smoke Storm was outlined and the premise of the intervention was described briefly. The aims, methods and initial findings of the research project were then summarised. The group were then asked about any discussions they had had about smoking when they were at school (either through formal teaching or with parents and/or friends).

Panel members then completed the same two activities as the NCB Young Research Advisors. The findings from the two activities from the NCB and Nottingham Smokers’ Panel meetings are presented together below.

Control population

The Nottingham School Smoking Survey collected data from students in Years 7–10 (aged 11–15 years) in spring 2011, 2012 and 2013 in several schools in Nottingham City and Nottinghamshire (although not all schools participated in every wave of data collection). The primary aim of this survey was to evaluate changes in young people’s smoking behaviour following the introduction of point-of-sale tobacco display legislation. 12,61 By mid-2013 data were available on current smoking and susceptibility to smoking in Year 7 and Year 8 for two successive cohorts of students (i.e. students who were in Year 7 in 2011 and Year 8 in 2012, and students who were in Year 7 in 2012 and Year 8 in 2013).

Changes to pre-planned analyses

Originally, we planned to use linked data to judge the potential effectiveness of Operation Smoke Storm by comparing the combined prevalence of ever smoking and susceptibility to smoking in Year 8 students in the two study schools, after students had received the booster session, with the combined prevalence of ever smoking and susceptibility to smoking in Year 8 students in schools that took part in the Nottingham School Smoking Survey, adjusting for smoking behaviour in Year 7 as well as for other relevant confounders. However, logistical problems, which became evident over the course of the follow-up period, meant that it was impossible to link Year 7 and Year 8 questionnaire data from all but a handful of study participants. In order to preserve students’ anonymity they were each given a cardboard folder containing questionnaires which were coded to enable linking (e.g. 1a, 1b, 1c to indicate the three responses for student 1). Students were asked to write their names only on their folder and not on the questionnaires, and the Head of PSHE at each school was asked to store the folders safely for the duration of the study period. However, at one school all folders were misplaced over the summer holiday period between Year 7 and Year 8 when building work necessitated the emptying of classrooms. At the other school, several PSHE teachers did not return their teaching group’s folders to the Head of PSHE for safekeeping. Given the very small number of questionnaires that could be linked, from one school only, the statistical power to model these linked data was severely limited. Therefore, with the agreement of the Study Steering Committee, this analysis was not pursued further. Instead, cross-sectional comparisons were pursued.

Statistical methods

All data management and analysis was carried out using Stata version 13 (StataCorp, College Station, TX, USA).

This study adopted a set of three questions which have been used previously to assess children’s susceptibility to smoking. 11 Students were classified as ‘non-susceptible’ if they answered ‘No’ to the question ‘Do you think that you will try a cigarette soon?’ and ‘Definitely not’ to the questions ‘If one of your best friends were to offer you a cigarette, would you smoke it?’ and ‘Do you think you will smoke a cigarette at any time during the next year?’. Students who answered ‘Definitely yes’, ‘Probably yes’ or ‘Probably not’ to either of the last two questions or ‘Yes’ to the first question were classified as ‘susceptible’.

Our primary analysis used logistic regression to compare the odds of smoking and susceptibility to smoking (as a combined outcome measure relative to non-susceptible, never smoking) after delivery of the booster session in Year 8 in the two study schools with Year 8 data from schools that participated in the Nottingham School Smoking Survey. In a secondary analysis the odds of ever smoking relative to never smoking (regardless of susceptibility) were compared between intervention and control schools. Initially, combined data from the two successive cohorts of students in control schools were used as the comparison in order to maximise statistical power. However, in a sensitivity analysis the comparison was repeated using only data from the most recent cohort of students in control schools. A multilevel framework was used to account for the clustering of students within schools, with the effect of school modelled as a random intercept. We could not account for clustering at other levels, such as teaching group or family, as there were too few observations in some clusters to permit model fitting. Results are presented as odds ratios (ORs), with 95% CIs and p-values. Given the exploratory nature of the study, we have not applied a correction for multiple hypothesis testing but, instead, have presented results with CIs and p-values in order to allow the reader to evaluate the findings fully.

This evaluation of Operation Smoke Storm was not a RCT and, therefore, student characteristics known to influence smoking behaviours were potentially unevenly distributed between the intervention and control groups. Therefore, the unadjusted logistic regression models were adjusted for potential confounders, with backwards elimination used to derive a parsimonious adjusted model. Confounders tested were students’ gender, ethnic group (coded as a binary variable owing to the low prevalence of non-white groups), whether one or more of students’ parents, siblings or friends smoke, whether smoking is allowed in the family home, and perceived academic performance. Finally, a measure of rebelliousness and sensation seeking was defined based on a series of four previously defined questionnaire items62 (see study questionnaires in Appendix 3 for full questions). Students who did not answer all four questions were excluded, and the median total score among students with complete data was used to define groups of low and high rebelliousness/sensation seeking.

Unfortunately, a comparable measure of deprivation was not available for students in intervention schools and for those who took part in the Nottingham School Smoking Survey. The survey used students’ postcodes to assign a deprivation score and quintile for their home address according to the 2010 Index of Multiple Deprivation (IMD). 63 Ethical requirements meant that personal identifiers could not be collected from students who took part in the evaluation of Operation Smoke Storm. Therefore, self-reported eligibility to receive free school meals was collected as an indicator of deprivation. A proxy indicator of deprivation for all intervention and control schools was created, considering students in the most deprived quintile of the IMD in the control schools and those who reported being eligible for free school meals in the intervention schools as deprived relative to all others.

Given the absence of linked Year 7 and Year 8 data, it was not possible to adjust for any differences in the prevalence of smoking at baseline in our comparison of Year 8 data. Therefore, ORs were also calculated to compare the prevalence of ever smoking and susceptibility to smoking in Year 7 between intervention and control schools (before the delivery of Operation Smoke Storm in intervention schools) to enable comparison with any differences in Year 8.

Unadjusted and adjusted risk differences comparing intervention and control schools were calculated; these were derived using the ‘adjrr’ post-estimation command in Stata, again using a logistic regression model but one in which clustering was accounted for by using survey commands (the adjrr command cannot be run for a multilevel model).

Power calculation

The work described here was a non-randomised comparison intended to provide preliminary evidence of the potential effectiveness of the intervention, and, as such, was not fully powered. The effect of the intervention would have to be large in order for it to be deemed significant based on data from just two schools. However, in order to justify progression to trial, an estimate consistent with the intervention having some benefit (i.e. an estimate consistent with a prevalence reduction greater than zero) was required. We therefore judged the intervention to be effective if the CI for the difference in prevalence between the intervention and control groups in Year 8 excluded the possibility of no change.

Our pre-study power calculation, based on estimates of the probable achieved sample size in intervention and control schools and the self-reported prevalence of ever smoking and susceptibility among Year 8 students, suggested that we would be able to estimate the difference in prevalence to within 6.6% (i.e. if the observed effect was 6.7% or greater, the CIs would preclude the possibility of no effect or a negative effect of the intervention). This effect size was consistent with the size of effect that a subsequent cluster RCT would be powered to detect, and in line with the size of effect used to power the ASSIST study. 16 A post hoc power calculation based on the achieved data indicated that we were able to detect the difference in prevalence to within 6.9%.

Students’ views on Operation Smoke Storm and their attitudes to smoking

Other quantitative data collected from participants across the study period provided an insight into students’ views of Operation Smoke Storm and changes in their attitudes towards smoking over time.

Year 7 and Year 8 students were asked for their overall impression of Operation Smoke Storm, and students who received both the original and revised versions of the Year 7 lessons (in phase 1 and phase 2, respectively) were asked if they had talked to family and friends about these. Year 7 students who received the family booklet in phase 2 were asked about their use of this resource.

At three time points (baseline, after receiving the original Year 7 sessions, and after receiving the Year 8 booster), students were asked to respond to Likert-scale questions to ascertain their attitudes to four smoking-related statements (see Table 4 for the full wording of questions). The mean and standard deviation (SD) were calculated for normally distributed responses and the median and interquartile range (IQR) for non-normally distributed responses. Mean and median responses across the three time points were compared using an analysis of variance (ANOVA) F-test or Kruskal–Wallis test as appropriate.

At the same three time points described above, students were asked if they agreed, disagreed or were not sure about whether or not it was OK to try a cigarette once to see what it is like, as well as to smoke once a week. Categorical responses at baseline and at the Year 8 follow-up were compared using a chi-squared test with data from Year 7 and Year 8 students, respectively, who took part in the 2013 nationally representative Smoking, Drinking and Drug Use in Young People in England survey. 64 Finally, students were asked if they thought that Operation Smoke Storm had made it less likely that they would ever try a cigarette.

Students’ views on Operation Smoke Storm and their attitudes to smoking

Overall, students liked the lessons that they received. The vast majority of Year 7 students in phase 1 said that Operation Smoke Storm was ‘very good’ (36.3%) or ‘OK’ (47.3%); these proportions were similar (31.9% and 45.2%, respectively) when the revised version was delivered to the new cohort of Year 7 students in phase 2. Following delivery of the Year 8 booster in phase 2, nearly three-quarters of students reported that the session was ‘very good’ (14.7%) or ‘OK’ (57.7%). Approximately two-thirds of Year 7 students reported having talked to other people about Operation Smoke Storm (Table 4).

A total of 16.7% of students reported that they were given the family booklet but that they did not take it home, and 21.7% reported that they did not receive the booklet at all. Of students who did receive the booklet and reported taking it home, 43.0% said that they showed it to their mother or another adult female, 21.5% reported showing to their father or another adult male, and 24.4% said that they did not show the booklet to anyone. Very few students reported that they completed the activities in the booklet with a parent or carer. The completion rate was highest (17.2%) for the activity on the first page but declined to approximately 10% for the activities on subsequent pages.

Students displayed some changes in attitudes towards smoking over the course of the study. Table 5 presents mean Likert-scale scores in response to four statements given to the same cohort of students at baseline, after they had received the Year 7 Operation Smoke Storm sessions in phase 1, and after the Year 8 booster session in phase 2.

There was a statistically significant difference in students’ responses to statement 1 at the three time points (p < 0.001). Scores increased between baseline and follow-up after the Year 7 lessons, and again between Year 7 follow-up and follow-up after the Year 8 booster session, meaning that students were more likely to disagree with the statement that ‘companies making cigarettes only try to attract customers over 18 years old’.

There was a statistically significant difference in students’ responses to statement 2 at the three time points (p = 0.030), although the absolute difference in scores was smaller than for statement 1. There was no difference in score between baseline and follow-up after the Year 7 lessons, but the mean score was marginally higher at follow-up after the Year 8 booster session, meaning that students were more likely to disagree with the statement that ‘companies that make cigarettes sell dangerous products, but still operate in a fair and decent way’.

There was also a statistically significant difference in students’ responses to statement 3 at the three time points (p = 0.034), although, again, the absolute difference in scores was small. A small increase in score appears to have occurred between baseline and follow-up after the Year 7 sessions, meaning that students were more likely to disagree with the statement that ‘smoking is not that serious compared with other drugs young people use’. There was no further substantial increase in score between Year 7 follow-up and follow-up after the Year 8 sessions.

Responses to the statement ‘nicotine in cigarettes is one of the most addictive drugs that people use’ were highly positively skewed. Scores were significantly lower (i.e. students were more likely to agree with the statement) after the Year 7 sessions than at the other two time points (p < 0.001).

Students’ responses to the questions ‘do you think it’s OK for someone your age to try a cigarette to see what it’s like?’ and ‘do you think it’s OK for someone your age to smoke cigarettes once a week?’ were compared with data for comparable age groups from the 2013 Smoking, Drinking and Drug Use in Young People in England survey64 (Figure 2).

FIGURE 2.

Attitudes towards trying a cigarette and smoking regularly. SDDUYP, Smoking, Drinking and Drug Use in Young People in England survey. 64

There were no statistically significant differences in attitudes towards trying a cigarette to see what it is like between students in the two study schools at the start of phase 1 and the nationally representative sample of Year 7 students from the Smoking, Drinking and Drug Use in Young People survey64 (p = 0.135). However, there were differences evident in Year 8 (p < 0.001). Although similar numbers of students responded that yes, trying a cigarette was OK, proportionally more students from the study schools did not know, and fewer were certain that it was not OK, compared with the national survey data. This might suggest that exposure to Operation Smoke Storm increased uncertainty about trying a cigarette among students who might otherwise have thought that it was not OK.

At baseline, marginally more students in the study schools did not know or thought that it was OK to smoke cigarettes once a week than in the data from the national survey (p = 0.020). However, the absolute proportion of students reporting ‘yes’ or ‘don’t know’ to this question was very small. In Year 8 there was no significant difference between students in the two groups (p = 0.076). From these results it does not appear that exposure to Operation Smoke Storm had any substantial impact on changing attitudes towards smoking cigarettes once a week.

Finally, students were asked if they thought that completing Operation Smoke Storm had made it less likely that they would ever try a cigarette (Table 6).

After the Year 7 lessons, approximately two-thirds of students thought that Operation Smoke Storm had made it less likely that they would ever try a cigarette, although this number was smaller after the Year 8 booster. This may simply reflect increases in susceptibility to smoking with age that might have occurred regardless of receiving the intervention.

Delivery of Operation Smoke Storm

The revised version of Operation Smoke Storm Year 7 lessons, plus the new family booklet, was delivered to all Year 7 students (aged 11–12 years) at School 1 only. Changes to the PSHE curriculum at School 2 precluded the delivery of the Year 7 lessons in this second phase of the project. The Operation Smoke Storm booster session was delivered to all Year 8 students (aged 12–13 years) in both schools. In School 1 the booster session was delivered in PSHE lessons, but changes in School 2’s PSHE curriculum meant that, for logistical reasons, the booster was delivered during science lessons by specialist science teachers.

Prior to the delivery of the sessions the research team provided a brief training session to teachers, which outlined how to access and navigate the Operation Smoke Storm resource, and described the planned research-related activities. In total, 350 Year 7 students across 14 classes in School 1 received the revised Operation Smoke Storm lessons, plus the new family booklet. Again, School 1 had shorter lessons (40 minutes per week), and, therefore, some teachers covered the material over more than three sessions.

Across the two schools, a total of 538 Year 8 students received the Operation Smoke Storm booster session (School 1: 333 students across 12 classes; School 2: 205 students across nine classes). One class in each school did not complete the booster lesson; at School 1 this was because a teacher deemed it to be too difficult for the academic ability of the students, and in School 2 this was attributable to lack of time. The shorter lessons in School 1 meant that teachers needed two lessons to cover the material (including the follow-up questionnaire), whereas School 2 had sufficient time to deliver the booster session during a single lesson (although the follow-up questionnaire was completed in a subsequent lesson).

Study design and participant recruitment

The accompanying qualitative evaluation comprised focus groups, paired student–parent interviews (to evaluate the family booklet) and teacher interviews. The same procedures were used to gain consent, select students for focus groups and invite teachers to interviews as were used in phase 1 (see Chapter 3). Four focus groups were planned for each participating year group at each school (two for each gender), with up to 12 students in each.

All Year 7 students in School 1 were given the family booklet to take home to their parents. The booklet was accompanied by a letter informing and inviting parents to participate in a paired student–parent interview to explore their views on the booklet. An inconvenience allowance in the form of a £15 high-street voucher was offered to all parents who participated in the interviews. Parents were asked to contact a member of the research team by e-mail or telephone if they wished to participate, and a suitable time and location (either a private room on school premises or in the participants’ home) for each interview was arranged. Up to 15 paired interviews were planned.

Focus group and interview procedures

Five separate semistructured discussion guides were developed (Year 7 focus groups, Year 8 focus groups, paired student–parent interviews, Year 7 teacher interviews and Year 8 teacher interviews). The Year 7 focus groups considered students’ views of the revised Operation Smoke Storm Year 7 sessions and their awareness of, and attitudes towards, the tobacco industry and smoking. Students were also asked about their views of the family booklet, how they used it, and, if they did not, their reasons for not doing so. The Year 8 focus groups considered students’ view of the Operation Smoke Storm booster session and their awareness of, and attitudes towards, the tobacco industry and smoking. Students were also asked of ways in which the booster could be improved to increase engagement.

The Year 7 and 8 teacher interviews focused on the design, suitability and usability of the classroom-based sessions and resources, the extent to which Operation Smoke Storm was an ‘off-the-shelf’ resource, how well delivery integrated with the existing timetable, the perceived effectiveness of the sessions at raising awareness of smoking-related issues, and suggestions for improving the sessions to increase student engagement. Teachers were also asked for their views on the design and suitability of the take-home family booklet and how it was used.

Paired student–parent interviews focused on students’ and parents’ views of the take-home family booklet, how it was used at home, if it triggered any smoking-related discussions, and suggestions for improving the booklet.

For the full focus group and interview guides, see Appendix 6.

Procedures during the focus groups and interviews were the same as those used in phase 1 (see Chapter 3).

Data analysis

Data were analysed using the framework approach,58,59 as employed in phase 1 (see Chapter 3). Data presented for Year 7 lessons and the family booklet reflect the views of participants from School 1, and data on the Year 8 booster session reflect the views of participants from both schools.

Process evaluation

In addition to the qualitative work described here, a process evaluation was conducted in phase 2 to assess if the intervention was being delivered in the intended way. Both Year 7 and Year 8 teachers were asked to complete a fidelity questionnaire to capture information on how they delivered Operation Smoke Storm, as it was felt that observing teachers directly may have altered the way in which they delivered the intervention and may have deterred schools from continuing to take part in the study. The results of this process evaluation are in line with the qualitative data presented below and, thus, for brevity are not discussed here in full. However, the questionnaire and a summary of the results are included in Appendix 7.

Teachers’ preparedness and delivery of Operation Smoke Storm

None of the Year 7 teachers had taught the original version of Operation Smoke Storm the previous year (during phase 1). Some teachers had previously covered the topic of smoking during science or PSHE lessons.

I’ve had to deliver stuff before . . . It was just general health and well-being things as part of PSHE days in my school last year through tutor period as a general thing.

School 1, Teacher 7

The provision of an ‘off-the-shelf’ resource once again appealed to teachers and meant that the preparation time required for most was minimal. Those who read the new introductory information in the teachers’ handbook found it to be useful to set the scene of the lessons. Many read through lesson plans and watched clips of the videos to get an idea of the resource and storyline before delivering the first lesson. A couple of teachers spent time looking over the resource in more detail and one of those who did not do this acknowledged that they should have.

I feel like I probably would’ve benefited from going through the entire thing myself before. It was made to sound like it was just pick it up, go, and it wasn’t that straightforward because after I’d done the first session I was like, ah, it felt a bit broken in me presenting it.

School 1, Teacher 5

The training session and/or spending time reviewing the resource beforehand helped most teachers to feel confident about delivering the sessions. Most of the teachers felt that their general knowledge about smoking was sufficient for the ability of Year 7 students, which also helped many to feel confident about delivering the sessions.

I looked through the materials, I kind of knew how they were going to work and I felt confident that I would be able to explain and talk about what’s needed in the lessons. I’m not a particular expert but I think general knowledge about smoking and its effects should be enough.

School 1, Teacher 9

The shorter lesson time in School 1 was not an issue for most teachers and they reported picking up from where they had finished in the last lesson. Although the lesson plans provided timings for activities, some teachers did not follow these timings exactly, adapting the pace as necessary for the students.

I followed the programme through, and went through session one, went through session two, etc., but I didn’t use the timing guides because I wanted to make sure that they asked the questions they wanted, and they got the information they wanted.

School 1, Teacher 7

Many teachers did not use or exactly follow the lesson plans in class, as they felt that the online resource was sufficient.

I think I more looked at it, as I say, at the start and then took it more from clicking through the software as we went along because it was pretty self-explanatory there and more adapting to the discussions that the kids wanted to have as opposed to going, we’ve got to spend this much time on each of them.

School 1, Teacher 4

Many teachers missed the additional discussion points and question-and-answer section added to the back of the teacher handbook. None of the teachers was aware that a username and password, which provided students with access to an out-takes video once they had completed the resource, were available.

To be honest, I didn’t even look that far [discussion points]. I didn’t know that was there. I would literally every week just open to the page that I needed.

School 1, Teacher 5

Raised awareness

The areas in which students’ awareness about smoking increased were the same as those reported in phase 1 (see Chapter 3). Many had some existing knowledge about smoking from primary school but the Operation Smoke Storm resource provided further insight into the content and harms of cigarettes. A number of students were shocked by the information and subsequently many said that this knowledge had strengthened their desire to remain smoke-free. Many of the students, however, misinterpreted one of the facts in the resource, that smokers are ‘sheep-like’, that is, they tend to be followers of the crowd. Instead, students interpreted this as meaning that smoking would result in looking like a sheep.

You look like a sheep when you smoke.

School 1, Year 7, Female

Many of the teachers felt that learning about the tobacco industry was eye-opening for many of the students and that they engaged with the resource and information it provided.

I thought it was really good and it got them thinking, you could see that some of the stuff, the factual stuff, was quite surprising and it got them thinking about the morals of the companies that are selling cigarettes . . . I think when they’re asking you questions about it, it shows that they’ve taken it in and are actually thinking about well, ‘What if?’.

School 1, Teacher 10

This was reflected in the way in which some students engaged in further discussions about tobacco industry tactics. In the focus groups, students were able to recall a number of tobacco industry tactics that they had learnt in Operation Smoke Storm, although some students needed prompting in order to do so.

Well we like split into groups and learned about different things, like how they sold cigarettes with like tricks and scams almost.

School 1, Year 7, Male

However, some teachers felt that not all students picked up on information about the tobacco industry as much as on information about the content of cigarettes or the harms of smoking.

I think definitely what stuck with them the most is that activity and what’s in the cigarettes and everything like that, and I think they were shocked by that, and that’s something that’s stuck with them.

School 1, Teacher 4

Although some teachers were hopeful that the resource had averted students from trying a cigarette, others were doubtful of its lasting impact or felt that other factors, such as peer pressure, could have a stronger influence on the uptake of smoking among students.

I think definitely would make them think twice. Sometimes ultimately if they’re going to smoke, they’re going to smoke and it’s a mind-set thing. The peer pressure probably is the biggest thing.

School 1, Teacher 6

Engagement with Operation Smoke Storm

Feedback from teachers on students’ engagement with the revised Operation Smoke Storm and their ability to access the resource was similar to findings from phase 1 (see Chapter 3). Teachers felt that students bought into the secret agent theme. Although some students said that they enjoyed the secret agent theme, a few thought this storyline was unrealistic and that the acting was too dramatic.

On the second bit, this girl runs out when the alarm was on and I just thought, ‘Wow, this is so over the top.’

School 1, Year 7, Female

Many teachers stated that timings were an issue, as the prescribed session length did not match the length of the schools’ lessons. This was felt to restrict the quality of delivery as some activities had to be shortened.

Sometimes we were a little bit pushed for time because obviously, our lessons are 40 minutes, 45 minutes in the afternoon, so some of the features we had to kind of skip through quite quickly to fit it all in.

School 1, Teacher 10

Some teachers felt that the lesson objectives added following phase 1 feedback were sufficient, but others said they would like to see a greater focus on them to support learning. Teachers suggested displaying the objectives on the screen, or at the top of each page in the student handbook.

Maybe just even having at the top [of each page of the booklet] like learning objective today is that you will know all the chemicals that are present within a cigarette. Really simple, but the kids will know and then we go, ‘OK this is going to go on your presentation so we need to know this.’

School 1, Teacher 8

Issues with navigating through the resource appeared to be alleviated by the addition of the menu bar, which enabled teachers to skip to different sections of a session. Some teachers had an issue with streaming the lesson over the internet in real time, although some were able to load the session in advance.

I’d loaded it the day before at home on my laptop and when I got on to it, it froze, and when I tried to reload it, it just wouldn’t have any of it. So that hindered the first lesson.

School 1, Teacher 5

Teachers’ preparedness and delivery of Operation Smoke Storm

Having seen the content of the resource either in the training session or during their own preparation, all teachers felt confident about delivering the sessions. However, one teacher, who had not delivered Operation Smoke Storm the previous year, would have liked to have known what students had learnt in Year 7.

I didn’t really know what they’d done before. Some of them could remember, some of them couldn’t, so that’s what threw me a little bit. The content, I was fine with. I had a flick through it so I was happy to deliver that but it was more, not really knowing what their prior knowledge was.

School 1, Teacher 12

Overall, teachers liked the flexibility and off-the-shelf nature of the lessons. A number of the teachers who delivered the booster session were not specialist PSHE teachers, although the more experienced teachers said the off-the-shelf nature of the resource meant that it was suitable for non-specialist staff to deliver.

You could definitely give it to a non-PSHE teacher to deliver, and they would be comfortable in delivering it I think, because it’s so well resourced.

School 1, Teacher 1

The teachers reported that, although the lesson plans provided useful timings and guidance, they did not need to use the lesson plans when teaching. This was because the plans were too detailed to use in class and/or the online resource was easy to follow.

You kind of didn’t need to use them too much once you got going. You kind of looked to it for the suggested timings but then once you’d gone through the resources you didn’t need the teacher notes too much because I think the resources are quite self-explanatory.

School 1, Teacher 2

Raised awareness

When prompted, students were able to discuss what they had learnt about the subtle and inventive tobacco industry practices that appear to encourage young people to try smoking. Some students showed an appreciation of why the industry might target young people.

If they target to young people and try and get to young people, then they will get more money, ’cause there’ll be more people getting addicted to it.

School 2, Year 8, Female

Students expressed their disapproval of the tobacco industry’s apparent persuasive tactics to attract young smokers.

I think that’s really bad [that young people might be targeted], because like we’ve been doing the Operation Smoke Storm to stop us, and then they’ve been trying to sell it to younger generations.

School 2, Year 8, Female

Many students felt that the booster session elaborated on information they had learnt in Year 7, particularly around the health effects of smoking and the topic of the tobacco industry.

I didn’t know about like all the effects until this year, and it’s just like, it just shows you what actually smoking does. It just opened my eyes a bit.

School 1, Year 8, Female

I thought it was better this year because it’s explaining how companies market more in detail, making you do slogans for it.

School 1, Year 8, Male

Some students, however, felt that there was insufficient time to cover the topic of the tobacco industry and that it was repetitive of the Year 7 material.

I thought it, kind of, a bit repeated what it did last year but it was a bit too short to explain all the other stuff that we just learnt this year. About how they’ve managed to sell them and how they managed to get them to sell very easily, like a bit more in depth, because it was a bit quick.

School 1, Year 8, Male

Some students shared facts that they had learnt about the health effects of smoking and the tobacco industry with family members. A few also reported using the information to persuade family members to stop smoking. Those who did not share any learning with their families said that it was because their parents were non-smokers so it was not of importance, or because their parents were smokers and there was no point or they did not want to start an argument.

Also at the time my stepdad, he was smoking, he still is but I didn’t want to treat him . . . because it would just end up in an argument, because he would just start an argument saying it’s his life, so I just decided not to.

School 1, Year 8, Male

Teachers said that, although the booster session helped students to appreciate the manipulative tactics used by the tobacco industry, they felt that not all students understood this message. For example, some teachers mentioned that some students were unable to use the new information presented in activities and instead used their existing knowledge, for example, when completing the activity in which they had to write a Tweet to discourage their peers from smoking.

So whenever they talked about like their Tweets for social media, they kind of went for, ‘You shouldn’t smoke, it’s bad for you. You shouldn’t smoke. Cigarettes have got all this stuff in them,’ so kind of the obvious stuff from it, but they then don’t take it that step further to think, like, should they be publicising it, yeah, taking that conversation a bit deeper.

School 1, Teacher 2

Many students mentioned that the new information they had learnt across the Year 7 and Year 8 sessions had strengthened their resolve not to smoke. Conversely, some students said the Operation Smoke Storm lessons may encourage experimentation with tobacco and they would still be interested in trying a cigarette; one student said they had not even considered this before completing the lessons.

Before I did Operation Smoke Storm I never thought of smoking but then when it, like here into my head, a couple of people saying, ‘Oh I might just try smoking,’ it kind of brings the idea to your head but then you realise you have to try and fight off that idea before it takes over.

School 1, Year 8, Male

Some also felt that the decision to try a cigarette would be more heavily influenced by other factors, for instance, peer pressure and role models or family members who smoke.

I’m not sure if once they hit Year 9, Year 10 or they’ve got the peer pressure or older brothers and sisters that are doing it, I’m not sure how much impact it [Operation Smoke Storm] will have in the longer term, but in the short term I definitely think it would make them think twice.

School 1, Teacher 12

Engagement with Operation Smoke Storm

Most students reported that they enjoyed the Operation Smoke Storm booster session, with it being viewed as interactive and informative, more so than usual PSHE lessons. However, there appeared to be a difference between some of the female and male students’ views of the videos. Many of the female students enjoyed that the videos were recorded in the style of a blog, which they could relate to, and felt that the storyline was more realistic than the Year 7 lessons.

This year is better because it kind of showed what actually could have happened and like what people actually go on to watch stuff, so like a blog.

School 2, Year 8, Female

A few of the male students, however, felt that it was immature and could not relate to it; one male student said that he would have preferred a male character which he felt he would have been able to relate to more easily.

In Year 7 it was good, when you enjoyed it, it made it more interesting than it would have been if it was just like on a blank PowerPoint^® [Microsoft Corporation, Redmond, WA, USA], but then in Year 8 I found it a bit patronising, in a way.

School 1, Year 8, Male

Many students would have liked to have spent more time on the activities in the lesson, as they felt that they were rushed at times.

I think we could have had like a bit more time to do it, because it’s only lesson in one hour. We only had like five to 10 minutes to do one activity. It was quite hard to get all of that done in one part.

School 2, Year 8, Male

In addition, a few students said that they would have liked the lesson to be more interactive and to involve less writing.

Maybe, like something interactive actually on the board. Instead of just like writing something, maybe make it interactive on the computers or something like that.

School 2, Year 8, Male

Some teachers and students also felt that the resource was too repetitive. Teachers and students suggested introducing some variety in the form of more interactive activities in order to increase engagement.

I think the thing that I think it would benefit from is some more types of activities, and that came across from the students going, ‘Miss I’m bored, we’re just discussing and watching, discussing and watching.’ . . . Like you could do a role play, you could give them scenario cards with characters of the people who . . . You know two of the characters in that, right you’re going to play that person.

School 1, Teacher 1

Students particularly enjoyed the slogan, Twitter and discussion activities, because it gave them the opportunity to share their opinion.

It was your own opinion and good. Like you can write about it and do your own opinion.

School 2, Year 8, Female

A few students also enjoyed the slogan activity because it helped them to realise how they might be targeted in real life.

I thought it was good how it told you, like, people try and make things sound really good and you can actually believe it because of what they put on a packet or whatever or how they make it look. You don’t really realise sometimes that actually what’s maybe inside is bad for you but just because of the packaging you think, ‘Yeah, it’s cool, it’s nice.’

School 2, Year 8, Female

Although the teachers in part agreed that this activity would raise students’ awareness of how they might be targeted by tobacco companies, they felt that this did not resonate with lower-ability students.

Do you know that little clip where the boss is being very subtle going, ‘oh do you use social media?’. And, ‘oh we could do brand placement. Oh but we’re not allowed to.’ And it was all very subtle . . .Yeah and do you know lower-ability pupils wouldn’t have got that. I think that would have confused them, where the other pupils it wouldn’t have.

School 1, Teacher 1

In some cases, teachers had to provide students with further explanations of why they were doing the activity or gave additional examples of how products are marketed to people [such as in the cases of Nike (Nike, Inc., Beaverton, OR, USA) and Red Bull (Red Bull GmbH, Fuschl am See, Austria)], in order for students to understand that they were learning about how tobacco companies might target young people. Teachers felt that this was necessary so that students did not confuse it with learning how to promote tobacco products.

The way I did it was I did some energy drink ones because just before that we’d done energy drinks as well. So I did, you know like ‘Red Bull gives you wings’ and we looked at that, we talked about how that made it a slogan that people . . . you wanted to buy it, but we did slogans in general, you know, like Nike, ‘Just Do It’ and things like that. So they understood what a slogan was and what the purpose of it was before they wrote it.

School 1, Teacher 12

As a result of the potential misunderstanding that she felt might arise, one teacher decided to not complete this activity with some classes.

We didn’t get to do the slogan either because our teacher didn’t think it was appropriate for our age. For year ten’s but not our age group.

School 2, Year 8, Female

I think for lower abilities in Year eight, they would be wondering why they were supposed to [be] advertising tobacco rather than trying to get into the head of . . . some of them just aren’t quite mature enough to be able to do that yet . . . I wanted the message to be clear to them that it was anti-smoking rather than, oh well sometimes it’s OK to say it’s good to smoke. So I just ignored that part.

School 2, Teacher 1

To overcome the problem, teachers suggested that making the purpose of the task clearer for students was important. Some also suggested including a prompt in the teachers’ handbook for teachers to use examples of how products are advertised to young people.

I think if you put in the resource discuss celebrities you know about, then you’re not naming anyone, but it gives a focus point for . . . Especially for lower-ability groups, you can talk about sports people, it’s really easy.

School 1, Teacher 1

Students also enjoyed giving their opinion in the Twitter activity. They also stated that it was enjoyable because it was familiar territory for them. Others, however, said that they found it difficult to complete because they found it difficult to think of something original, or they were not used to using Twitter and were confused about using 140 characters.

The Tweet one was obviously harder to do, because we had a certain amount of words and it was like hard to put smoking into words without, because normally you get, ‘Oh yeah, it kills’ and stuff, but you want to put something more, so it actually convinces people.

School 1, Year 8, Female

The Tweet could have been a bit less wordy, like some Tweets tend to be . . . I think we felt implied [sic] to fill out all 140.

School 1, Year 8, Male

To overcome this problem, teachers suggested that students might be given some more examples of Tweets, as well as more direction about what the Tweet needed to achieve (i.e. to try to persuade someone to not smoke, using new information about the tobacco industry rather than reverting to using existing knowledge).

They just stick with what they know so they’ve thought about everything before and had the discussions, but I think when it comes to that kind of stuff they do go back to what they know . . . maybe get them to be a bit more specific about what they have to write their slogans [referring to Twitter activity] about, like maybe it has to be a bit more targeted towards other things.

School 1, Teacher 2

In addition to these activities, some teachers also felt that lower-ability students generally struggled with the language used in the resource because it was too advanced for them. As a result, a teacher took the decision not to deliver the lesson to their lowest ability set.

Some of the language was quite high level, I thought. And for my higher groups they lapped it up, but when I looked at it, and I’d done it with higher groups, I made a decision not to do it with my bottom set. They are almost non-readers, very low ability pupils, but they would have struggled with this.

School 1, Teacher 1

Teachers suggested that differentiating the existing activities would enable teachers to provide students with an activity suited to their ability level. This was considered to be usual practice for schools in curriculum subjects.

What we do in a lot of subjects in this school, if we’re teaching we’d have maybe three levels of ability. So for the lower ability it would be a lot more . . . You know, why do you think people stop smoking, and there might be lots of different answers, circle the one that you think, rather than writing it down. So they’re not writing as much, or for them it will just be put your hand up and tell you teacher rather than writing it down.

School 1, Teacher 1

Yes because that’s what we do as teachers anyway, we have to differentiate so we don’t give them all the same task because not everyone is starting at the same point. Like [name] said, a choice of maybe two activities so the teacher can say, right this is the one that we’re doing, or which one do you think that you could do.

School 2, Teacher 1

Resources used in the development and delivery of Operation Smoke Storm

Two main components of resource use were considered in the calculation of the intervention costs. First, resources used in developing the intervention were identified, including those spent improving the original version of Operation Smoke Storm, as well as those incurred developing the booster and family components. Second, resource inputs required for delivering Operation Smoke Storm were identified.

A microcosting approach was used to calculate costs whereby each cost component was computed by attaching a unit cost to the quantity of resource used. 65 Unit costs were identified from study records and published sources. All costs are reported in UK pounds sterling in 2013/14 prices. An NHS and education perspective is adopted in the analysis and other methods are in line with the NICE Technology Appraisal Guidelines. 66 Numbers may not add up owing to rounding.

Collection of health economic data from students

Health economics questions were piloted with Year 8 students in phase 2 as part of the follow-up after the Operation Smoke Storm booster session. At this point, no decision had been made over whether progression to a fully powered trial was justified, and piloting these questions was undertaken as a learning opportunity to highlight any necessary changes to the survey instruments needed should the trial have progressed.

Following guidance on collecting data from children,67 the questionnaires utilised in this study were designed to be easy to understand, and difficult vocabulary was avoided. The format of the questions was short and simple and only particularly relevant questions were asked. The questions administered were broadly divided into two sections (see Appendix 3 for the full questionnaire). Receiving a smoking-prevention intervention may change students’ smoking behaviours and thus their health-care resource use. Therefore, data were first collected on the prevalence of smoking-related diseases and disease-related health-care resource use, including general practitioner (GP) visits, use of hospital care and use of medications. Ever smokers were also asked about their use of resources to help them stop smoking, including NHS smoking cessation services and nicotine replacement therapies.

The second part of the health economics questionnaire collected data on students’ health condition using the European Quality of Life-5 Dimensions Youth version (EQ-5D-Y; www.euroqol.org/eq-5d-products/eq-5d-y.html). 68,69 This survey instrument has been developed specifically for children and adolescents aged from 8 to 18 years, but to date has not been widely used in studies in UK schools. We chose to use the EQ-5D-Y as it is a cognitively simple tool, designed for self-completion, which provides a standardised, single-index measure of health status reflecting both physical and psychological health. Compared with other health outcome measures such as smoking status, EQ-5D-Y was used as a generic measure that can help decision-makers to make direct comparisons with other health-care programmes.

EQ-5D-Y comprises five questions, each of which relates to a different health dimension: mobility; ability to look after oneself; ability to take part in usual activities; having pain or discomfort; and feeling worried, sad or unhappy. 70 Each dimension has three levels of response (no problems, some problems, or a lot of problems) and the combination of three levels of five questions can define a total of 243 possible health states. In addition to the EQ-5D-Y questions, students were asked to rate their health on the EQ-5D-Y vertical, visual analogue scale (VAS) on which the end points are labelled 100 (best imaginable health state) and 0 (worst imaginable health state).

The results first describe the completeness of data collection for the various health economic survey items. This information is supplemented by feedback from the teacher interviews describing problems encountered in teachers administering, and students completing, the health economics questions. Collected data are then used to describe the health status and use of health-care resources among the study population over a 12-month period.

Resources used in the development and delivery of Operation Smoke Storm

Students’ health status

EQ-5D-Y

Figure 3 shows the distribution of EQ-5D-Y responses across the five dimensions among students who answered all five questions. The highest frequency of problems was reported on the ‘feeling worried, sad or unhappy’ dimension, followed by the ‘having pain or discomfort’ dimension. The distribution of responses was in line with those from a multinational study of the feasibility, reliability and validity of the EQ-5D-Y instrument. 69

FIGURE 3.

Distribution of EQ-5D-Y scores: frequency of reported problems.

Of the 395 students who answered all five EQ-5D-Y questions, 356 were never smokers and 30 students were classified as ever smokers (smoking status could not be ascertained for nine students). Table 14 summarises the number and proportion of respondents reporting problems in EQ-5D-Y dimensions by smoking status.

TABLE 14 - The EQ-5D-Y outcomes for ever smokers and never smokers

Dimension	Ever smoker	Never smoker	p-value for difference (chi-squared or t-test)
Mobility, n (%)
No problems	23 (76.7)	333 (93.5)	0.003
Some problems	7 (23.3)	22 (6.2)
A lot of problems	0 (0.0)	1 (0.3)
Self-care, n (%)
No problems	30 (100.0)	354 (99.4)	0.681
Some problems	0 (0.0)	2 (0.6)
A lot of problems	0 (0.0)	0 (0.0)
Usual activities, n (%)
No problems	23 (76.7)	322 (90.5)	0.033
Some problems	7 (23.3)	31 (8.7)
A lot of problems	0 (0.0)	3 (0.8)
Pain/discomfort, n (%)
No problems	12 (40.0)	245 (68.8)	0.002
Some problems	18 (60.0)	105 (29.5)
A lot of problems	0 (0.0)	6 (1.7)
Sad/unhappy, n (%)
No problems	14 (53.9)	228 (64.0)	0.037
Some problems	12 (46.2)	113 (31.7)
A lot of problems	0 (0.0)	15 (4.2)
EQ-5D-Y score, mean (SD)	0.75 (0.2)	0.86 (0.2)	0.004
EQ-5D-Y VAS, mean (SD)	79.96 (12.1)	84.66 (14.5)	0.097

Ever smokers reported significantly more problems in four of the five EQ-5D-Y dimensions. The biggest difference was observed in the pain and discomfort dimension, where 60% of ever smokers reported some or a lot of problems compared with 31.2% of never smokers (p = 0.002). The second largest difference was found in the worry/sad/unhappy dimension, where 46.2% of ever smokers reported some or a lot of problems compared with 35.9% of never smokers (p = 0.037).

The EQ-5D-Y results were converted into index scores using UK population values (based on the adult tariff for the EQ-5D, the best available option). 78 The overall mean EQ-5D-Y score was 0.85 (SD 0.2), which was almost identical to the mean EQ-5D score among the UK general population aged 16+ years [0.86 (SD 0.2)]. 79 Ever smokers had a mean EQ-5D-Y score that was statistically significantly lower (indicating a worse health state) than students who had never smoked (0.75 vs. 0.86; p = 0.004). To the best of our knowledge, no existing studies have quantified EQ-5D-Y according to smoking status in adolescents. However, the differences observed here reflect those seen among adults, where EQ-5D scores were highest among never smokers (and not dissimilar among ex-regular and light smokers) and lowest among the heaviest smokers. 79

The EQ-5D-Y scores were significantly higher (indicating a better health state) for boys than for girls (0.88 vs. 0.82; p = 0.0007) but were similar between students who were and were not eligible for free school meals (0.848 vs. 0.851; p = 0.9402). There was also no significant difference in score between students who reported exposure to second-hand smoking and those who did not (0.83 vs. 0.85; p = 0.444).

Figure 4 displays the distribution of EQ-5D-Y VAS scores among respondents, on a scale of the best health they can imagine (score 100) to the worse health they can imagine (score of 0). The median VAS score was 90 (IQR 80–95). Only 2.8% of students rated their health at a score of < 50. EQ-5D-Y VAS scores were significantly lower (indicating a worse health state) in ever smokers than in never smokers [median 80, (IQR 75–90) vs. median 90 (IQR 80–95); p = 0.015].

FIGURE 4.

Distribution of EQ-5D-Y VAS scores.

Collection and analysis of quantitative data

The conclusions regarding the potential effectiveness of Operation Smoke Storm are based on data from only two schools and may not be generalisable to schools more widely. As acknowledged in Chapter 5, the two study schools served a relatively more affluent, ethnically white population than all schools nationally. Although an attempt was made to select two study schools serving different sociodemographic catchment areas, limits were imposed by the nature of the schools within the study area and by the willingness of schools to participate. Smoking is associated with both deprivation and ethnicity9 and, thus, smoking behaviours may not be representative of the national picture. Similarly, it is possible that the effects of the intervention may be different in schools with a different population of students. Our conclusions also rely on self-reported data on smoking and susceptibility, as it was felt that biochemical validation of smoking status (e.g. through the measurement of exhaled carbon monoxide or cotinine levels in saliva) would be an unacceptable burden on schools already stretched in terms of demands on their time. In addition, the majority of young people of this age are not daily smokers and thus analysis of cotinine would not have been appropriate. However, students were asked to complete the questionnaires on their own and were not asked for any personal identifiers in an attempt to encourage honest responses.

All students in Year 7 and Year 8 at the two study schools received the intervention and, therefore, there were no internal controls with which to compare smoking behaviour. Data from an external source, the Nottingham School Smoking Survey, were used as a control. Although all schools in Nottingham City and Nottinghamshire were invited to participate in this survey, participation was voluntary. Again, control schools were not representative of all schools nationally with respect to deprivation and ethnicity. Data were also collected 1–2 years prior to the current study, albeit using identically worded questions.

The non-randomised comparison meant there were significant differences between the characteristics of students in intervention and control schools, and the unfortunate inability to link Year 7 and Year 8 data from study schools meant that it was not possible to adjust for characteristics as measured at baseline. Particularly relevant is the lack of adjustment for smoking status at baseline, although there were only small differences in the prevalence of smoking and susceptibility in Year 7 in intervention and control schools and so this is unlikely to be a major problem. In addition, there was no comparable indicator of deprivation available for both intervention and control schools. However, the proxy indicator tested was not a significant confounder in the adjusted logistic regression models, and any effects of deprivation may have been accounted for by the inclusion of other confounding variables (as measured in Year 8), such as parental smoking and smoking in the home. If the work were to be repeated, students could be asked to generate their own anonymous code using a set of pre-defined questions, responses to which would not change over time (e.g. birth date of parent, first letter of mother’s name) and write this on their questionnaire. If the same code were generated at subsequent survey waves, linkage of responses over time would then be possible.

The comparison of study students’ beliefs about trying a cigarette and smoking regularly with findings from the Smoking, Drinking and Drug Use in Young People in England survey64 is again limited by use of an external control and lack of adjustment for confounders. However, the survey used a multistage probability sampling design to select schools by geographical region (response rate 44% of selected schools) and students within schools (response rate 88% of eligible students) to give a nationally representative sample weighted to account for non-response. Responses are based on an achieved sample size of over 1037 students in Year 7 and 1049 students in Year 8.

Collection and analysis of qualitative data

Qualitative data were collected and analysed using standardised, transparent procedures, although, again, it is possible that the use of only two schools might limit transferability of the findings to other schools. However, the fact that similar themes were reported across these two schools, which had different characteristics, increases confidence that the findings are likely to be transferable and externally valid.

The focus groups had to be conducted during school time and, thus, lesson length at each school governed the time available for these. As a result, it was often not possible to follow up discussion points in more detail. Furthermore, in phase 2 the Year 7 focus groups took place in one school only, as the other school was unable to deliver the revised Year 7 resource. In addition, some of the Year 8 focus groups had only a small number of participants, which meant that there was a less than ideal group dynamic, making discussion more difficult. The character and questioning style of the focus group facilitators and interviewers might have influenced the responses received from participants, although the use of topic guides helped to standardise procedures, and the male and female facilitators alternated the facilitation of the male and female focus groups. The data collected were, however, similar across all focus groups, suggesting a minimal impact of these limitations.

The students, teachers and parents who took part in the focus groups and interviews were a self-selecting sample, which introduced a potential for bias. Those who did not take part may have given different responses to those who did choose to participate. For example, the students who volunteered may have been those who already had a strong commitment not to smoke, or who were most engaged by the resource and receptive to its anti-smoking message; these students’ responses may have given a biased impression that the resource was suitable for all students. However, students were forthcoming when discussing what they did not like about the intervention. Ideally, focus group participants would have been purposefully sampled based on not only gender but also smoking status and, potentially, other factors such as academic ability and exposure to family and peer smoking. However, this was deemed too difficult and would have required students to provide classification information that they might not have been happy to share.

The use of the framework approach provided a rigorous series of steps to follow during data analysis. This facilitated the process of using multiple coders to generate an analytical framework and the charting stage enabled comparison across and within cases to identify patterns in the data to aid interpretation. This method provided transparency in terms of the analytical processes employed and an audit trail to justify how decisions were reached. This transparent analytical process counterbalanced any potential for biased interpretation in favour of the intervention.

Involving external public groups in the study strengthened the research. The Nottingham Smokers’ Panel and NCB Young Research Advisors provided a fresh perspective on the family booklet, although, given that the NCB group was a mixed-age group, in which most participants were older than the Year 7 age group, it was difficult to know the extent to which their ideas would mirror those of the target age group.

Future work on how to prevent smoking uptake

Despite there being no evidence that Operation Smoke Storm is effective in preventing smoking uptake, there is scope for further work to address some of the limitations discussed above and to explore in greater detail the potential reasons for the lack of effectiveness, particularly given that both students and teachers generally enjoyed the intervention.

Operation Smoke Storm as it stands is probably not suitable for use with students much older than in the Year 7 and Year 8 groups studied here; some of these students even suggested that the storylines were too immature for them. However, the concept behind Operation Smoke Storm might prove effective if used as the basis of an age-appropriate intervention with older students. Older students might be more able to engage with subtle messages about industry influences on their behaviours, although, of course, it might then be too late if they have already started experimenting with tobacco. The applicability of the approach of Operation Smoke Storm to the prevention of other unhealthy behaviours on which there is industry influence, such as harmful drinking and unhealthy eating, might also be explored. It is also possible that an intervention such as Operation Smoke Storm, which focuses on one unhealthy behaviour in particular, might in fact lead to broader changes in young people’s awareness of the role of advertising, and thereby impact on other outcomes. Secondary outcomes, such as alcohol consumption and dietary choices, were not measured in this study, but a range of outcomes could be studied in any future work.

Alternatively, Operation Smoke Storm might usefully be adapted to include fully differentiated activities and resources appropriate for use with students of different academic abilities. If the active ingredient of Operation Smoke Storm is thought to be the content relating to the tobacco industry, this content needs to come across more clearly to students of all ability levels. The teachers interviewed did report that they adapted their delivery in order to suit their students, for example by extending or shortening activities, although exactly how they did this is not certain. It is possible, for example, that crucial discussion points were missed out. Direct teacher observation would be useful, providing that teachers do not change their delivery in response to an awareness of being observed. Longer focus groups and interviews, not limited by the time available in the school day, might also help to shed further light on how best to differentiate an intervention.

There is also scope to understand whether or not Operation Smoke Storm could be delivered acceptably and effectively in other settings. This may be beneficial in allowing the problems associated with the erosion of PSHE teaching time to be circumvented and might also allow the intervention to be targeted at particular groups of young people. Targeting the most deprived young people might, for instance, be more effective than delivering the intervention to everyone. A scoping exercise would be useful to understand the alternative settings, such as youth groups or sports clubs, that might be potential options, and the characteristics of the young people who use these.

The potential to deliver Operation Smoke Storm, or other interventions based on its premise, within schools but outside PSHE lessons might also be usefully investigated. There is a recognition that in some schools there is a false dichotomy whereby PSHE is seen as a discrete subject rather than something to be embedded across the curriculum, like other subjects such as literacy and numeracy. 85 Aspects of Operation Smoke Storm could potentially be delivered through subjects such as English, where learning to express ideas confidently and persuasively is a part of the curriculum, business studies, or art and design where the visual mechanisms used to market tobacco (such as package design and advertising) could be studied.

Operation Smoke Storm is suitable for delivery in a one-off session and thus might suit some schools in which PSHE is taught in ‘drop-down days’ rather than in discrete PSHE lessons. This mode of delivery could usefully be evaluated, as it might be that students’ learning is enhanced by condensed delivery and not having to recall information over a period of several weeks. However, there is recognition that drop-down days can be ‘tokenistic and ineffective’, particularly if students happen to be absent from school on those day, as they do not allow progressive learning and the reinforcement of knowledge over time. 85 In some schools, non-teaching staff (frequently school nurses) are used to deliver PSHE. Kick It has previously trained youth workers to deliver Operation Smoke Storm in London schools, and further evaluation of this would be useful.

Finally, further work is warranted to explore how to engage parents and guardians more in supporting their child to remain smoke free. The parents interviewed as part of this study, although a self-selected group, were supportive of the concept behind Operation Smoke Storm. Alternative formats to the booklet developed here might prove to be more effective in engaging parents.

These recommendations are primarily focused on understanding whether or not Operation Smoke Storm can be amended in such a way that it is then effective in changing young people’s behaviour. Of course, the alternative would be to accept that the intervention does not work, and from this might follow the suggestion that money should be spent elsewhere. However, this recommendation is difficult to make when the majority of health education resources used in schools have no evidence base and may in fact be far worse than Operation Smoke Storm, which is proven here to be ineffective but popular with students, parents and teachers.

Recommendations for conducting research in schools

On the face of it, schools appear to be an ideal setting in which to conduct research with young people. They provide a captive audience in which large numbers of the target population can be reached at any one time to deliver an intervention and to collect data for subsequent evaluation. However, informal conversations with local public health practitioners, as well as national reports on the state of PSHE teaching,85 suggest that increasing pressures on schools to reach academic targets mean that timetabling non-core subjects has become more problematic. Indeed, schools’ interest in, and ability to accommodate, the intervention and accompanying evaluation activities appears to be lower now than in 2001 when ASSIST was recruiting, although even then some teachers felt that ‘there seems to be one thing after another, after another’ in school life. 38 Curriculum time devoted to health and well-being, in which research such as this is often best situated, is being eroded. This was highlighted in one of the schools in which the amount of time devoted to PSHE was reduced between phase 1 and phase 2 of our study to such a degree that it was not possible to deliver the revised version of Operation Smoke Storm to the new cohort of Year 7 students and in which negotiations were needed to run the Year 8 booster session in science lessons instead.

Although the teachers involved in this study were committed to the research project, some did struggle to accommodate the demands of the research process. Although most teachers reported that the research processes were easy to follow and manageable, some did not return their teaching group’s folders to the Head of PSHE for safekeeping as requested, which meant that questionnaire data from Year 7 and Year 8 could not be linked. Heads of PSHE found it challenging to find space within the curriculum to deliver the intervention and to organise times for the student focus groups to run. Moreover, other demands on their time meant that not all teachers were able to participate in interviews or to complete fidelity questionnaires.

Recruitment of the two schools into this project was challenging and dependent on the level of support from key members of staff, such as the Head of PSHE. Difficulty in engaging with schools is further evidenced by a response rate of just 6.7% (13/195) in the survey of smoking education practices among secondary schools in the East Midlands (see Appendix 1). In addition, retaining schools was also difficult, particularly where key staff left their posts part way through the study and their replacements were less supportive of the project or less able to deliver commitments previously agreed.

The experiences gained in this study suggest that it is important to minimise the demands placed on schools and school staff as much as possible in order to maximise the likelihood of success in research in this setting. Ideally, any intervention to be delivered should fit within the curriculum time available and the burden of research activities, such as administering questionnaires and running focus groups and interviews, should be light, particularly where these can be conducted only during curriculum time. Schools should be fully informed of what will be expected of them over the duration of a study, although experiences here suggest that having a back-up plan is crucial should elements such as staffing or timetabling change. The importance of good communication with schools, highlighted in a process evaluation with teachers who took part in the initial part of ASSIST, has not changed. 38

The schools that took part in this study received an inconvenience allowance for their time, although this alone was insufficient to secure the participation of the four other schools approached when recruiting for this work. Other incentives could perhaps be explored, such as offering schools support in delivering health education, although, of course, the time commitment involved here has cost implications.

Conference presentations

Bains M. Development and evaluation of a novel intervention providing insight into the tobacco industry to prevent the uptake of smoking in school-aged children. Paper presented at the UK National Smoking Cessation Conference, London, 12–13 June 2014.

Taylor A. Development and evaluation of a novel intervention providing insight into the tobacco industry to prevent the uptake of smoking in school-aged children. Paper presented at the UKCRC Public Health Research Centres of Excellence, Leeds, 19–20 June 2014.

Szatkowski L. Development and evaluation of a novel intervention providing insight into the tobacco industry to prevent the uptake of smoking in school-aged children. Paper presented at the Society for Social Medicine, Oxford, 10–12 September 2014.

Taylor J. Operation Smoke Storm: qualitative evaluation of a novel intervention to prevent uptake of smoking in schoolchildren. Paper presented at the Lancet Public Health Science Conference, Glasgow, 19 November 2014.

Taylor A. Operation Smoke Storm: Effectiveness of a school-based smoking prevention intervention providing insight into the tobacco industry. Paper presented at the UK National Smoking Cessation Conference, Manchester, 11–12 June 2015.

Szatkowski L. Operation Smoke Storm: effectiveness of a school-based smoking prevention intervention providing insight into the tobacco industry. Paper presented at the Lancet Public Health Science Conference, London, 13 November 2015.

Journal articles

Szatkowski L, Taylor J, Taylor A, Lewis S, Britton J, McNeill A, et al. Operation Smoke Storm: qualitative evaluation of a novel intervention to prevent uptake of smoking in schoolchildren. Lancet 2014;384:S77.

Taylor J, Taylor A, Lewis S, McNeill A, Britton J, Jones LL, et al. A qualitative evaluation of a novel intervention using insight into tobacco industry tactics to prevent the uptake of smoking in school-aged children. BMC Public Health 2016;16:539.

Szatkowski L, Taylor J, Taylor A, Lewis S, McNeill A, Britton J, et al. Development and evaluation of a novel intervention providing insight into the tobacco industry to prevent the uptake of smoking in school-aged children. J Epidemiol Community Health 2014;68:A15.

Szatkowski L, Taylor J, Taylor A, Lewis S, Britton J, McNeill A, et al. Operation Smoke Storm: effectiveness of a school-based smoking prevention intervention providing insight into the tobacco industry. Lancet 2015;386:S70.

Delivery

The 19 resources reviewed were diverse in their modes of delivery and varied in the amount of lesson time that they required. Some resources could be used as lesson supplements (e.g. Puffing Poisons chemistry set92), whereas others would fill one or more whole lessons (e.g. Operation Smoke Storm). Resource providers included charities, NHS organisations and companies specialising in producing educational resources. A total of 10 resources were free of charge or available for free loan. The remaining nine required a one-off payment, ranging from £10.50 (Smoke Out) to £400 (Breathless TV). 88

Content

The 19 resources covered a variety of topics. A total of 13 resources covered the short- and long-term effects of smoking, including effects on appearance and sexual performance, health and the chemicals inhaled from cigarettes. A total of 12 resources explored the wider effects of smoking, including topics such as economics, the environment, legislation, the history of tobacco and its industry. Ten resources covered aspects of self-perception (such as self-esteem and confidence) and taught skills including personal reasoning, problem solving, teamwork, communication, dealing with peer pressure and conflict resolution.

Classroom suitability

A total of 15 of the 19 resources were designed to be delivered by teachers, with the remaining four designed for delivery by outside parties or peer educators. Two resources could be differentiated or adapted for different age groups and ability levels.

Role of students

Activities requiring active learning were present in 18 of the 19 resources, including discussion, role-play, group work, games, independent research, quizzes, use of practical models or equipment and problem solving. The smoking packs from White Ribbon97 took a more passive approach, being predominantly teacher-led with students watching and listening.

Relevance

Many of the resources reviewed endeavoured to be attractive to young people, and some had teenagers’ input in their design process. Capturing young people’s interest was achieved in a variety of ways, such as basing the resource on popular television shows (Breathless TV),88 video games, comics, fashion and films (The Smokescreen,98 Smokey Joe94), use of celebrities (BHF lesson plan90), use of humour (Choked Up)89 and product/website design (Smoke’s No Joke,93 smoking education resource box,96 Puffing Poisons chemistry set92). The Smokescreen98 encouraged students to ‘#unfollow’ smoking on Twitter. Resources targeted young people from 10 years of age to early adulthood, but 11- to 14-year-olds were the most commonly targeted age group. Some resources were targeted at specific subgroups; Attitudes, Skills development, Knowledge87 had a particular emphasis on vulnerable and at-risk groups and The Smokescreen98 was aimed at those who did not yet smoke. The oldest resources reviewed became available in 1992. A total of 11 resources were released in the past 4 years.

Effectiveness and acceptability

Feedback from students and/or teachers was available for seven resources, all of which was positive. Teachers commented on the importance of resources being engaging and able to capture and maintain students’ interest and encourage active participation in lessons. Students placed similar value on lessons being enjoyable and engaging. Six resources had undergone independent review, although only ASSIST86 had been evaluated in a fully powered RCT for its impact on smoking uptake.

Session 1

Session 1 begins with an introductory video, which introduces a female ‘spy handler’ who is working undercover, posing as a cleaner, at the R. I. Payne Tobacco Company. She gives students their mission: to gain access to R. I. Payne by posing as a job candidate at a recruitment day, get hired and go on to collect top secret information about the company. The scene cuts to the recruitment day at R. I. Payne, introducing the tobacco company recruiter and R. I. Payne himself [the Chief Executive Officer (CEO)]. The CEO explains that his company has been described as immoral, but that it makes money and that is what is important. He tells the job candidates (i.e. the students) that they must remain loyal to the company and not reveal any confidential information and asks them to sign a confidentiality agreement. At this point, the video pauses and students sign the confidentiality agreement in their paper workbooks.

The teacher then resumes the video and the recruiter plays the job candidates clips of interviews with previous applicants who did not have what it takes to work for the company. The recruiter poses questions to these applicants about moral situations in which they might find themselves if they worked for the company; these are based on real-world facts about the tobacco industry. For example, how they would feel about 100,000 people a year dying as a result of using their product? Back at the recruitment day, the recruiter then explains to the job candidates that these traditional interviews have been replaced by tests, which they are about to take. The scene cuts back to the spy handler reminding students that they need to gather as much information as possible to send back to spy headquarters. The activities that follow provide students with information about the content of cigarettes, the health effects of smoking and moral issues related to the tobacco industry.

Session 2

Session 2 begins with a video from the spy handler, who tells the newly recruited candidates (i.e. students) that she is setting up secret recording equipment. The video cuts to ‘induction day’ with the recruiter and CEO. The CEO gives a ‘know the law’ speech in which he acknowledges that cigarettes are ‘deadly’ but that people are free to smoke them. However, he explains that it is important for job candidates to know which laws work for and against the company, and learn how to sell and promote their products. This leads into an activity in which students learn the laws surrounding tobacco marketing, and how tobacco companies are able to sell and promote their products.

Session 3

Students are given some time to finalise their presentations before delivering them to the rest of their class. The spy handler records the presentations (in the video, presentations are not actually recorded) and using the ‘Spy Ratings’ page in their workbooks, students are asked to score each other’s presentations and provide feedback.

Once the presentations are complete, the spy handler congratulates students on collecting the information. However, she is intercepted by the CEO who takes the recordings she has made of the students’ presentation. The spy handler flees and later informs students that the recording the CEO took was a fake and that she still has their presentations. The video cuts to several weeks later, which shows that R. I. Payne Tobacco’s profits have declined and that it is closing down, after the secrets the spies collected were revealed to the press.

The teacher gives students a username and password to access a video of secret out-takes having completed the sessions.

Introductory page

Page 1 reminds students (and informs parents) that they are a secret agent working to expose the tobacco industry, and that their mission is to infiltrate R. I. Payne Tobacco by posing as a job applicant to gather top-secret intelligence about the company’s products and business practices. A comic strip with stills from the Year 7 lessons videos and excerpts from the scripts acts to remind students of the story so far.

Page 2 lists some of the key facts that students have managed to uncover so far in the lessons, and sets them a new mission – to read the booklet to reveal more secrets about R. I. Payne Tobacco and recruit their family to help.

Activity 1: Involve your Family

The first activity, Involve your Family, is a repeat of some of the ‘Know Your Product’ and ‘Know Your Customer’ quiz questions students completed themselves in class. Students are told that one of the other agents managed to capture images of the test cards, and they are instructed to ask their family members to fill them in too. A scorecard enables students to mark their family members’ answers so they can see to which pay grade and job type they would be appointed.

Activity 2: Know the Industry

The second activity, Know the Industry, introduces students and family members to tobacco industry practices. It presents quotations from real tobacco companies, providing evidence that they tried to target young people, quotations from R. I. Payne’s CEO relaying information about why they want to target young people and the challenges they now face to do so (such as advertising restrictions), an explanation of where cigarette advertising is banned, and details on how cigarettes are being promoted to young people, such as through packaging, the internet and television.

The accompanying activity asks students to consider if they have seen cigarettes or other tobacco products featured in music videos, and how they and their family members feel now they know they are being targeted in these more subtle ways.

Activity 3: Supporting Others

The third activity, Supporting Others, asks students to give advice to other young people in various scenarios relating to smoking in which they might find themselves. For example, students are asked how they would advise someone to say no to friends pressuring them to try a cigarette, what they would say if asked if it is OK to try a cigarette once, and how they might advise someone to help them give up smoking. It encourages students to discuss what they might say in these situations with their parents or friends.

Answers

The final double-page spread provides answers to the Involve your Family quiz questions and examples of what advice students might give in the Supporting Others activity. The back page includes web links if students wish to get involved in campaigns aimed at reducing the tobacco industry’s influence over young people (run by Kick It), and signposts to NHS smoking support services.

Activity 1

Students are asked to consider and write down in their paper workbooks why people start and stop smoking. They are then given some time to discuss their view with the class. The presentation then poses a rhetorical question to the students: ‘Is it possible that all of the reasons you’ve mentioned are cleverly influenced by sophisticated marketing? Kiara is about to find out. . .’. In the video that follows, students are introduced to the main character, Kiara, who is talking to camera through an online blog. She discusses her upcoming work experience placement at a marketing agency.

The video cuts to Kiara at her placement, where she is asked to make coffee and take notes for a meeting with an external client. The client is the boss of a company that makes cigarettes, who explains that his firm is looking for ways to improve its image following R. I. Payne’s demise and the potential introduction of plain packaging to cigarette packs. The marketing agency discusses with him ways to promote his company and its products without breaking the law. Suggestions include: arguing that plain packaging would increase illegal sales of tobacco and negatively affect small businesses; promoting the company’s products by covertly placing them in films or music videos; or using staff members’ personal social media accounts to make personal endorsements of the brand. At one point, the company boss turns to Kiara, asking if she uses social media and what music she is interested in. The video clip cuts back to Kiara in her online blog, where she discusses what was said in the meeting. She asks her friends to give their point of view.

Activity 2

The teacher asks students to recall the tactics the tobacco industry might use to promote its products by writing their answers in their paper workbook and discussing them as a class. Students are then asked why they think the tobacco company boss included Kiara in the conversation and why he was keen to find out who her favourite band is. Again they write their answers in their workbook and discuss them with the class.

Kiara returns in another online blog, where she explains that she is unsure about whether or not to continue with her work experience. She decides to visits a health campaigner at a nearby university to find out more about what she heard in the meeting. The video cuts to Kiara meeting the health campaigner. He gives her facts about the harms of smoking, the reason for tobacco companies wanting to target young people as new customers, and examples of how they are doing so. He counters the arguments posed by the marketing agency, explaining that introducing plain packaging would have greater benefits than not doing so. He tells Kiara that the best way for young people to learn about the dangers of smoking is by talking to each other about it. As a result, Kiara decides not to return to the marketing agency.

Activity 3

Students are posed a series of questions and asked to write down their answers in their workbook and discuss them with the class. For example, they are asked whether they think that the tobacco industry is indirectly encouraging young people to smoking by promoting smoking within adult popular culture, and whether they think that more laws would protect children and prevent them from starting smoking.

Activity 4

The next activity encourages students to consider how cigarettes are promoted. They are asked to think like a tobacco company and consider the tactics that they might use to target young people. Students are presented with an advertisement that might be used on a billboard (in a country where advertising such as this is legal), depicting young people having fun in the sun and branded with a fake cigarette company’s logo. Students are asked to write an inspirational slogan that could be used for this advertisement.

Activity 5

The last activity encourages students to use the information they learnt from health campaigner. They are asked to write a Tweet about the tobacco industry to share their current opinion with their peers and to convince others to not smoke. Finally, students are asked to consider whether Kiara was right to stop working at the marketing agency and discuss this with the class. What would they do if they were in her position?