Steps Towards Alcohol Misuse Prevention Programme (STAMPP): a school- and community-based cluster randomised controlled trial

Article history

The research reported in this issue of the journal was funded by the PHR programme as project number 10/3002/09. The contractual start date was in November 2011. The final report began editorial review in April 2016 and was accepted for publication in September 2016. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The PHR editors and production house have tried to ensure the accuracy of the authors’ report and would like to thank the reviewers for their constructive comments on the final report document. However, they do not accept liability for damages or losses arising from material published in this report.

Declared competing interests of authors

Harry Sumnall reports that his department has previously received funding from the alcohol industry (indirectly via the industry-funded Drinkaware) for unrelated primary research. David Foxcroft reports that his department has previously received funding from the alcohol industry for unrelated prevention programme training work. The sponsor university (Liverpool John Moores University) received and administered a payment from the alcohol industry for the printing of pupil workbooks in the Glasgow trial site.

Copyright statement

© Queen’s Printer and Controller of HMSO 2017. This work was produced by Sumnall et al. under the terms of a commissioning contract issued by the Secretary of State for Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK.

Scientific background and explanation of rationale

Adolescence is a period in which young people experiment with alcohol and establish use behaviours, and, as they age, the amount and frequency of use increases. 1 There is some evidence to suggest that earlier initiation of drinking is associated with later problematic misuse of alcohol (e.g. Bonomo et al. 2 and DeWit et al. 3), although systematic review has highlighted weaknesses in the evidence base for this. 4 The complexity of the relationship between earlier use and later problems is confounded by factors such as parental drinking and problem behaviours and/or behavioural disinhibition (e.g. Donovan et al. 5) and, accordingly, some authors have concluded that earlier initiation is better characterised as a marker of general risk proneness than as a causal influence. 6 However, there is stronger evidence to suggest that earlier age of self-reported drunkenness and the establishment of regular alcohol drinking is associated with a greater risk of adult alcohol-related problems. 4 Other acute and long-term consequences of heavy alcohol use in young people are evident, and these adversely impact on health, educational achievement, societal cohesion, criminality, welfare and well-being. 7,8 There are also clear geographic differences in the burden that alcohol places on the population, and these are closely associated with other major indicators of ill health and health inequalities (e.g. Public Health England9). Indeed, differences in alcohol use and the consequences of alcohol use are thought to be one of the major determinants of health and social inequalities. 10,11

Prevalence of alcohol use in the UK

The consumption of alcohol by those under the age of 18 years remains a public health concern in the UK. Evidence continues to suggest that, although the proportion of adolescents drinking alcohol across the UK has declined in recent years, those who do drink appear to be consuming more on each occasion. 12–18 Although this may be true at a national level, regional variations in drinking patterns also exist. 14,16,19 In comparison with the rest of the UK, drinking prevalence and excessive weekly drinking among adolescents has increased in Northern Ireland (NI) in recent years. 15 The results of the most recent Young Persons’ Behaviour and Attitudes Survey20 show that, of those who had ever drunk a full alcoholic drink (not just had a sip or taste), 56% had done so by the age of 13 years and 84% had done so by the age of 15 years. This is in comparison with 32% of 13-year-olds and 70% of 15-year-olds in Scotland reporting lifetime consumption of a full drink. 21 This does suggest a greater degree of alcohol use overall in NI than in Scotland (period prevalence). However, when comparing lifetime drunkenness in Scotland and NI, figures show that 39.3% of 11- to 16-year-olds in NI report lifetime drunkenness20 compared with 44% of 13-year-olds and 70% of 15-year-olds in Scotland. 21

Consequences of drinking

Adolescents are much more vulnerable than adults to the adverse effects of alcohol because of a range of physical and psychosocial factors that often interact (e.g. Newbury-Birch et al. 7). These adverse effects include (1) neurological factors due to changes that occur in the developing adolescent brain after alcohol exposure (e.g. Windle et al. ,22 Zeigler et al. 23 and Witt24); (2) cognitive factors due to the psychoactive effects of alcohol, which impair judgement and increase the likelihood of accidents and trauma (e.g. Rodham et al. 25); (3) social factors that arise from a typically high-intensity drinking pattern that leads to intoxication and risk-taking behaviour (e.g. Ellickson et al. 8 and MacArthur et al. 26); and (4) physiological factors resulting from a typically lower body mass and less efficient metabolism of alcohol (e.g. Windle et al. 22 and Zucker et al. 27). Physiological factors are compounded by the fact that young people have less experience of dealing with the effects of alcohol than adults, and that have fewer financial resources to help buffer the social and environmental risks that result from drinking alcohol. 28

Parental influence on young people’s drinking

Family factors are important in determining the nature and extent of adolescent alcohol use. These relate not only to the structure of families but also to family cohesion, family communication about issues such as substance use, parental modelling of behaviour (e.g. parental use of substances or rules on substance use), family management, parental monitoring/supervision, parent/peer influences and availability of alcohol in the family home. 16 For example, it has been argued that a trusting relationship between adolescents and their parents with open expression of ideas and feelings is an important factor in the reduction of health risk behaviours (e.g. Bahr et al. 29 and Riesch et al. 30). Moreover, parent–child communication processes have been proposed to mediate the effects of risk factors on problematic behaviour30 and better family communication processes have been shown to be protective against negative alcohol-related outcomes in young people. 31–35

The rapid escalation in the numbers of lifetime users and levels of use throughout adolescence is mirrored by the progressive detachment of adolescents from their parents and an increase in parental tolerance of adolescent drinking behaviour. 36 Although there are significant shifts in attachments of adolescents from parents to peers, there is still evidence that the influence of parents is considerable up to later adolescence and into early adulthood. 37 In a review of current evidence, Gilligan et al. 38 classified the environmental factors that determine adolescents’ propensity to engage in risky drinking as (1) social and (2) peer or family/parental. In the case of the latter, children are exposed to and learn about alcohol from an early age. 38 There has been much debate regarding the extent (if at all) to which parental tolerance of adolescent supervised drinking in the home, and by extension parental supply of alcohol to their children, can reduce heavier drinking and result in greater responsibility in terms of alcohol use. Young people’s drinking behaviours are said to be affected by their parents’ attitudes towards this behaviour and by parental supervision of their drinking (e.g. van der Vorst et al. 36), and parents often supply alcohol to their children, believing that it teaches them responsible drinking. 39 However, the risk arising from parental supply of alcohol is not well understood, and there is little evidence to support this as a harm-reducing practice. 38 In fact, although there is evidence suggesting that parental disapproval of drinking and limiting the supply of alcohol reduce adolescent drinking behaviour,36,40 some have suggested that parental supply of alcohol may reduce barriers to drinking, encouraging more frequent drinking and consumption of greater amounts of alcohol and even promoting a progression to unsupervised drinking. 41

Perceived parental approval of drinking has been linked to heavy drinking among high school and college students (e.g. Abar et al. 42). In support of the argument that permitting drinking at home promotes drinking in other contexts, van der Vorst et al. 43 reported that adolescents who were permitted to drink at home were also more likely to drink outside the home and to report more alcohol-related problems over a 2-year period than those who were not permitted to drink at all. In a survey of around 12,000 15- to 16-year-olds in the UK, Bellis et al. 44 reported that among those identifying any measure of unsupervised consumption, or heavy or frequent drinking, there was a significantly greater likelihood of alcohol-related violence, regretted sex or forgetting things after drinking. Furthermore, those reporting any measure of unsupervised consumption were also more likely to drink frequently and to drink heavily. 44 Livingston et al. ,45 in a 1-year follow-up of young women making the transition from high school to college, reported that those who were allowed to drink at home, either at meals or with friends, reported more frequent heavy episodic drinking (HED) at college, but those allowed to drink with friends reported the heaviest drinking episodes at both time points. However, in one Dutch longitudinal study, van der Vorst et al. 43 reported no differences in progression to problem drinking among young people whose parents provided high or low levels of supervision of alcohol use.

Universal interventions for preventing alcohol-related problems

Reviews of effective school-based universal alcohol prevention programmes for adolescents have failed to consistently identify interventions that are well designed, well implemented and properly evaluated (e.g. Jones et al. ,46 Foxcroft and Tsertsvadze,47–49 Nation et al. ,50 Faggiano et al. 51 and Spoth et al. 52). Foxcroft and Tsertsvadze,47–49 in their reviews of school-based universal interventions, were unable to recommend any single prevention initiative. However, one conclusion, which is consistent in all reviews, is that prevention interventions that effectively develop social skills appear to be superior in their impact to those that seek to enhance only knowledge (e.g. Foxcroft and Tsertsvadze,48,49 Nation et al. 50 and Faggiano et al. 51). In the absence of substantial evidence on particular programmes, guidance issued by the National Institute for Health and Care Excellence (NICE)53 in 2007 called for partnership working between schools and other stakeholders in efforts to prevent misuse. NICE also suggested that school-based interventions should aim to increase knowledge about alcohol, to explore perceptions about alcohol use and to help develop decision-making skills, self-efficacy and self-esteem. In family settings, universal prevention typically takes the form of supporting the development of parenting skills including parental support, nurturing behaviours, establishing clear boundaries or rules and parental monitoring. 47 Social and peer resistance skills, the development of behavioural norms and positive peer affiliations can also be addressed with these types of approaches. Most of the studies included in Foxcroft and Tsertsvadze’s 2011 Cochrane review47 of family-based alcohol prevention activities reported positive effects on behaviour and, although these tended to be small, they were generally consistent and persisted into the medium to longer term.

School-based alcohol education programmes in the UK for those aged < 18 years have predominantly been classified as universal, as they have been typically targeted at all pupils regardless of screened or perceived level of alcohol-related risk. 54 Outcomes assessed in universal prevention programmes have included those related to quantity and frequency of alcohol use (e.g. period prevalence, frequency of drunkenness, HED), as well as harms associated with consumption. 48 With respect to this last set of outcomes, harms can arise both from the actions of the drinker (e.g. accidents, health problems) and from the drinking of others (e.g. drunk driving, violence). Universally targeted alcohol prevention programmes (e.g. McBride et al. ,55 Newton et al. 56 and Vogl et al. 57) that aim to reduce harms associated with alcohol may, therefore, provide messages of harm reduction rather than focus on abstinence. In addition to aiming to reduce alcohol-related harm through reducing consumption, these types of programmes aim to reduce those direct and indirect harms reported by those recipients who continue to drink.

Introduction to the intervention components of STAMPP

The Steps Towards Alcohol Misuse Prevention Programme (STAMPP) combined a school-based alcohol harm reduction curriculum and a brief parental intervention that is designed to support parents in setting family rules around drinking. Chapter 2 (see Intervention) provides further information on the development, delivery and content of the intervention.

The classroom component of STAMPP is the School Health and Alcohol Harm Reduction Project (SHAHRP), which is an example of a universally targeted classroom intervention. It combines a harm reduction philosophy with skills training, education and activities designed to encourage positive behavioural change. 55,58 It is a curriculum-based programme delivered in two phases over a 2-year period, and is described by its developers as having an explicit harm reduction goal. The development of the SHAHRP is described by Farringdon et al. 59 It was originally developed in the 1990s in Western Australia, and the core components of the intervention were based on a systematic literature review of effective substance use education. The curriculum was written by practising teachers (with experience of developing student-centred learning approaches), with the assistance of research academics, and underwent piloting, evaluation and further development processes. Key evidence-based features of the programme include (according to the formative evaluation of the SHAHRP):59

• social inoculation (phase 1 of the intervention, delivered prior to alcohol initiation)

• relevance to drinking trajectories of recipients (i.e. phase 2 of the intervention, introducing harm reduction, is implemented after pupils are most likely to have initiated alcohol use)

• core intervention (phase 1) with booster sessions (phase 2)

• experientially focused and based on the drinking experiences of young people

• skills based with normative components

• incorporation of utility knowledge about alcohol use.

Includes specialist training of programme deliverers (e.g. teachers)

In the original Australian programme evaluation,55 which compared the intervention group with the control group receiving education as normal (EAN), the intervention group reported significantly less alcohol use: a difference in quantity of 31.4%, 31.7% and 9.2% at 8, 17 and 32 months after baseline, respectively; and significant differences in reports of hazardous drinking, defined as consuming more than two (female)/four (male) standard drinks (10 g of alcohol) per occasion, once per month or more often at 8 months (25.7%) and 17 months (33.8%) after baseline but not at the 32-month follow-up (4.2%). Intervention students also reported significantly greater knowledge at the 8-month follow-up, and this was maintained at the 20-month follow-up but not at 32 months. In addition, there was a significant difference between the study groups in the number of self-reported harms they experienced from their own use of alcohol after both phases of the intervention. This was maintained 17 months after the intervention but not at the final follow-up at 32 months. Finally, the intervention group developed significantly better alcohol-related attitudes (attitudes that supported less harmful alcohol-related behaviours) from first follow-up at 8 months, and this was maintained to the 32-month follow-up point.

A previous investigation of the SHAHRP utilising a non-experimental design was conducted in NI by some of the current STAMPP investigators,60 and found that, after appropriate adaptation (e.g. normative epidemiological facts updated, timings of lessons altered), participation in the SHAHRP was associated (across 32 months of follow-up) with benefits for pupils. Between-group comparison showed that intervention pupils reported significantly fewer alcohol-related harms over time, and, when drinking behaviour trajectories were modelled using latent class growth modelling, intervention pupils were significantly more likely than pupils receiving EAN to be members of those latent classes that reported less increase in drinking over time, that had a larger increase in alcohol knowledge and healthy attitudes, and that were more likely to report a smaller increase or no increase at all in alcohol-related harms.

The parental component of STAMPP was developed by the trial team and was based on earlier work by Koutakis et al. ,61 who found that giving advice to parents about setting strict rules around alcohol consumption reduced drunkenness and delinquency in 13- to 16-year-olds in Sweden (the Örebro Prevention Programme). The original Swedish intervention was based on empirical evidence that suggested that lower levels of youth alcohol drinking were associated with stricter parental attitudes against youth alcohol use and involvement in structured, adult-led activities. Similarly, permissive parental attitudes towards children’s alcohol use have been shown to be better predictors of offspring alcohol use than parents’ own use. 62 However, the original Swedish programme was relatively intensive (six 20-minute standardised presentations and discussion given to parents of 13- to 16-year-olds during regular school-based parent–teacher meetings) and so Koning et al. 63,64 adapted this intervention further (a single parents evening) and combined it with a school-based alcohol curriculum (the Dutch Healthy School and Drugs programme). They found that this combined intervention was associated with a significantly reduced rate of frequency of drinking or weekly drinking, and this was partly mediated by changes in parental rules and attitudes towards alcohol (i.e. stricter rules and attitudes were developed).

Explanation of rationale

Given the prevalence of underage drinking in the UK, the reported problems, costs and harms associated with this behaviour and the lack of a robust UK evidence base for universal alcohol prevention interventions, this work aimed to investigate an adapted form of the evidence-based SHAHRP55,60 in a culturally appropriate and curriculum consistent manner in the NI and Glasgow/Inverclyde post-primary school settings. Furthermore, considering the strong links between family behaviours and young people’s substance use, the effect of introducing a parental component to the core SHAHRP curriculum was examined.

Specific objectives

Chapter summary: introduction to the research

Although alcohol use in the general population of young people is falling in the UK, there are regional differences, and those who initiate use early, are regular drinkers or report early drunkenness are more likely to experience adverse outcomes or a greater number of years of ill health. The responses to young people’s alcohol use have traditionally focused on school-based educational approaches, although general population policies, such as restrictions on marketing and pricing increases, are also likely to affect consumption. However, the evidence base for school-based universal alcohol interventions (i.e. those that target a whole population, regardless of level of risk) is weak and, although some skills-based approaches have been shown to produce changes in different indicators of alcohol use, effect sizes are often small and the longevity of the intervention effect is limited. Other research has shown that family factors are an important determinant of young people’s alcohol use. For example, in those families in which there is good communication and authoritative rules on alcohol are in place, young people are less likely to drink. In keeping with the literature on school-based interventions, however, there are few family-based programmes that have found significant reductions in indicators of alcohol use and alcohol harm.

This research sought to determine the effectiveness of a programme, STAMPP, that is a school-based alcohol harm reduction curriculum with a brief parental intervention designed to support parents/carers in setting family rules around drinking. We examined whether or not STAMPP was effective in reducing HED and self-reported harms related to recipients’ own use of alcohol. The programme rationale was that stricter parental/carer rules and attitudes towards alcohol would reinforce learning and skills development in the classroom. The classroom component, SHAHRP, was a universally targeted curriculum that combines a harm reduction philosophy with skills training, education and activities designed to encourage positive behavioural change. It was delivered in two phases over a 2-year period and was adapted from an original Australian programme in an early study,55 with the assistance of education and prevention specialists. The brief intervention delivered to intervention pupils’ parent(s)/carer(s) comprised a short, standardised presentation delivered at specially arranged evenings on school premises. The presentation included an overview of the Chief Medical Officer (CMO)’s guidelines for drinking in childhood,65 information on alcohol prevalence in young people, corrected (under)estimates of youth drinking rates and highlighted the importance of setting strict family rules around alcohol. The presentation was followed by a brief discussion on setting and implementing authoritative family rules on alcohol. All intervention pupils’ parents/carers were followed up by a mailed leaflet, whether or not they attended the parents’ evening, which provided a summary of the key information delivered in the evening and coincided with phase 2 of the classroom intervention.

Trial design and setting

A cluster randomised controlled trial (CRCT) in NI and Glasgow/Inverclyde Education Authority areas in the UK, with schools as the unit of randomisation.

Sample and participants

The sampling frame comprised all mainstream post-primary schools in NI (n = 208) and Glasgow/Inverclyde (n = 36). All schools in the sampling frame were initially assessed for satisfaction of the inclusion criteria and willingness to participate in the trial. Schools in the Eastern Health Board of NI, which included the capital city Belfast, were excluded, as the classroom component, SHAHRP, was already being delivered to some schools in that area by a non-governmental organisation independently of the trial.

Male and female students attending mainstream secondary schools in NI and Glasgow/Inverclyde were included. Schools were randomised into the trial and baseline data were collected when pupils were in school year 8 or S1, and the intervention was delivered when pupils were in school year 9 or S2 in the academic year 2012–13 and aged 12–13 years.

Pupils not in the specified school year and age group, and pupils in non-mainstream and vocational education (e.g. pupil referral units, further education colleges) were excluded. Pupils with special educational needs in mainstream classrooms were excluded at the discretion of teachers, as the intervention materials had not been developed for use with this population.

In each participating school, all students in attendance at the time of data collection were asked to complete the project questionnaires. Questionnaires were administered to participants at the baseline time point (T0) in June 2012 and at three follow-up points: T1, T2 and T3.

Intervention

Students in the intervention condition received the SHAHRP,55 as previously adapted and tested for use in NI. 60 The parent(s) or carer(s) of intervention condition students were also invited to receive a brief intervention. All intervention pupil parents/carers, regardless of attendance at the parents’ evenings, were mailed a follow-up information leaflet.

Phase 1 of the NI SHAHRP classroom curriculum consisted of six lessons (with 16 activities) delivered to students in school year 9 or S2 (aged at least 12 years) by trained teachers. Phase 2 consisted of four lessons (with 10 activities) delivered in school year 10 or S3 (aged 13–14 years) by trained teachers.

Training sessions for teachers took place annually in a neutral location and included an introduction to the concepts involved in alcohol harm reduction, rehearsal of delivery of each of the sessions in that phase of delivery and raising of awareness of potentially difficult issues/areas. Teachers were trained by the STAMPP trial manager prior to phase 1, and by the STAMPP trial manager, STAMPP research assistant and an alcohol worker from a local third-sector organisation before phase 2. Phase 1 was delivered between September and December 2012 and phase 2 was delivered between September and December 2013. Curriculum activities incorporated various strategies for interactive dissemination including delivery of utility information, skill rehearsal, individual and small group decision-making, and discussions based on scenarios suggested by students. There was a particular emphasis on identifying alcohol-related harms in specific scenario-based exercises (e.g. a night out) and specific discussions on strategies that might be employed to reduce harms.

Phase 1 lessons broadly examined myths about alcohol, reasons why people drink or do not drink alcohol, alcohol and the body, the relationship between amount consumed and behavioural consequences, alcohol and the media, and real-life situations. Phase 2 lessons focused on more specific adolescent drinking behaviours, real-life scenarios or potential experiences while in an environment in which alcohol is consumed. These lessons specifically examined peer pressure, similarities and differences between males and females in a drinking context, drink spiking, responsibilities towards friends, grading of risk environments or situations and peer advice around alcohol.

Interactive involvement was a key feature of the lessons, and a workbook and compact disc (CD) accompanied both phases of the project, allowing for more active learning. Further details of the SHAHRP curriculum used in this study can be found elsewhere. 60 However, an important difference between the present study and that of McKay et al. 60 is that the pupils in the present study were 1 year younger at both intervention stages. The targeting of younger pupils in the current trial (i.e. ages 12–13 years) was justified on the basis of survey data suggesting that the median age of initiation of alcohol use was < 13 years. 20,21 In addition, the intervention was only delivered by teachers in the current study, whereas in the earlier work, both teachers and external facilitators (youth workers) were used. Pupils in Scotland received the SHAHRP curriculum but, when necessary, the materials were further refined for the cultural context. For example, information that was provided about emergency services related to the Scottish Ambulance Service rather than the Police Service of Northern Ireland.

The classroom component of STAMPP differed from the original Australian SHAHRP curriculum in a number of ways. The Australian programme was targeted at pupils aged ≥ 13 years in phase 1 (as was the original NI adaptation of the SHAHRP described in Chapter 1, Introduction to the intervention components of Steps Towards Alcohol Misuse Prevention Programme), the curriculum was longer (17 activities delivered in 8–10 lessons in phase 1, and 12 activities delivered over 5–7 weeks in phase 2) and Australian reference data were used in the lessons.

The brief intervention delivered to the parent(s)/carer(s) of children in the intervention comprised a short, standardised presentation delivered by a team of trained facilitators (independent of the trial team) at specially arranged parent evenings on school premises. The presentation included an overview of the CMO’s 2009 guidelines for drinking in childhood,65 information on alcohol prevalence in young people and corrected (under)estimates of youth drinking rates, and it highlighted the importance of setting strict family rules around alcohol, with the recognition that children often model their own alcohol use behaviour on that of their parent(s)/carer(s). The presentation was followed by a brief discussion on setting and implementing authoritative family rules on alcohol. All intervention pupils’ parent(s)/carer(s) were followed up by a mailed leaflet (March 2014) that provided a summary of the key information delivered over the course of the evening. The delivery of the parental intervention coincided with phase 2 of the SHAHRP between September and December 2013.

The parental/carer activity was developed by the research team for this trial, and was partly based on the Dutch adaptation of the Swedish Örebro Prevention Programme undertaken by Koning et al. 64 These researchers delivered a brief intervention to parents on setting strict rules around alcohol in combination with a school-based alcohol curriculum (the Dutch Healthy School and Drugs programme). The parental component in STAMPP differed from the Dutch intervention in a number of ways. First, the Dutch activity was delivered at two annual parent evenings as part of general school discussions; second, the intervention was delivered by a member of the Dutch research team; third, the content of the presentations used Dutch data and was orientated towards challenging societal alcohol norms; and, fourth, attendees set their own family alcohol rules through discussion with a classroom learning mentor, whereas in STAMPP rules were based on the CMO’s guidance. 65 Both approaches utilised a follow-up mailed information leaflet.

The control group participants continued with EAN within their school, which would include standard personal, social and health education but would not be uniform across all such schools. Parents/carers of control students received no intervention. Provision of alcohol use education as part of statutory education or usual school activities (and, therefore, not able to be experimentally manipulated) was monitored through information collected as part of an online teacher questionnaire (see Chapter 4, Online survey with teachers).

Please refer to Appendix 1 for the logic models underpinning the intervention.

Outcome measures

Parent/carer completed measures

Parents/carers completed a short postal questionnaire that measured family rules on alcohol and parental self-efficacy in implementing rules and controlling adolescent behaviour. Alcohol Rules was a 10-item scale measuring the degree to which parents permit their children to consume alcohol in various situations, such as ‘in the absence of parents at home’ or ‘at a friend’s party’ (α = 0.86–0.90). 62 Parental self-efficacy was measured using a three-item scale assessing the level of confidence that a parent had in their own ability to prevent their child from drinking (α = 0.67). 74 These data were collected to inform future mediation analysis and were not included in the DAP; therefore, analysis of these data is not included in this report.

Definitions/calculations

The questionnaire outcome scores were calculated in accordance with the original coding system outlined in their corresponding publication (see Outcome measures). A data dictionary was prepared and is available from the authors upon request.

To assess the costs of the intervention for the purposes of economic analysis, overall costs of providing the programme were estimated. This included programme set-up costs and the costs of implementing and delivering the programme. The overall cost per participant and cost per school were also estimated. The costs of implementing and delivering the programme included staff time, venue and equipment costs, provision of support facilities and materials utilised.

Data collection

Sample size

It was calculated that a sample size of 90 schools (45 per study arm; 80 pupils per school) would be powerful enough [80%, α = 0.05, intracluster correlation coefficient (ICC) = 0.09, based on data from the Belfast Youth Development Study75] to detect a standardised effect size of δ = 0.2 or a 10% absolute reduction in risk (51% vs. 41%) for the primary outcome (HED). Assuming 20% attrition within each cluster (from 100 to 80 pupils), the target sample size was 90 schools and 9000 students at baseline.

Research ethics committee approvals, consent and research governance

The research was approved by Liverpool John Moores University Research Ethics Committee (reference number 11/HEA/097). For data collection from students within schools, the research ethics committee required opt-out consent [i.e. parent(s)/carer(s) were advised about the research requirements and could communicate a refusal for their child to participate]. Students could also refuse to participate at the time of data collection. For data collection from parent(s)/carer(s), consent was provided through the return of a completed questionnaire.

The trial was funded by the National Institute for Health Research (NIHR) Public Health Research programme, sponsored by Liverpool John Moores University, managed by a TMG of study investigators and staff, and overseen by an independent SSC appointed by, and reporting to, the NIHR. The International Standard Randomised Controlled Trial Number is ISRCTN47028486.

Randomisation

Stratified randomisation of clusters (schools) to intervention or control conditions was performed by a NICTU staff member not involved in the trial and blind to the identity of the schools. Schools were stratified by school type (all-boys’ school/all-girls’ school/coeducation school), socioeconomic status (SES) [percentage of pupils entitled to free school meals (FSMs), categorised as a tertile split: low, moderate or high] and school size.

Randomisation was conducted as an electronic ‘card sort’. Within each stratum, each school had a random number attached. The schools were then sorted by ascending random number and this process was repeated several times by holding down the refresh formula function key. This made it impossible to view intermediate allocations, and the final order was taken as the school allocation.

Two schools in NI that were in very close geographical proximity, and as a result shared staff and facilities, and, therefore, they were treated as one unit to avoid contamination. Two schools in NI that shared pastoral care arrangements were also treated as one unit to avoid contamination. Stratified randomisation was used to balance the arms and was performed separately for Glasgow/Inverclyde and for NI. Schools in Glasgow/Inverclyde were stratified based on FSM provision. As a larger number of schools were recruited in NI, two stratification factors were identified: FSM provision and school type.

The trial manager matched the school codes used in the randomisation to school names, and this matching was then independently verified by the NICTU to ensure allocation concealment. Randomisation took place when children were in school year 8 or S1.

Blinding

Schools and participants were not blinded to study condition. Intervention trainers and delivery staff (teachers) were not blinded to study condition. Data collection was undertaken by a team of researchers that included the trial manager and a dedicated research assistant, neither of whom was blinded to study condition, or by temporarily contracted research staff who were blinded to study condition.

Data analysis of primary and secondary outcomes was undertaken by the project statistician and health economist, with a data set that was blinded for study condition.

Statistical methods

Statistical analyses

Descriptive analysis

Summary statistics on school and pupil recruitment, withdrawal and dropout were collated for both trial arms and reported as a participant flow diagram for reporting of CRCTs (Figure 1).

FIGURE 1.

School and participant flow diagram: STAMPP. Analysis was conducted at T3 on pupils who had completed each of the primary outcome measures. N, number of schools; n, number of pupils.

Intraclass correlation coefficient

The ICC for the primary outcomes was calculated and is reported in Chapter 3 [see Heavy episodic drinking T3 (binary outcome)]. This was calculated overall and for each arm separately.

Fidelity test

Appropriate descriptive analysis was used to examine the extent to which the necessary conditions required to permit a valid test of the treatment efficacy were met. This included assessment of achieved statistical power, patterns of attrition, and treatment integrity and discriminability (i.e. that STAMPP was sufficiently distinct from EAN) across the trial sites. This work included analysis of both qualitative and quantitative data.

Randomisation check

Descriptive summaries of baseline participant characteristics from the two trial arms were tabulated to assess between-group equivalence across the trial arms (checked on randomisation). The descriptive data were tabulated to compare attendees at the parental session with those who completed the follow-up questionnaire only. Descriptive summaries were produced for baseline data at the school level. These data were used to check comparability between study arms and generalisability of the study population.

Outcome measure scores from the questionnaires were summarised and tabulated for the trial arms. Descriptive statistics, with confidence intervals (CIs) when appropriate, were used for the tabulation of outcomes in the trial arms. CIs presented were adjusted to allow for clustering effects.

Health economic evaluation

A within-trial cost-effectiveness analysis (CEA) was undertaken to assess the cost-effectiveness of STAMPP compared with EAN in reducing HED (defined as self-reported consumption of ≥ 6 units for males and ≥ 4.5 units for females in a single episode in the previous 30 days, and which was dichotomised at never/one or more occasions) in year 9 or S2 pupils (aged ≥ 13 years on the 1 September 2012) at T2 and T3. The methodology adhered to the NICE’s Guide to the Methods of Technology Appraisal. 77

A societal perspective was adopted for the analysis in order to capture resource use related to each child’s contact with the NHS, Personal Social Services and criminal justice services. Data on service use by all participants from T0 to T3 were collected using the tool described in Description of primary and secondary outcome measures.

Pupil service use and intervention-related resource use from T0 to T3 were quantified as outlined in the paragraphs above, and unit costs were applied from national sources such as the NHS reference costs, the Personal Social Services Research Unit’s unit costs of health and social care and the unit costs of criminal justice. When national costs were not available, unit costs were identified in consultation with the appropriate finance departments of the resource provider.

Consistent with the primary outcome of the study, the primary economic effectiveness measure was the number of pupils who reported HED at T3. The secondary economic effectiveness measure was the number of heavy drinking episodes at T3. The latter was calculated using data on the frequency of heavy drinking episodes in the previous 30 days collected at the four survey time points. Data were linearly interpolated over the study period to fill in gaps in survey periods and to obtain an estimate of the number of heavy drinking episodes over the study period.

As the time horizon of STAMPP extended beyond a 12-month period, a 3.5% discounting rate was applied, as recommended by NICE. 77

Incremental cost-effectiveness ratios (ICERs) were calculated. The primary CEA estimated the incremental cost per young person experiencing HED avoided due to STAMPP at T2 and T3. The secondary analysis estimated the incremental cost per episode of heavy drinking avoided due to STAMPP at T2 and T3. Multiple regression models were used to predict costs and effects, adjusted for covariates and taking into account the clustered nature of the data. CRCTs raise analytical issues for CEA; costs in particular may be more similar within, rather than between, clusters. CEA should recognise both that costs and effects are correlated and that individuals are clustered within settings. Thus, appropriate models were used that recognised the clustered nature of the data. 78

Uncertainty in the cost-effectiveness measures was investigated using non-parametric bootstrapping with 1000 replications of the ICERs. The resulting replicates were plotted on the cost-effectiveness plane and used to construct cost-effectiveness acceptability curves (CEACs).

Software used

Data cleaning, data management and preliminary analysis were undertaken using IBM SPSS, version 21 (IBM Corporation, Armonk, NY, USA). Mplus, version 7.11 (Muthén & Muthén, Los Angeles, CA, USA), was used for the multivariate regression models. Stata/IC, version 12.0 (StataCorp LP, College Station, TX, USA), was used for the health economic analysis and to verify Mplus models and generate odds ratios (ORs). NVivo, version 10 (QSR International, Warrington, UK), was used for the qualitative analysis.

Public and patient involvement

In addition to participation in the intervention, data collection activities and process evaluation, target group and stakeholder involvement had already been built into the development of the classroom intervention. During the development process, a working group was established, which consulted teachers, local service co-ordinators and voluntary/community sector colleagues on curriculum adaptation. Adapted materials were piloted with pupils in school years 10 and 11 in NI. During the trial, the primary outcome measures were also piloted with a sample of pupils not taking part in the research and their feedback was received. Refinements were made to the questionnaire materials on the basis of this.

A number of public-orientated dissemination activities also took place across the life of the trial, including seminars aimed at academics, school staff and other stakeholders; informational presentations; and the preparation of summary documents for schools describing emerging findings across the trial. Two summary seminars for representatives of schools and stakeholder organisations are also planned for the school year 2016–17 after trial completion.

Changes to protocol

The final version of the protocol was version 7. Significant changes (i.e. those not related to simple language and terminology) to the protocol are detailed below (signed off by the SSC and accepted by the funder):

1. Version 4, 14 June 2013, in accordance with guidance from the SSC:

   1. The primary research question was specified: ‘What is the (cost) effectiveness of STAMPP in reducing alcohol consumption (frequency of consuming > 5 drinks in a single episode in the previous 30 days) in year 9 pupils at + 18 months after intervention?’

   2. Assessments at other time points were specified as secondary research questions.

2. Version 5, 21 November 2013, in accordance with discussions at the TMG held in October 2013:

   1. The primary and secondary research questions were restated as objectives to match the DAP and timed as months from baseline and not post intervention.

   2. The primary and secondary measures (as defined within the DAP) were added to the protocol, in addition to the general description of the measures used within the study.

   3. The immediate post-intervention follow-up was removed to match the DAP.

   4. Parent/carer measures were described.

3. Version 6, 15 January 2014, in accordance with discussions at the TMG held in December 2013:

   1. A second primary outcome was added to reflect the harm reduction basis of the intervention: ‘To ascertain the effectiveness of STAMPP in reducing alcohol-related harms as measured by the number of self-reported harms (harms caused by own drinking) in second-form pupils (school year 9 or S2 in the academic year 2012/2013) at + 33 months (T3) from baseline’.

4. Version 7, 8 April 2014, in accordance with discussions at the TMG held in March 2014:

   1. Clarified the primary outcome measure: ‘To ascertain the effectiveness and cost-effectiveness of STAMPP in reducing alcohol consumption (defined as self-reported consumption of ≥ 6 units in a single episode in the previous 30 days for males and ≥ 4.5 units for females) in school pupils (school year 9 or S2 in the academic year 2012/2013) at + 33 months (T3) from baseline. This will be dichotomised at never/one or more occasions’.

Summary of methods

This was a CRCT conducted in NI and Glasgow/Inverclyde Education Authority areas, with schools as the unit of randomisation. The participants were male and female school students (school year 9 or S2 in the academic year 2012–13 and aged 12–13 years) attending mainstream secondary schools in the study areas. In each participating school, all students in attendance at the time of data collection were asked to complete the project questionnaires. Schools were randomised into the trial and baseline data were collected when pupils were in school year 8 or S1.

The study was powered to detect a standardised effect size of δ = 0.2, or a 10% absolute reduction in risk (51% vs. 41%) for the primary outcome of HED (80%, α = 0.05, ICC 0.09) and the target sample size was 90 schools and 9000 students at baseline. Following recruitment, schools (n = 105) were randomised into intervention (schools, n = 52; pupils, n = 6379) or control (schools, n = 53; pupils, n = 6359) groups.

The primary outcomes were (1) self-reported alcohol use (HED, defined as the self-reported number of occasions of consumption of ≥ 6 units for males and ≥ 4.5 units for females in a single episode in the previous 30 days, which was dichotomised at never and one or more occasion); and (2) the number of self-reported harms (harms caused by own drinking). The primary economic effectiveness measures were in line with the primary outcome measures. Primary outcomes were assessed at T3 using a self-completed questionnaire.

A number of secondary outcomes were also assessed and included the primary outcomes assessed at T1 and T2, self-reported alcohol use (lifetime, previous year and previous month), support service utilisation, the number of self-reported harms caused by the drinking of others, age of alcohol initiation, unsupervised alcohol use and the number of units of alcohol consumed in ‘typical’ and last-use episodes.

Primary and secondary analyses and the health economic analysis were performed using the CC population. For each primary outcome, a two-level regression model was fitted, with pupils nested within schools. For self-reported consumption of ≥ 6/≥ 4.5 units, the model used was logistic regression. For the number of self-reported harms, a negative binomial model was used. For each primary outcome, significance was set at p < 0.025.

The cost-effectiveness of STAMPP was estimated using conventional decision rules and reported as ICERs when appropriate. Uncertainty in the cost-effectiveness measures was investigated by bootstrapping multilevel models relating to public service costs and outcomes and using the incremental costs and outcomes to generate 1000 replications of the ICERs. The resulting replicates were plotted on the cost-effectiveness plane and used to construct CEACs.

Process outcomes were assessed across eight prespecified domains using nine data sources. Assessments included focus group with pupils, an online survey with teachers, interviews with senior school staff and stakeholders. Fidelity and completeness of delivery were assessed using bespoke tools and calculation of participation rates at the parent/carer evening.

Participant flow

Figure 1 shows the participant flow through the trial. Recruitment began in November 2011 and ended in January 2012. Our original sample size calculation indicated that a total of 90 schools (with a mean of 80 pupils) were needed to ensure sufficient statistical power. However, we oversampled (105 schools) and, as no schools dropped out over the course of the trial, all schools were retained. As this was a CRCT of an intervention taking place over several weeks, pupil numbers refer to those who completed the questionnaire at each data collection period. No participant or parent/carer requested that data be retrospectively removed from analysis. Multiple data collection ‘mop-up’ visits were undertaken with schools; therefore, attrition represents pupils who were absent on data collection days rather than formal dropout.

Baseline characteristics of participants

A total of 11,316 pupils across 105 school participated in the T0 data sweep. Tables 1 and 2 provide descriptive statistics of the pupils at T0 across the study arms.

Just under two-thirds (62%) of study participants were located in NI, with the remainder sampled from schools in Glasgow/Inverclyde. There was an equal split of girls and boys in the total baseline sample and across the study arms. Around one-quarter (23%) of the participants reported being in receipt of FSMs, and fewer than 1 in 10 (8%) reported any HED (primary outcome measures) at baseline (T0). The sample was predominantly from a white ethnic background.

Table 2 provides a summary of the baseline characteristics for two continuous indicators; respondents’ age and self-reported alcohol-related harms experienced as a result of own drinking (primary outcome measure). The mean age of the sample at baseline (T0) was 12.5 years [standard deviation (SD) 0.4 years] and the mean reported number of harms was 0.8 (SD 2.0).

In addition to the primary alcohol outcomes (HED and alcohol-related harms), the study also assessed a range of basic alcohol consumption indicators at baseline (T0) (Table 3). Although one-quarter of the pupils had consumed at least one full drink at some stage in their lives, 1 in 5 had drunk alcohol in the previous year, over 1 in 10 had drunk alcohol in the previous 6 months and 1 in 20 had consumed alcohol within the previous month.

In addition to those pupils attending participating schools at T0, pupils who were absent at T0 but present at T1 data collection (i.e. missing on the day of the T0 data collection) and pupils who joined participating schools before the delivery of phase 1 of the intervention in the autumn term of 2012 (between T0 and T1) were included in the study population. Table 4 provides the characteristics of the full combined baseline sample (i.e. pupils present at T0 and/or at T1).

Randomisation check

A formal test of the randomisation process was performed on the baseline assessments of the original primary outcomes and key individual pupils’ characteristics [gender, SES (recipient of FSMs), consuming five or more drinks in the previous month, number of self-reported harms] to assess between-group equivalence across the trial arms (Table 5). Consuming five or more drinks in the previous month was the original primary outcome in the trial, but this was amended after commencement (see Chapter 2, Changes to protocol). The trial arms were equivalent across all of the variables assessed at T0.

Response rates

Of the full sample (those who completed a questionnaire at either T0 or T1, N = 12,738), 10,405 also completed the questionnaire at T3 (81.7%). Table 6 presents the dropout rate by sample characteristic. Dropout rates were higher among male pupils (19%), those who were in receipt of FSMs (25.8%) and those who had used alcohol at baseline (25.4%). There was little difference in dropout between the control and intervention arms of the trial (around 1 percentage point difference). Dropout also varied by location, with a higher rate in Scotland (24%) than in NI (15%), and by school; across schools, dropout rates varied from 1.5% to 32% (see Table 35, Appendix 5).

Description of primary outcomes

The primary outcome analysis employed two primary outcome measures. The first related to self-reported episodes of single-session HED and the second to self-reported experiences of harms related to own drinking.

HED (T3): self-reported alcohol use defined as self-reported consumption of ≥ 6 units for males and ≥ 14.5 units for females in a single episode in the previous 30 days, assessed at T3. This was dichotomised at none/one or more occasions.

Drinking harms (T3): the number of self-reported harms (harms caused by own drinking) assessed at T3. Items included harms such as getting into physical fight or being sick after drinking. The outcome was a count of the number of discrete harms reported (0–16). Therefore, for example, a score of 3 represented a report of three separate harms experienced. The individual harms were not weighted in any way. Pupils who provided a valid response to at least 1 of the 16 items were included in the count. Only pupils who failed to answer any of the 16 items were set to item missing.

Table 7 provides the basic count and percentages of respondents reporting HED at T3 by study arm. Around one in five participants reported at least one episode of HED in the previous 30 days. The prevalence of HED was 9 percentage points higher in the control group (26%) than in the intervention group (17%). It was also possible to examine HED prevalence calculated on a base of those pupils reporting lifetime alcohol use at baseline. This gives an indication of the progression from drinking onset to heavier drinking patterns (Table 8). Among drinkers at baseline, around half had engaged in HED at T3 within the control schools, compared with just over one-third in intervention schools.

Table 9 provides the basic count and percentages of respondents reporting alcohol-related harms (due to own drinking) at T3 by study arm. This variable was heavily skewed with a preponderance of zeros (no harms reported). Around two-thirds of pupils (63%) reported no alcohol-related harms at T3. The median number of harms was equivalent in each study arm (0), although the interquartile range was smaller in the intervention arm than in the control arm (2 and 3, respectively).

As with the HED outcome, the distribution of harms among those pupils who were drinking at the outset of the intervention (here defined as those who reported ever drinking at T0) was examined. In control schools, around 68% of existing drinkers reported experiencing one or more harms at T3, compared with around 62% of pupils in intervention schools (Table 10).

Primary outcome analysis

The primary outcome analysis consisted of a series of multilevel regression models, adjusted for the clustering of pupils within schools (two-level regression models with a random intercept) using the CC population (those pupils who provided alcohol data at T3). School was the clustering variable. Models with random slopes and intercepts were also estimated (slopes and intercepts as outcomes). However, as none of the slope parameters was significant, the random intercept models were used. The slope and intercepts as outcomes models can be found in Appendix 5 (see Tables 36 and 37).

In addition to the primary outcomes, the regression models also included covariates at both level 1 (within/pupil level) and level 2 (between/school level).

Between-level (school) covariates

Intervention arm: this was a binary covariate in which schools in the control arm were coded 0 and schools in the intervention arm were coded 1.

Free school meal provision (randomisation stratification factor): schools were classified into three groups based on FSM provision. The allocation was based on a tertile split using information provided by head teachers on the proportion of pupils in receipt of FSMs: low FSM provision (0–15.4%), moderate FSM provision (15.5–30.4%) and high FSM provision (≥ 30.5%). This covariate was treated as a continuous indicator.

School type (randomisation stratification factor): given the larger number of schools in NI, an additional stratification factor was used in the randomisation. This was school type (all-boys’ school/all-girls’ school/coeducation school). Schools in Glasgow/Inverclyde were all assigned to the coeducation type. This covariate was represented by two dummy variables (coeducation was the comparison category).

Location: a binary variable was generated to indicate the location of the schools (NI/Scotland). This variable was used to adjust for clustering at the Education Authority level.

The path diagram of the regression model is presented in Figure 2. For the HED outcome, a two-level logistic model was estimated. For the harm outcome, a two-level negative binomial regression was estimated. All models were estimated in Mplus.

FIGURE 2.

Path diagram for the multilevel model.

Secondary outcomes

A range of secondary outcomes was also examined within the study. These included the primary outcomes assessed at T2:

• HED (T2): self-reported alcohol use defined as self-reported consumption of more than five drinks, assessed at T2. This was dichotomised at none/one or more occasions. This outcome was assessed via a two-level logistic regression model. Around 12.4% of respondents reported HED at T2 using this measure. In the intervention arm, HED was reported by 10.9% (n = 573) and in the control arm by 13.9% (n = 722) of participants.

• Drinking harms to self (T2): the number of self-reported harms (harms caused by own drinking) assessed at T2. The outcome was a count of the number of discrete harms reported (0–16) and was assessed by a two-level negative binomial model. In the intervention arm, 74.3% of participants reported no drinking harms and in the control arm 71.5% of participants reported no harms.

In addition, a number of secondary outcomes at T3 and T2 were also examined, including:

• Lifetime drinking (T3): whether or not the pupils had ever consumed a full drink of alcohol at T3 (two-level logistic regression model).

• Previous year’s drinking (T3): whether or not the pupils had consumed a full drink of alcohol in the previous year, assessed at T3 (two-level logistic regression model).

• Previous month’s drinking (T3): whether or not the pupils had consumed a full drink of alcohol in the previous month, assessed at T3 (two-level logistic regression model).

• Harm from others (T3 and T2): the number of self-reported harms experienced that were the result of other people’s drinking, assessed at both T2 and T3 (two-level negative binomial models).

• Age at onset (T2 and T3): self-reported age at which respondent first consumed a full drink, assessed at both T2 and T3 (two-level Cox regression model).

• Unsupervised drinking (T2 and T3): whether or not the pupils were permitted, by their parent(s)/carer(s), to consume alcohol (with small group of friends or at parties) with no adult present, assessed at both T2 and T3 (two-level logistic regression model).

• Number of drinks consumed (T3 and T2): pupils were asked whether or not they usually drank from a range of different alcoholic drinks [beer, alcopops, spirits cider, wine, [brand name] (a popular brand of fortified wine, with caffeine), others] and, if so, how much they usually drank. The values for each drink were summed together to give a total. As the underlying items continued decimals, the total value was multiplied by 10 to create whole numbers.

The secondary outcome analysis also included covariates at level 1 (individual) and level 2 (school) when appropriate.

Subgroup analyses on primary outcomes

To explore differential treatment effects on the primary measures, interaction terms were fitted between trial arm and baseline measures thought to predict the effect of treatment. Initial prespecified subgroup analysis examined baseline alcohol consumption (lifetime use, previous year’s use, age at onset and unsupervised drinking). Given the high correlations of lifetime use, previous year’s use and the two primary outcomes assessed at baseline (HED and alcohol-related harms) (Table 17), subgroup models were estimated on a base of just baseline drinkers (lifetime and previous year’s use) to examine the possibility of the intervention having a differential impact on drinkers compared with non-drinkers at baseline.

For HED, the treatment arm was significant in both the drinker-only models (both previous year and ever use) and the corresponding non-drinker-only models (Table 18). This means that no differential intervention effect on HED, dependent on baseline drinking, was detected. However, for alcohol-related harms as a result of own drinking, although the intervention was associated with a significant reduction in the number of self-reported harms among drinkers (either defined as lifetime or previous year’s use at baseline), it did not reduce self-reported harms among the non-drinkers at baseline. When the lifetime use and previous year’s use subgroup effects were examined via interaction terms (on the full CC population) the interaction terms for harms were non-significant, as were the interaction terms for age at onset and unsupervised drinking.

In the additional prespecified subgroup analysis model that was estimated (age, gender), the corresponding interaction terms were all non-significant.

Adverse events

No adverse events were reported throughout the duration of the trial related to receipt of the intervention or to participation in the STAMPP research process.

Summary of results

Overall retention was high (81.7%). No schools withdrew from the trial and no pupils or parents/carers withdrew consent. The intervention was effective in reducing HED, but not in reducing harms resulting from own drinking. The prevalence of HED was 9 percentage points higher in the control group (26%) than in the intervention group (17%) at T3. This represented a significant intervention effect (estimate –0.516, SE 0.102; p < 0.001). The OR for the intervention effect was 0.596 (SE 0.0596). The corresponding 95% CI for the OR ranged from 0.490 to 0.725. Around two-thirds of pupils (63%) reported no alcohol-related harms at T3 (control, 60.7%; intervention, 65.1%). The median number of harms was equivalent in each study arm (n = 0), although the interquartile range was smaller in the intervention arm than in the control arm (2 and 3, respectively). The negative binomial model used showed that the intervention arm was not a significant predictor of harms (estimate –0.101, SE 0.083; p = 0.222; incident rate ratio 0.916, 95% CI 0.780 to 1.052). Examining secondary outcomes, no parameter estimates were significant for the intervention arm, indicating no effects of the intervention. Similarly, no significant interactions were identified in the prespecified subgroup analyses.

Process measures

The overall process evaluation was informed by the framework developed by Grant et al. ,83 for CRCTs, and ran alongside the assessment of outcomes. Process measure data were collected across a number of different studies with a view to obtaining information in the following domains (Figure 3 shows a summary).

FIGURE 3.

Summary of data used to assess process outcomes.

1. Recruitment of clusters: the number of schools agreeing or refusing to participate was documented. Consideration was given to the differences between schools that agreed to participate and those that did not.

2. Delivery to clusters: the interventions delivered to pupils and parents were described, including intervention development and the intervention logic model.

3. Response of clusters: teacher experiences of the classroom component (or EAN for the control group), such as participation in delivery, perceptions of the effectiveness of the classroom component and perceptions of the engagement of recipients, were assessed.

4. Recruitment and reach in individuals: statistical comparison of subject demographics (age, sex) and baseline alcohol use behaviours was undertaken in order to identify differences between control and intervention conditions, to compare representativeness against general population data and to test for randomisation failure.

5. Delivery to individuals: this component assessed implementation fidelity, that is, the intervention that was delivered to recipients and whether or not this was delivered as intended. Assessment of implementation fidelity in STAMPP provided data on the adherence, dose and quality of delivery, participant responsiveness and programme differentiation.

6. Response of individuals: this component assessed the target population’s experience of, and response to, the intervention, and analysed experiences of the intervention in the classroom, the implementation of lessons to alcohol use/opportunities outside of school (to probe experiences of the rule-setting component of the intervention) and understanding of alcohol-related risks (as presented in the SHAHRP curriculum).

7. Maintenance and policy context: this component assessed intervention implementation and maintenance with regard to stakeholder support, staff, resources, setting and support organisations required in order to maintain the reach of target populations, provision of future intervention services and to achieve wider (local) policy objectives.

8. Unintended consequences: the analysis of primary and secondary outcomes was the main source of information about unintended consequences of the intervention. The trial was considered to have produced unintended consequences if a significantly greater number of students reported an episode of consuming ≥ 6/≥ 4.5 units in the previous 30 days at T3. A trial risk register was also maintained throughout the research and adverse events were recorded.

Data were collected through a mixture of monitoring exercises, and through surveys and qualitative work with pupils, teachers and other stakeholders. Information on other alcohol-related activities delivered concurrently in schools was collected as part of an online teacher questionnaire (see Chapter 4, Online survey with teachers). Although a rich data set was obtained from the qualitative work, only key findings and specific discussions for each component are presented here, and the general discussion/conclusions are synthesised in the discussion (see Chapter 6) and conclusions (see Chapter 7) chapters.

Qualitative process evaluation methods

Findings

Additional process measures

Summary of process evaluation

With regard to process evaluation components, clusters were successfully recruited into STAMPP, randomisation was successful and schools were comparable across intervention arms at baseline. No adverse events were reported.

The classroom component of STAMPP was delivered largely as intended, although there was some variation in fidelity of implementation scores between schools, namely through the number of lessons required to deliver the content. Our survey did not provide the reasons why the recommended number of lessons was not adhered to; however, considering the busy school curriculum, this may have been a result of competing demands or extension of some of the lessons and themes. For example, we became aware that one class had produced a short play based on their discussions and learning about the curriculum, so these type of activities may have been included. The curriculum was enjoyed by pupils, who reported that they found it interesting, informative and relevant to their own experiences or how they believed they might use alcohol in future. This contrasted with the largely negative perceptions of alcohol education reported by pupils in the EAN condition. However, it may be possible that bias in pupil interviews was non-equivalent. As they generally reported positive engagement with SHAHRP, this may have led to discounting or under-reporting of other negative aspects. Similarly, although alcohol EAN tended to be viewed negatively overall, this may have led to under-reporting of positive aspects. On the whole, the classroom materials were perceived as useful and were used as intended by the majority of teachers and pupils. Teacher evaluations of the classroom component were also largely positive, and it was viewed as complementing the schools’ wider health and well-being strategies. Teachers and school management believed that it was possible to accommodate the programme in the curriculum, supporting resources were useful and content was both experientially and age-appropriate.

Stakeholders in both trial geographies reported that STAMPP corresponded to their current policy and strategic aims, and that, if it was shown to be effective, it may form part of their future delivery of support for young people. In keeping with the views of some teachers, stakeholders emphasised the importance of assessing the impact of SHAHRP as a targeted intervention, as this was in line with many of their priorities. This would require further investigation.

In contrast, there was very low uptake of the parental/carer component, and postal returns of the parent/carer survey, which were used as an indicator of implementation of mailed intervention materials, were also relatively low. It should therefore be concluded that this component of the intervention was not successfully delivered.

Aim and perspective

The overall aim of the economic evaluation was to assess the cost-effectiveness of STAMPP compared with EAN in reducing HED in year 9 or S2 pupils. The study was a CEA alongside the main trial. The primary CEA estimated the incremental cost per young person experiencing HED avoided because of STAMPP at T2 and T3. The secondary CEA estimated the incremental cost per episode of heavy drinking avoided because of STAMPP at T2 and T3. A public sector perspective was adopted for the analysis that encompassed the costs to local authorities, NHS, personal social services and criminal justice services. Within the economic evaluation, we undertook and reported on the following:

• analysis of intervention resource use and costs

• analysis of public sector resource use and costs at each time point using all data that were available

• analysis of total costs at T2 and T3 using data from pupils with complete cost data only

• analysis of outcomes at T2 and T3 using data from pupils with complete cost data only

• CEA at T2 and T3 using data from pupils with complete cost and outcome data.

Intervention resource use and costs

We calculated the economic cost of STAMPP according to the principle of opportunity cost, that is, we attempted to place a value on the benefits that were forgone by STAMPP being delivered instead of something else. We therefore included the full value of all the resources it used, regardless of whether or not the resources were directly purchased for the study.

The content of EAN within schools was assessed as part of the process evaluation (see Chapter 4, Usual alcohol-related activities in both intervention and control schools over the course of the study), and this supported the assumption that a similar level of resources were used, thus the costs were not included in the analysis.

The identification of appropriate intervention resource use was guided by a number of sources. These included the practical guide by Ritzwoller et al. 93 to costing behavioural interventions and methods used in other behavioural interventions such as the Diabetes Education and Self-Management for Ongoing and Newly Diagnosed (DESMOND) programme,94 A Stop Smoking In Schools Trial (ASSIST)95 and the First-aid Advice and Safety Training (FAST) parent programme. 96 In keeping with both Hollingworth et al. 95 and Mytton et al. ,96 resources were categorised according to the stage at which they were used in the research process: planning and preparation for delivery (stage 1) or delivery itself (stage 2).

Pre-startup resources associated with the development of STAMPP were not included in the analyses, as they would not be incurred should the intervention be mainstreamed in the future. These included the development of the teacher manual and pupil workbook content, the planning of the lessons and the design of the materials.

Stage 1 resources in STAMPP covered the recurring costs that were associated with printing/copying the intervention materials (teacher manual, pupil workbook, parent/guardian information sheets and CDs) and the delivery of teacher training (trainer fees, venue, catering and teacher cover). No cost associated with the purchasing and licensing of STAMPP intervention is foreseen, and so it is not included here. Stage 2 resources covered the recurring costs that were associated with teacher time input for delivery of phases 1 and 2 of the programme. The cost of teaching time was included even though the intervention was delivered within normal working hours and so would not have incurred an extra cost to the public sector. In doing so, we captured the opportunity cost of the benefits to pupils that were forgone by STAMPP being delivered instead of other lessons.

Resources used during stages 1 and 2 were recorded retrospectively by the project team, and costs were obtained from invoices when available. The costs associated with printing the intervention materials were not available, as these resources were present at the start of the trial. We therefore obtained printing quotations and deflated it to the appropriate price year using the Consumer Price Index. 97 The local authority in Glasgow/Inverclyde covered all of the costs associated with the training of the teachers including teacher cover, venue and catering, and so these were also not available. We therefore used the costs that were available for NI and applied them to Scotland. The majority of teachers delivering the intervention were at a junior level, thus we used point 3 on the main pay scale,98 including 24% for employers’ costs.

Pupils’ service use and costs

Data on service use by all pupils from baseline to T3 were collected using an instrument administered at T0, T1, T2 and T3. The instrument incorporated some items taken from the Client Service Receipt Inventory68 specifically adapted for childhood69 and items relating to the use of judicial services (see Appendix 9). The instrument included an information page with definitions of some of the public services in case the students were unfamiliar with them. The instrument was designed with input from relevant professionals (e.g. educational psychologist, social workers, Scottish and Northern Irish teachers) and reviewed by a social researcher who was experienced in delivering questionnaires to children, and other health economists. The instrument asked pupils to report their use of services in the previous 6 months, thus providing service use data for the 6 months before baseline, 7–12 months, 19–24 months and 28–33 months.

Individual-level service use was combined with unit costs (Table 25) to estimate a cost for each pupil for each of the four survey time periods. Unit costs were obtained from publicly available sources99–101 and set at 2013–14. For school counsellors/guidance teachers we assumed a mid-point, point 2, on the upper pay scale98 and included 24% for employers’ costs. As there were gaps in the survey period in which no cost data were collected (baseline to 6 months, 12–18 months and 24–27 months), we used interpolation of the available cost data to fill in these gaps. This was done separately for each participant to obtain an estimate of their total service use costs at T2 and T3. Total costs could not be calculated for those pupils who had any missing cost data points, as these were required for the interpolation.

Analysis of intervention resource use and costs

Stage 1 costs were added together to get a total cost. Stage 2 costs relating to the delivery of the intervention at a class level were multiplied by the total number of classes that received the intervention to get a total cost. This was estimated by dividing the number of pupils receiving the intervention at baseline (n = 5749) by 30, which was used as an estimate of the maximum number of pupils per class. Stage 2 costs relating to the delivery of the parental component were calculated by multiplying the number of schools by the costs of delivering one session. The stage 1 and stage 2 costs were totalled and divided by the number of pupils receiving the intervention at baseline to obtain the mean cost per pupil.

Analysis of pupils’ public sector service use and costs

The analysis of service use included the pupils present at each survey time point. If there were any missing fields within the service use questionnaire, it was assumed that the relevant service had not been used. Descriptive statistics were used to summarise service use and costs at each time point for the different categories of the public sector (education, health and criminal justice).

As interpolation was used to calculate total costs, only pupils with complete cost data up to T2 or T3 could be included in the analysis of total costs. The costs of service use in the 6 months before T0 were not included in the total costs but were used in the interpolation. The total costs were analysed using a multilevel mixed-effects regression model controlling for pupils’ baseline costs, school location (NI/Scotland), school level of FSM provision (low, 0–15.4%; moderate, 15.5–30.4%; high, ≥ 30.5%), school type (all-boys’ school/all-girls’ school/coeducation school) and clustering. With the exception of baseline costs, the same covariates were used in the analysis of the primary outcome data and prespecified in the data management plan. The incremental costs associated with STAMPP were estimated using the margins command in Stata. This command first calculates the marginal (treatment) effect for each individual with their observed levels of covariates and then averages these values across all individuals to obtain the average marginal effect.

The costs occurring in years 2 and 3 of the study were discounted at 3.5%, in keeping with NICE (2013) guidance. 77

Outcomes

Consistent with the primary outcome of the study, the primary economic effectiveness measure was the number of pupils who reported any HED in the previous 30 days at T3. This was based on responses to the question ‘How often in the past month have you drank [sic] 4.5 (female)/6 (male) or more units of alcohol?’. As we were also interested in outcomes at T2, we dichotomised responses to the question ‘How often in the past month would you have drank [sic] more than five full drinks in the one drinking session?’, and thus the outcome at T2 was slightly different from the primary outcome at T3.

The secondary economic effectiveness measure was the number of heavy drinking episodes at T3. This was calculated using the survey data on the frequency of heavy drinking episodes in the previous 30 days collected at the four survey time points. As pupils reported only on the number of heavy drinking episodes they had experienced in the previous 30 days at each of the time point, linear interpolation was used to complete gaps in the survey periods separately for each participant in order to obtain an estimate of their total number of heavy drinking episodes at T2 and T3.

Analysis of outcomes

The analysis of the primary economic effectiveness measure included all pupils responding to the relevant outcome variable at T2 and T3. Descriptive statistics were used to summarise the proportion of pupils in each arm reporting a heavy drinking episode in the previous 30 days at T2 and T3. These effects were analysed using a multilevel mixed-effects logistic regression model controlling for pupils’ baseline drinking, school location, school level of FSM provision, school type and clustering. The incremental effects associated with STAMPP were estimated using the margins command in Stata.

As interpolation was used to calculate the total number of heavy drinking episodes, the analysis of the secondary effectiveness measure included only pupils who had complete heavy drinking episode data to T2 or T3. Descriptive statistics were used to summarise heavy drinking episodes at T2 and T3. These effects were analysed using a multilevel mixed-effects regression model controlling for pupils’ baseline drinking, school location, school level of entitlement to FSMs and clustering. It was not appropriate to discount the primary outcome, as it was dichotomous and measured at a single time point. Non-parametric bootstrapping was used to calculate 95% bootstrap CIs of mean incremental effects drawing 1000 samples of the same size as the original sample separately for each group with replacement. Significance (p < 0.05) was judged when the CIs excluded zero. The number of heavy drinking episodes occurring in T2 and T3 of the study were discounted at 3.5%, in keeping with the NICE (2013) guidance. 77

Cost-effectiveness analysis

For pupils to be included in the CEA they needed to have complete cost and outcome data. Therefore, the population differed from that used in the separate analyses of total costs and outcomes. For the primary analysis at T3, this meant pupils with cost data from baseline to T3 and a response to the primary outcome variable at T3. For the primary CEA at T2, this meant pupils with cost data from baseline to T2 and a response to the outcome variable at T2. For pupils to be included in the secondary CEAs at T3, they needed complete cost data from baseline to T3 and complete heavy drinking episodes data from baseline to T3. For inclusion in the secondary CEA at T2, pupils needed complete cost data from baseline to T2 and complete heavy drinking episodes data from baseline to T2.

The cost-effectiveness of STAMPP was estimated using conventional decision rules102 and reported as an ICER if appropriate. The ICER is a measure of the additional cost per additional unit of effect produced by one intervention compared with another. For an intervention to be considered cost-effective, the ICER must be less than the maximum amount of money that a decision-maker would be willing to pay per unit of effect, that is, their willingness-to-pay (WTP) threshold. Uncertainty in the cost-effectiveness measures was investigated by bootstrapping the multilevel models and using the incremental costs and effects to generate 1000 replications of the ICERs. The resulting replicates were plotted on the cost-effectiveness plane and used to construct CEACs. Construction of these curves involved a series of lines being placed on the plane representing the different WTP thresholds that a decision-maker may have. The proportion of ICER replicates falling below each WTP threshold equated to the probability of STAMPP being cost-effective at that threshold. The curves for the primary CEA specifically showed the probability of STAMPP being cost-effective compared with EAN at different threshold levels of WTP to avoid a pupil experiencing an episode of heavy drinking in the previous 30 days at (1) T3 and (2) T2. The curves for the secondary CEA showed the probability of STAMPP being cost-effective at different threshold levels of WTP to avoid an episode of heavy drinking at (1) T3 and (2) T2. There is no generally accepted threshold value for cost per pupil experiencing any HED avoided or cost per heavy drinking episode avoided. We looked at range of thresholds, including the cost of the intervention per pupil, and compared our findings with those of other economic evaluations that have been performed in this research area.

All analyses were performed using Stata/IC version 12.

Sensitivity analysis

The robustness of the results from the CEA was explored via the following one-way sensitivity analyses.

• Missing total costs and the primary outcome at T3 were filled simultaneously using multiple imputation by chained equations. In the multiple imputation model we included all of the variables that were to be included in the subsequent multilevel models, that is, the treatment variable, baseline costs, baseline drinking, school location, school level of entitlement to FSMs and clustering. We also included 6- to 12-month costs and 6- to 12-month drinking. We used predictive mean matching for costs and a logit model for the primary outcome. Five imputed data sets were generated and the results were combined.

• Total costs and total heavy drinking episodes at T3 were discounted at a rate of 1.5%, as suggested by NICE103 for public health interventions. The multilevel models were re-estimated without adjusting for baseline covariates but still adjusting for clustering.

• As a linear time trend was assumed between data time points, this might have led to total costs and total heavy drinking episodes being under-/overestimated if said trend was not appropriate. The impact of increasing and decreasing total costs and total heavy drinking episodes at T3 by 5% were, therefore, explored.

Results

There were 11,316 pupils present at the baseline assessment: 5749 in the intervention group and 5567 in usual education. A total number of 12,738 pupils took part in the study, with 6379 in the intervention group and 6359 in the control group.

Public sector service use by pupils and associated costs

Pupils’ use of public sector services in the 6 months before baseline, from 6 to 12 months, from 18 to 24 months and from 27 to 33 months is in Appendix 10 (see Tables 39–42, respectively). The public service costs over the study period are shown in Figure 4. The public service costs (£) over the study period by group and by category are shown in Table 43, Appendix 10. As stated in Analysis of pupils’ public sector service use and costs, the analysis of service use included the pupils present at each survey time point.

FIGURE 4.

Public service costs (£) over the study period by group and by category.

The total costs of pupils’ use of public services over the study period following interpolation of costs and adjustment for baseline covariates and clustering are shown in Figure 5 (see also Table 44, Appendix 10). As detailed in Analysis of pupils’ public sector service use and costs, only pupils with complete cost data were included in the analysis of total costs. The numbers were similar across groups for the analysis of total costs at T2 and T3. At T2, the mean cost was £2006 for intervention pupils and £1916 for control pupils. At T3, the mean cost was £2307 for intervention pupils and £2292 for control pupils. Thus, differences were small and not statistically significant. The total costs did not include the cost per pupil to deliver the intervention.

FIGURE 5.

Total public service costs (£) by group and by category at T2 and T3.

Cost-effectiveness analysis

For the purposes of the CEA, outcomes are reported in terms of cases avoided; thus, a positive difference reflects a smaller number of pupils in the intervention arm experiencing a heavy drinking episode in the previous 30 days or a smaller number of heavy drinking episodes being experienced in the intervention arm. Negative costs reflect a cost saving in favour of the intervention. The cost per pupil for the intervention (£15) was added to each pupil in the intervention arm.

At T3 the number of pupils that had complete cost and outcome data and could therefore be included in the CEA was similar across groups (approximately two-thirds of all pupils). At T2 the number of pupils included in the analysis was again similar across groups, but higher than at T3 (approximately three-quarters of all pupils).

The results from the primary CEA at T2 and T3 are shown in Table 30. At T3 there was small cost saving associated with STAMPP (–£17.19) and a significantly greater proportion of pupils experiencing a heavy drinking episode avoided (0.07 or 7%). In other words, the STAMPP intervention is cost-saving and is beneficial in reducing HED. In this situation, the negative ICER is not calculated, as its magnitude does not convey any meaning. 104 STAMPP can be said to dominate EAN; however, as the difference in costs was not statistically different, only weak dominance can be claimed. 105

TABLE 30 - Results of the primary CEA at T2 and T3 and the related sensitivity analyses

Note

The total number of participants in the study was 12,738: intervention, n = 6379 and control, n = 6359.

Analysis	Intervention, number analysed (% total)	Control, number analysed (% total)	Difference in total health service costs (£) (95% CI)	Difference in effect (proportion of pupils who reported a heavy drinking episode in previous 30 days avoided) (95% CI)	Cost per pupil who reported a heavy drinking episode in previous 30 days avoided (95% CI)
T2	4757 (74.8)	4653 (73.2)	62.88 (–262.56 to 388.32)	0.02 (0.01 to 0.03)	3162.09 (–19,921.24 to 26,245.43)
T3	4189 (65.7)	4037 (63.5)	–17.19 (–402.84 to 368.46)	0.07 (0.06 to 0.09)	Dominant
Sensitivity analyses for primary CEA at T3
Multiple imputation for missing cost and outcome data at T3	6379 (100)	6359 (100)	34.10 (–299.44 to 367.44)	0.08 (0.07 to 0.09)	404.88 (3625.58 to 4435.35)
Discounting costs and outcome at 1.5% at T3	–	–	–17.63 (–410.62 to 375.36)	0.07 (0.06 to 0.09)	Dominant
Adjustment of costs and outcome data for cluster only	–	–	–49.23 (–419.52 to 321.07)	0.08 (0.06 to 0.10)	Dominant
5% increase in costs	–	–	–18.05 (–422.98 to 386.88)	0.07 (0.06 to 0.09)	Dominant
5% decrease in costs	–	–	–16.33 (–382.70 to 350.04)	0.07 (0.06 to 0.09)	Dominant

Uncertainty surrounding the estimates of total costs and outcomes is represented by the bootstrapped ICERs on the cost-effectiveness plane for the primary analysis at T3 (Figure 6). The majority of points straddle the north-east and south-east quadrants, indicating that, although STAMPP is likely to be more effective than EAN, there is considerable variability about the cost estimates.

FIGURE 6.

Cost-effectiveness plane for the primary CEA at T3 showing bootstrapped replications of mean incremental costs and pupils who avoided a heavy drinking episode.

At T2, the intervention arm had marginally higher costs and again a significantly greater proportion of pupils who avoided a heavy drinking episode (0.02), equating to a cost per pupil who avoided a heavy drinking episode of £3162. The cost-effectiveness plane for the analysis (Figure 7) shows a similar pattern to the T3 analysis but with a slightly greater proportion of points in the north-east quadrant, indicating that the intervention is associated with slightly higher costs than the analysis at T3.

FIGURE 7.

Cost-effectiveness plane for the primary CEA at T2 showing bootstrapped replications of mean incremental costs and pupils who avoided a heavy drinking episode.

The CEACs for the primary CEA are presented in Figure 8. When WTP thresholds ranging from £0 to £800 are considered, it can be seen that at T3 the probability of STAMPP being cost-effective compared with EAN ranges from 55% to 67%. Uncertainty in the cost-effectiveness of the intervention remains substantial until much higher WTP values, with an 80% probability being displayed at a WTP of £2000. At T2, the probability of cost-effectiveness is consistently lower at each WTP threshold, with values ranging from 35% to 38%. The implication is that, in the short term, STAMPP is not likely to be cost-effective. However, taking a longer-term view, STAMPP is likely to be cost-effective. If decision-makers were only willing to pay the £15 (the cost of STAMPP per pupil), the probability would be 56% at T3 and 35% at T2.

FIGURE 8.

Cost-effectiveness acceptability curves showing the probability of STAMPP being cost-effective compared with EAN for the primary CEA at T2 and T3.

The results of the sensitivity analyses for the primary CEA are presented in Table 30 and the corresponding CEACs are in Figure 9. After the multiple imputation of missing data, the probability of STAMPP being cost-effective was lower at each WTP threshold, ranging from 40% to 55%. The outputs from the models including the Monte Carlo error are presented in Appendix 8, Tables 45 and 46. In contrast, when the multilevel models were re-estimated adjusting only for baseline covariates, the probability of cost-effectiveness was consistently higher, ranging from 65% to 76%. Reducing the discount rate to 1.5% and increasing/decreasing costs by 5% had little effect on the cost-effectiveness of STAMPP.

FIGURE 9.

Cost-effectiveness acceptability curves showing the probability of STAMPP being cost-effective compared with EAN for the sensitivity analyses for primary CEA.

The results from the secondary CEAs are presented in Table 31. For the secondary CEAs at T3, STAMPP was associated with higher costs (£31) and more episodes of heavy drinking avoided (0.73), equating to a cost per heavy drinking episode avoided of £42. Neither difference was statistically significant. The cost-effectiveness plane for this analysis is presented in Figure 10. The majority of points lie in the north-east and south-east quadrants, indicating that STAMPP is likely to be more effective but that there is considerable variability in the cost estimates. For the secondary CEA at T2, STAMPP was again associated with higher costs (£84) and more heavy drinking episodes avoided (0.72), equating to £116.34 per heavy drinking episode avoided. Again, the differences were not statistically significant. The cost-effectiveness plane for the analysis (Figure 11) shows a similar pattern to the T3 analysis but with a slightly greater proportion of points in the north-east quadrant, indicating that the intervention is associated with slightly higher costs than the analysis at T3. The CEACs for the secondary CEAs are presented in Figure 12. When WTP thresholds ranging from £0 to £800 are considered, it can be seen that, at T3, the probability of STAMPP being cost-effective compared with EAN ranges from 43% to 96%. At T2, this probability is lower for lower WTP thresholds, ranging from 30% to 98%. If decision-makers were willing to pay only £15 (the cost of STAMPP per pupil), the probability would be 45% at T3 and 32% at T2.

TABLE 31 - Results of secondary CEA

Note

The total number of participants in the study was 12,738: intervention, n = 6379 and control, n = 6359.

Analysis	Intervention, number analysed	Control, number analysed	Difference in total health service costs (£) (95% CI)	Difference in effect (cases of heavy drinking episodes avoided) (95% CI)	Cost per heavy drinking episode avoided (95% CI)
T2	4683	4578	83.98 (–234.26 to 402.22)	0.72 (0.29 to 1.16)	116.34 (–1426.35 to 1659.03)
T3	4128	3945	31.27 (–351.67 to 414.21)	0.73 (0.15 to 1.30)	42.93 (–2530.65 to 2616.51)
Sensitivity analyses for secondary analysis
Discounting costs and outcome at 1.5% at T3	4128	3945	30.42 (–360.24 to 421.08)	0.74 (0.16 to 1.33)	40.85 (–2616.84 to 2698.53)
Adjustment of costs and outcome data for cluster only	4128	3945	–8.47 (–383.42 to 366.47)	1.00 (0.34 to 1.65)	Dominant
5% increase in costs	4128	3945	29.71 (–334.09 to 393.50)	0.73 (0.15 to 1.30)	40.78 (–2404.11 to 2485.68)
5% decrease in costs	4128	3945	32.83 (–369.25 to 434.92)	0.73 (0.15 to 1.30)	45.08 (–2657.18 to 2747.33)
5% increase in cases of heavy drinking	4128	3945	31.27 (–351.67 to 414.21)	0.77 (0.16 to 1.37)	40.89 (–2410.14 to 2491.91)
5% decrease in cases of heavy drinking	4128	3945	31.27 (–351.67 to 414.21)	0.69 (0.15 to 1.24)	45.19 (–2663.84 to 2754.22)

FIGURE 10.

Cost-effectiveness plane for the secondary CEA at T3 showing bootstrapped replications of mean incremental costs and heavy drinking episodes avoided.

FIGURE 11.

Cost-effectiveness plane for the secondary CEA at T2 showing bootstrapped replications of mean incremental costs and heavy drinking episodes avoided.

FIGURE 12.

Cost-effectiveness acceptability curves showing the probability of STAMPP being cost-effective compared with EAN for the secondary CEAs at T2 and T3.

The results of the sensitivity analyses for the secondary CEAs are presented in Table 31, and the corresponding CEACs are presented in Figure 13. When the multilevel models were re-estimated, adjusting only for baseline covariates, the probability of STAMPP being cost-effective was consistently higher, ranging from 52% to 100%. Reducing the discount rate to 1.5% and increasing/decreasing costs and effects by 5% had little effect on the cost-effectiveness of STAMPP.

FIGURE 13.

Cost-effectiveness acceptability curves showing the probability of STAMPP being cost-effective compared with EAN for the sensitivity analyses for the secondary CEAs.

Summary of economic analysis

The Steps Towards Alcohol Misuse Prevention Programme is a relatively low-cost intervention that successfully reduces HED. STAMPP did not bring about clear public sector cost-savings; however, it neither increased them nor lead to any cost-shifting within the public sector categories. STAMPP can therefore be considered to weakly dominate EAN because it is both cost-neutral and more effective. Furthermore, STAMPP appears to be more cost-effective in the longer term, as it has a greater impact on pupils when they are older and more likely to be consuming alcohol. Further research is required to establish whether or not the cost-effectiveness of STAMPP is sustained in the long term.

Introduction

In a large CRCT (STAMPP) combining an adapted version of the SHAHRP with a brief parental information component, we found that the intervention reduced self-reported HED in the past 30 days at T3 follow-up compared with EAN, but that it did not reduce self-reported harms associated with own drinking. Similarly, intervention pupils reported significantly fewer heavy drinking episodes in total at T2 and T3 than control pupils. There were no clear cost savings in terms of service utilisation associated with the intervention. However, from a health economic perspective, STAMPP was considered to weakly dominate EAN, as it was associated with a cost saving and was more effective.

At first glance, it is difficult to reconcile these apparently inconsistent findings. However, one explanation may lie in the manner in which all three outcomes were assessed and/or analysed. HED was assessed using one question written on a detailed and colourful page including pictures of drinks and corresponding unit content, allowing participants to accurately determine drinking in the past month. With regard to harms, it is possible that the specific harms examined were not age-appropriate. Although they are similar to those used in previous Australian and Northern Irish studies, the participants in the present study were approximately 1 year younger than the participants in both of those studies. On reflection, harms examined such as getting sick, getting in trouble with the police or attending a hospital may have been less relevant than the unasked harms of, for example, losing possessions, losing friendships or damaging their reputation that probably accompany early adolescent drinking behaviour. In addition, not taking frequency of harms into account in the primary analyses may have resulted in a loss of information. This suggests that our primary outcome measure of alcohol-related harm, as specified in the DAP, may not have been sensitive enough to detect age-appropriate harms, and thus was not able to identify any other alcohol-related harm (in addition to harm related to alcohol toxicity) that may have occurred and/or been prevented during the trial.

There were no clear or consistent effects identified in planned secondary or subgroup analyses [age, gender, SES, alcohol use at baseline, location (Scotland vs. NI)]. Stronger claims about the preventative impact of the intervention could be made if a clear pattern of consistent effects across a range of related outcome measures had been identified. It is possible that longer-term follow-up might reveal such effects, especially with regard to self-reported harms, which were low in both control and intervention pupils (the median number of harms was 0 in both arms; 68% of existing drinkers in control schools reported experiencing one or more harms at final follow-up compared with around 62% of existing drinkers in intervention schools). One indication of the potential developmental prevention impact of the intervention is that, at T2, HED showed a small reduction associated with the intervention, an impact that was stronger at T3 follow-up, albeit with a different follow-up measure. It is plausible that the final outcome measure for HED used at T3 was a more sensitive measure than the other measure used at earlier follow-ups (more than five drinks in an episode) and we cannot discount this possibility. The use of mediation analysis to assess lagged effects of changes in hypothesised targeted programme constructs may help to better understand these developmental effects.

In the light of the lack of statistically significant subgroup interaction effects and suggestions of multicollinearity between HED at baseline and the main effects in the model, we undertook additional exploratory analyses outside of the DAP. When the primary outcome models were restricted to just drinkers (defined as either lifetime use or previous year’s use at baseline), drinking pupils in the intervention schools reported fewer harms than those drinking pupils in the control schools. However, it must be noted that when the lifetime use and previous year’s use subgroup effects were examined via interaction terms (on the full CC population), the interaction terms for harms were non-significant, as were the interaction terms for age at onset and unsupervised drinking.

A number of sensitivity analyses were conducted to estimate the impact of analytical decisions on the HED primary outcome. These all supported the chosen analysis, with the exception of the conservative case model (for which missing cases in the intervention arm were all assumed to have engaged in HED and missing cases in the control arm were set to non-drinking), in which the sign of the intervention coefficient changed while remaining significant. This indicated that the model results were robust (i.e. a school effect was found in four of the five tests), except when subject to extreme missing data assumptions (i.e. the conservative case model).

At baseline, 7.8% of control and 7.6% of intervention pupils reported HED (defined as drinking more than five drinks in one episode). The prevalence of HED (using the primary outcome definition) was 26% in the control group and 17% in the intervention group at final follow-up. It is difficult to directly compare these figures with other contemporaneous UK surveys of adolescent alcohol use because of the use of different questions, sampled age groups and recall periods,106 but data from the control group seem broadly equivalent to other estimates. For example, 53.3% of 11- to 16-year-old pupils surveyed in the 2013 NI Young Persons Behaviour and Attitudes survey reported being drunk on at least one occasion in the previous month. The 2013 Scottish Schools Adolescent Lifestyle and Substance Use Survey107 reported that 44% of 15-year-olds reported having ever felt ‘really drunk/drunk’ in their lifetime. In England, from the Smoking, Drinking and Drug Use among Young People in England survey,12 12.3% of 13-year-olds (equivalent to our baseline sample mean age) reported being drunk (self-defined) in the previous 4 weeks, and this increased to 53% of 15-year-old lifetime drinkers (comparable with our T3 sample). For comparison, in the 2011 pan-European Union ESPAD, 54% of 15- to 16-year-olds reported consuming more than five drinks in a single episode.

A Cochrane review of the effectiveness of universal school-based alcohol education programmes identified 53 studies for inclusion, including 11 alcohol-specific interventions (the remainder were generic approaches or targeted multiple substances). 47,108 No included studies were conducted in the UK. Of the 11 alcohol-specific interventions, six found beneficial effects of intervention. Outcome measures differed between those studies reporting significant findings, although three reported similar outcome measures to those included here (‘binge drinking’). In the original SHAHRP evaluation conducted in Australia, significant intervention effects were reported at 32 months and, although ORs were not presented, it was estimated that intervention pupils were 4.2% less likely to consume at risky levels [defined as reporting two (female)/four (male) standard drinks (10 g of alcohol) per occasion once per month or more often]. 55 A German skills-based activity delivered over four interactive lessons to a similarly aged sample as STAMPP was associated with an OR of lifetime ‘binge drinking’ of OR 0.56 (95% CI 0.41 to 0.77) at 4 months and an OR of 0.74 (95% CI 0.57 to 0.97) at 12 months. 109 Finally, an Australian online alcohol harm reduction curriculum for 13-year-olds (CLIMATE Alcohol Program) reported significant intervention effects (no OR reported) for ‘binge drinking’ (a single occasion over the previous 3 months) for girls but not for boys 12 months after delivery. 57 Interestingly, and in contrast with STAMPP, self-reported harms were also significantly reduced in girls using the same assessment scale employed in the current study. Finally, for comparison, the earlier NI study of the SHAHRP60 (32 months of follow-up), which employed a non-randomised design and used latent class growth modelling, intervention pupils were more likely to be in trajectories that reported fewer alcohol units consumed in the previous episode and fewer alcohol-related harms (no OR reported). Overall, the OR associated with STAMPP (OR 0.596, 95% CI 0.49 to 0.73) suggests that programme effects are comparable with those reported previously for similar interventions using similar outcome measures. However, caution is warranted when drawing comparisons between STAMPP and earlier studies conducted in Australia and NI. STAMPP used a different research design (CRCT vs. quasi-experimental designs in the Australian and NI SHAHRP studies), a younger age group and a different outcome measure at final follow-up. We also incorporated a parental intervention (albeit with low uptake). Furthermore, the earlier NI study was delivered against a higher population prevalence of drinking.

The total cost to deliver STAMPP was £85,900, equivalent to £818 per school and £15 per pupil. NICE published public health guidance53 for use in primary and secondary schools on sensible alcohol consumption, which revealed the paucity of evidence from economic evaluations in this area. The authors of the review identified three studies that provided sufficient information on resource use to allow a cost per student to be calculated. The costs ranged from £20 to £150 (cost year 2005/6). 55,110,111 Thus, at a cost of £15 per pupil, STAMPP is a relatively low-cost intervention that successfully reduces HED. These costs reflect the cost of introducing and delivering STAMPP on one occasion. In reality, once teachers have been trained they will not need to be retrained on an annual basis, requiring only refresher training. Similarly, if STAMPP was delivered instead of existing alcohol EAN, which would likely be the case based on the findings of our process interviews, the cost per pupil would also fall.

The analysis of public service costs showed that there was an overall reduction in the use of public sector services over the 33-month study period for both groups. There were, however, no differences between groups and no differences in the use of the subcategories of education, health and criminal justice services. Costs were estimated from a public sector perspective, which was justified considering one of the principal objectives of STAMPP was to reduce alcohol-related harms in teenagers. It was hypothesised that this would in turn reduce the use of health and judicial services and the need for additional support within the school setting to address behavioural and emotional problems. The absence of statistically significant difference in public service costs between groups is in keeping with the analysis of the harms data from the trial [see Chapter 3, Drinking harms (T3)]. No differences were observed in the number of self-reported harms by pupils between groups and, indeed, both groups reported low levels of harms overall.

The primary CEA (using the number of pupils experiencing a heavy drinking episode in the previous 30 days) at T3 indicated that STAMPP weakly dominated EAN. At a notional WTP threshold of £15 (reflecting the cost of STAMPP per pupil observed in this study), the probability of STAMPP being cost-effective was 56%. This level of uncertainty reflects the considerable variability in the cost differences between groups. At T2, this probability of cost-effectiveness was considerably lower, at 35%, because of the additional variability in the effectiveness of the intervention. A similar pattern of results was observed in the secondary CEAs (using the number of heavy drinking episodes); the intervention was more cost-effective at T3 than at T2. However, the probability of cost-effectiveness was lower at each time point (45% and 32%, respectively, at a WTP of £15) than in the primary CEA. This was because of greater variability in both costs and outcomes. The greater variability in effectiveness is not surprising considering that the effectiveness measure used in the secondary CEAs was based on a continuous outcome, in contrast to the binary outcome used in the primary analysis. Overall, the implication is that STAMPP is more cost-effective in the longer term, as it has a greater impact on pupils when they are older, more likely to drink and drinking more. In light of the literature, which links heavy drinking in adolescence to alcohol dependence and poor health outcomes in adulthood (e.g. Bonomo et al. 2), it is important to investigate if the (cost) effectiveness of STAMPP is sustained or even increases in the long term.

The sensitivity analyses indicated that the results of the CEA were robust to small changes in the parameters, that is, discounting and small increases in cost and effectiveness. However, when costs and effects were not adjusted for baseline covariates, the probability of STAMPP being cost-effective increased. This suggests that the cost-effectiveness of STAMPP may vary between subgroups and warrants further investigation to identify which pupils and/or schools might benefit the most from the receiving the intervention. Furthermore, when multiple imputation was used to impute missing cost and outcome data, the cost-effectiveness of STAMPP decreased. Considering multiple imputation is based on the assumption that data are missing at random, it is of some concern that the results of this sensitivity analysis differ somewhat from the primary analysis. The proportions of missing data in different groups were similar within all of the health economic analyses; thus, further investigation is warranted into the imputation model used as well as the pattern of missingness.

The process evaluation showed that clusters were successfully recruited into STAMPP, randomisation was successful, and pupils in schools were comparable across intervention arms at baseline. No adverse events were reported. The intervention was delivered with a good degree of fidelity and was enjoyed and/or acceptable to students, teachers, schools and other stakeholders. Overall, pupils thought that the intervention content was age appropriate, although from the comments received in some focus groups (and from the survey data), even within the same year group, pupils will have different drinking histories and there will be differences in rates of initiation and in establishment of more regular drinking patterns. The standard materials provided to pupils, in particular those focusing on the consequences of drinking, may therefore need to be adapted for some target groups if the intervention is delivered in routine practice. Furthermore, although participants valued the workbooks, pupils have changing expectations with regard to modes of delivery of learning materials, particularly with regard to new technologies and platforms, and so intervention materials may need to be adapted to keep pace with changes in learning platforms (e.g. electronic materials delivered through online platforms or tablets).

In contrast, there was very low uptake of the parental/carer component, and postal returns of the parent/carer survey, which were used as an indicator of implementation of mailed intervention materials, were also relatively low. This component was therefore not successfully delivered. We did not have a comprehensive response to the teacher survey assessing EAN practice in control schools, but reports from responders indicated that EAN was rarely delivered (n = 3 reports) and primarily consisted of single sessions of general alcohol awareness activities delivered by external organisations (e.g. police, alcohol charities). Therefore, we concluded that there was a clear differentiation between intervention and control schools.

Strengths and limitations

The key strengths of the trial were the large sample size (schools and pupils), low rates of attrition (no schools dropped out) and relatively high rates of matched data (> 80% depending on outcome) across survey waves. This means that the primary analysis on HED was sufficiently powered. This calculation used estimates of HED derived from the 2011 ESPAD study13 for the same age as the STAMPP pupils, with the estimated ICC derived from the Belfast Youth Development Survey. 75 Unfortunately, neither the Belfast Youth Development Survey nor the ESPAD contained comparable measures of drinking harms. The NI SHAHRP study,60 which did use a self-report harm measure, was undertaken with an older age group of pupils, and so was also not an appropriate base for a sample size calculation. Therefore, as no formal sample size calculation was undertaken for alcohol-related harms, there is the possibility that the null result for this outcome was because of lack of power. However, given the large achieved sample size, the relatively low levels of subject attrition, the use of covariates within the models (providing additional power) and the relatively small observed differences between study arms, we do not suspect that the null finding was because of insufficient sample size. It is more likely that the self-reported harm measures were not sensitive enough to likely harms experienced by the participants and thus any potential benefit was hidden. The classroom component was delivered with acceptable fidelity and was positively received by both pupils and teachers. However, there was a higher dropout rate (i.e. non-matched data across T0 to T3) among pupils who were male (19%), who were in receipt of FSMs (25.8%) or who had used alcohol at baseline (25.4%). Although we controlled for these variables in our analyses, it is uncertain why there was a higher level of missing data in these groups. Sex, SES and previous alcohol use have been shown in other UK studies to be predictors of school non-attendance (e.g. truanting, exclusion),12,112,113 although this would account for only some of the missing data. We were unable to identify previous studies examining predictors of retention or missing data in UK adolescent alcohol prevention interventions, but in the Australian and NI trials of the SHAHRP, there was an overall attrition of 24.1% and 12.8% at 32 months, respectively. 58,60 In the trial of the Dutch Preventing Heavy Alcohol Use in Adolescents (PAS) programme, from which our parental intervention was derived, overall attrition was 12.5%, and dropouts differed from completers in being older, drinking more and having parents with lower levels of education. 64 A systematic review of universal school-based interventions concluded that there was no difference between effective and non-effective interventions on the basis of attrition. 108 It would therefore be important for future work to determine why baseline drinking groups in particular produced more missing data, as this group would potentially benefit most from alcohol interventions, and inclusion in the data set may have adjusted our analyses.

A major limitation of the work was the failure to attract parents/carers to the brief intervention evening (9% in NI and 2.5% in Scotland), despite the support of many of the schools. Relatively low rates of return of the parental questionnaire (31% and 18%, respectively) also suggested that only a minority may have read the mailed information. Although we conducted an ITT analysis, which helped to preserve sample size, and achieved participation rates are likely to reflect family attendance in routine practice,114–116 this meant that we were unable to draw any confident inferences about the combined impact of the school and parental intervention (see Koning et al. 117) or the relative contribution of each component. In practical terms, this means that although the analyses presumed delivery of the combined intervention, discussions with stakeholders about research findings and future delivery are likely to focus on the classroom component (i.e. SHAHRP).

Failure to engage parents/carers in school-based substance use prevention is a consistent finding. 118–122 However, other trials have reported success at engaging family members. For example, in a recent feasibility study of the Welsh family-based alcohol prevention intervention Kids, Adults Together programme,123 50% of pupils (n = 158 intervention pupils in total) reported that at least one family member who was invited to a family event attended (although only 6.5% of eligible parents/carers returned a study questionnaire), suggesting that acceptable participation rates in the UK are achievable. These authors identified two key processes that they believe supported engagement. First, pupils were keen to attend the event with their parents/carers and, second, the family event was not marketed as an alcohol education event and was positioned around parents/carers wanting to attend the event to see their children’s school work and what activities they had been involved in. Similarly, in the Dutch PAS study there was a high level of parental retention in the parent only (75.9%) and combined parent and student (72.4%) intervention arms. 64 In keeping with the Kids, Adults Together programme, PAS parental events were part of regular school parents’ evenings, which a large number would have attended anyway. Future implementers of STAMPP should therefore consider such engagement approaches, which were not feasible in the current trial because of the timing of intervention delivery, the large number of schools involved which made co-ordination difficult, a lack of time within regularly planned parents evenings (which primarily focus on pupil progress) and education policy initiatives in one trial site that necessitated using parental evening time to introduce a new curriculum.

Our primary outcome assessments relied on self-report, which may have led to inaccurate reporting of alcohol use and associated harms through memory, social desirability and other biases. 124 Although adolescent self-reported alcohol questionnaires are generally reliable,125,126 there may be differences in reliability between early and late adolescence63 and studies of recanting in substance use surveys suggest that this may be an understudied bias in prevention research. 127 However, all pupils received the same questionnaire and pictorial prompts, and the recall period for the primary outcome used in this study was the previous 30 days, and so, if bias had existed, this would have been minimal and equivalent across trial arms. Less attention has been paid to the validity of assessments of alcohol-related harms, although similar social desirability and self-representation biases are likely to exist. 128 In this study, alcohol-related harms were measured using a 16-item scale, and previous work has shown the scale to have an internal consistency of α = 0.9. However, we do not know if pupils consistently interpreted the harms in the same way. Although some of the self-reported harms were likely to be interpreted in a straightforward manner (e.g. ‘did you vomit after drinking?’), similarly to differences in young people’s self-perception of drunkenness,129 there may be individual differences in interpretation of the harms assessed in this study and different thresholds applied for an indicator being perceived as being a ‘problem’ (e.g. having a hangover after drinking). As mentioned previously, it is also possible that some of these harms were not age appropriate and thus these were low frequency in our population. Without a method for objectively verifying the level of harm in this population, it is difficult to know whether or not these self-reported data were biased.

The assessment of HED (or ‘binge drinking’) in adolescents, as used in our consumption primary outcome, is complicated by the lack of standardised definitions in both adults and adolescents. 130 We adapted the current CMO’s guidelines for adults65 for this study, but regardless of this problem we were able to show a reduction in our measure of alcohol consumption. The introduction of the improved pictorial response sheet no doubt enhanced the accuracy of responses and reduced any potential bias given the problems with the concept of a ‘drink’ for not only the participants but for the research community as a whole.

There were a number of limitations to the economic evaluation. The study was not specifically powered to detect statistically significant differences in costs or cost-effectiveness. Although CEA does not typically make decisions based on significance rules,131 having a sufficiently powered study will allow decision-makers to be more confident in the value claim. 104 The resources used during the planning, preparation and delivery of the intervention were largely recorded retrospectively, and costs were obtained from invoices when these were available. We endeavoured to use plausible assumptions when actual data were not available, but the consistent and prospective collection of resource use and costs would lead to more robust data. The resource use questionnaire was completed by the pupils without any input from their parents or guardians. This was done because of resource limitations and to preserve confidentiality, and although definitions of the services were provided and the terminology simplified, it could be argued that more accurate costs would have been obtained with parental input. However, it was difficult to engage parents in the intervention, as reflected in the poor attendance to the parental evenings (see Chapter 4, Fidelity of implementation of STAMPP), so it is likely that response rates would have been poor.

We included pupils in the CEA only if they had complete cost and effect data. As a result, only two-thirds of the pupils were included in the T3 CEA and three-quarters in the T2, and the rest we assumed were missing at random. As discussed earlier, further investigation is required to establish whether or not this assumption is flawed.

The curriculum was delivered in most schools as part of their PSHE education (or local equivalent) curriculum and did not replace statutory activities. However, we did not assess spillover effects of STAMPP on other types of related school activity or curriculum, and so this must be considered a limitation of the trial.

Our approach to assessing fidelity of implementation and comparator bias was pragmatic in the context of the resources available and the large number of schools enrolled in the trial. Although our assessment of fidelity was based on an existing framework,132 and provided useful information, ideally, in addition to self-report we would have preferred to have recorded and/or observed some classroom and parent/carer sessions for independent rating of deliverer competencies, quality and completeness of delivery and target group responsiveness. Similarly, although we are confident that we identified other alcohol actions delivered to schools and we are able to conclude that delivery of competing interventions was very low, we were unable to make comparable assessments of exposure to community-based alcohol activities such as mass media campaigns and health promotion with an alcohol component. However, we were unaware of any major initiatives being delivered across the course of the trial, and the successful randomisation would have militated against some effects.

Other weaknesses and limitations of the research are identified in Self-assessment of risk of bias.

Further research

We found that STAMPP was effective in reducing self-reported HED T3, when pupils were aged 15–16 years. It will be important to assess whether or not these reductions are sustained and if effects on harms emerge as pupils get older, alcohol use behaviours become more frequent and patterns of use are established. 12,133,134 Booster interventions (e.g. brief interventions of proven effectiveness in adolescents and young adults) that build upon the skills developed in STAMPP may prove useful in sustaining behavioural change. 135 As the trial team included those who were responsible for the adaptation and development of the intervention, it is also important that any future replications are conducted independently. 136 The use of data linkage techniques to match the study cohort to additional educational, community and statutory service data sets (e.g. examination performance, hospital admissions, GP data, involvement with the criminal justice system) may also help to understand whether or not the changes reported in the current study lead to meaningful changes in health and well-being outcomes. 137 This recommendation also applies to other alcohol prevention research that predominantly uses simple proxy outcomes of use,49 and this practice has been criticised for not providing useful information for commissioners who are tasked with funding interventions on the basis of demonstrable improvements in health and well-being.

In an earlier secondary analysis assessing implementation of the adapted SHAHRP in NI,138 multilevel growth modelling was used to examine differential intervention impact when recipients were retrospectively classed according to alcohol use status (abstainer or existing drinker) and context of use (unsupervised or parental/carer supervised drinker) at baseline. Significant positive behavioural effects in terms of amounts consumed, frequency of drinking and self-reported alcohol-related harms were observed almost exclusively among pupils classed as baseline unsupervised drinkers. This was notable, as although the SHAHRP was delivered as a universal curriculum, it suggested that it might also have utility as a targeted intervention. In the current study, the preplanned subgroup analysis showed that there was no interaction between baseline drinking status and treatment effect. However, when exploratory analyses were undertaken, which examined drinking status groups independently, significant intervention effects emerged (HED and self-reported harms). These findings suggest that STAMPP may have had a differential impact on those pupils who would be considered most at risk from alcohol use (e.g. unsupervised baseline drinkers). 139

With respect to further development of STAMPP intervention, our rich data set means that it will be possible to conduct mediation analysis to further develop programme theory and to test the assumptions of our logic model (see Appendix 1). This may lead to a better understanding of which components of the intervention (e.g. specific lessons) were most successful and which require strengthening. 140 Identifying supportive or inert elements of the programme may lead to the development of a shorter optimised curriculum, which would reduce resource requirements and potentially increase the attractiveness of the intervention to funders. Similarly, analysis of moderation effects might identify local contextual and population factors that exert differential influences on outcomes. 141

Extending this line of work, research examining the fidelity of implementation in more detail may help to refine delivery. Although we reported relatively high completeness of delivery with respect to content, there was variation in the number of lessons required to deliver, and it is uncertain what effect this may have had on programme outcomes. In a secondary analysis of the European Drug Addiction Prevention (EU-Dap) trial substance use prevention curriculum, another skills-based interactive prevention programme, class size, composition (e.g. sex ratio, academic ability) and social connectedness between pupils, were shown to be important predictors of programme implementation. 114 Understanding these factors is important, because in routine practice, outside of the structures of a RCT, the intervention may not be delivered as intended, and formal and informal changes introduced by delivery staff may lead to a loss of programme integrity. 90 Furthermore, although based in the classroom, the adapted SHAHRP curriculum may not necessarily be optimally delivered by teachers,60 and some pupils in the process evaluation suggested that their response to the lessons was dependent on pre-existing relationships with school staff. In future programme development and evaluation, different trial arms should include the assessment of alternative deliverers, such as trained prevention and youth service workers, who have specialist skills to help better engage young people in health programmes.

Considering our failure to recruit into the parental intervention, further research is required to better understand how to engage and retain parents/carers in prevention activities. 91,123 This is also important, as delivery of preventative activities outside of the structures of research trials frequently leads to lower implementation quality. 90 Universal interventions such as STAMPP require a range of recruitment strategies, as there will be different barriers to, and facilitators of, attendance in parental/carer-based actions. Research is therefore needed to assess the relative efficacy of recruitment strategies such as incentives, mass media campaigns, the removal of barriers to attendance (e.g. providing transport and childcare) and the use of key community recruiters (influential individuals and organisations). 91 Furthermore, it is also important to understand if some parent/carer subgroups (e.g. differentiated according to child drinking risk) are more likely to respond to particular recruitment strategies and if this will lead to recruitment biases.

Broadening our research recommendations to the wider prevention field, and drawing upon our stakeholder interviews, it is clear that although universal interventions are valued, they compete for resources and must sit alongside other alcohol-related community actions and policy initiatives, which may moderate observed effects. 92 By-laws and licensing decisions can affect local alcohol environments (e.g. density of alcohol outlets, opening hours and local marketing), and actions such as industry-driven corporate social responsibility initiatives, marketing and packaging regulations and community-based initiatives may complement or disrupt school-based actions. 93–97 There is also increasing interest in schools as environments for health promotion through actions that modify the physical and social environment. 98 Interventions and curricula, such as STAMPP, are therefore not being delivered in isolation, and, although study design characteristics such as randomisation control for internal biases, there is a need to better understand how interventions complement each other, the dynamic interplay between intervention components at different levels (e.g. the interaction between norms correction activities in a prevention curriculum and the placement of alcohol advertisements in the local area), and how interventions can be optimised within such complex health systems. 99 The use of systems mapping exercises and the study of alcohol prevention as a complex system may be one means to maximise effects from combinations of different prevention types. 100

Implications for practice

Our findings suggest that STAMPP is a candidate for inclusion in local strategies to reduce alcohol-related harm where reduction in HED is a stated aim. Although the reported outcomes were relatively modest, STAMPP is one of the few UK school-based alcohol prevention programmes to show effectiveness in reducing HED in adolescents. Although any universal prevention programme on its own is unlikely to lead to sustained changes in population levels of alcohol use,92,101 the harm reduction focus of the classroom component of STAMPP may complement national and local actions targeting price, availability and affordability of alcohol and the licensing of alcohol outlets. Furthermore, although we have not yet conducted such an analysis, previous studies of universal prevention programmes have also shown benefits for participants considered at greater risk of harm, whether as a result of their substance use behaviour or population characteristics (e.g. Vigna-Taglianti et al. 102 and Spoth et al. 103). With appropriate adaptation (e.g. language and comprehensibility, delivery outside mainstream classrooms), STAMPP may be a useful form of alcohol education for higher-risk and vulnerable groups.

The classroom intervention was easily implemented by teachers (conditional on support by school management) after suitable training and support materials were relatively inexpensive (workbooks and supporting CD/electronic material). Schools that have identified alcohol education as a priority would be in a good position to offer the classroom curriculum as part of existing PSHE provision, and teachers in this subject are likely to have the required general professional skills to facilitate delivery. Although we acknowledge curriculum pressures, considering the current poor state of substance use education in the UK,104,105 STAMPP would make a useful contribution to health and social education in schools.

However, in the current trial, uptake of the brief intervention was poor, and although all intervention students parent(s)/carer(s) received the intervention leaflet, the return of materials to indicate reading/acknowledgement was low. It is, therefore, difficult to make any practice recommendations about this component of the programme. If commissioners or providers decide to implement STAMPP in future, then they must decide whether the full programme or only the classroom intervention is delivered. Relying on passive mechanisms (e.g. advertisements and information) is unlikely to encourage attendance and, although some of the techniques to encourage participation discussed in this section are likely to be costly (e.g. providing transport and childcare), others rely more on planning than resources (e.g. scheduling events as part of regular school parents’ evenings).

Generalisability

Although we are mindful of differences in school autonomy, governance and oversight, and we acknowledge regional variability in alcohol use behaviours (e.g. Public Health England9), we believe that the findings of this trial are generalisable to other geographies of the UK. However, as discussed throughout this report, parental/carer engagement was poor, and so generalisability may only be relevant to the classroom curriculum. Schools enrolled in the trial were drawn from urban and more rural areas, and from across the socioeconomic gradient. Furthermore, subgroup analysis showed that there were no differential intervention effects on the basis of school geography (i.e. NI vs. Glasgow/Inverclyde). Interviews with stakeholders, including local commissioners, did not identify any significant barriers to future delivery, beyond those expected relating to funding and local priorities. Similarly, teachers and senior school staff believed that STAMPP would help them to achieve their health and well-being aims in line with wider school policies.

Self-assessment of risk of bias

We conducted a self-assessment of bias in accordance with criteria adapted from the Cochrane Collaboration methodological handbook. 106 For each domain, TMG members assessed the level of bias with respect to three outcomes: unclear risk, which is interpreted as plausible bias that raises some doubt about the results; low risk, which is interpreted as plausible bias that is unlikely to seriously alter the results; medium risk, which is interpreted as plausible bias that moderately weakens confidence in the results; and high risk, which is interpreted as plausible bias that seriously weakens confidence in the results. Final ratings were achieved through consensus. We identified a medium risk of bias in relation to performance bias, as this was not a double-blind study. However, using an EAN comparator, it was not possible to conceal intervention allocation from teachers, who received specialised training and curriculum materials, or from pupils, who would typically receive little or no alcohol education in their usual school year. We self-rated detection bias as having a medium risk. This was because of resource constraints and because some of the data collection was undertaken by members of the trial team. We self-assessed conflict of interest bias as having a medium/high risk for two main reasons. The first reason was that the printing of curriculum workbooks in one intervention site (Glasgow only) was funded through dedicated money awarded by the alcohol industry. However, this funder was not involved in any aspects of research or intervention design and did not have any subsequent involvement in the trial, its management, analysis or write up. The second reason underpinning this rating was related to the involvement of three members of the trial team (including the principal investigator) in the original NI adaptation of the school curriculum, and all members of the TMG contributed to the development of the parental component. Finally, two members of the TMG reported that their university departments had historically received research grants from parts of the alcohol industry for unrelated research. Neither had individually directly benefited from this funding and it is reported here for transparency.

Contributions of authors

Harry Sumnall (Professor, Substance Use, Liverpool John Moores University) was the project lead; he contributed to the conception and overall design of the study, and to the writing and editing of the final report.

Ashley Agus (Health Economist, NICTU) conducted the health economic evaluation and write-up.

Jon Cole (Professor, Tactical Decision-Making, University of Liverpool) contributed to the conception and overall design of the study, and to the writing and editing of the final report.

Paul Doherty (Data Manager, NICTU) was responsible for data management, including quality assurance and data governance.

David Foxcroft (Professor, Community Psychology and Public Health, Oxford Brookes University) contributed to the conception and overall design of the study, and to the writing and editing of the final report.

Séamus Harvey (Researcher, Liverpool John Moores University) was the trial researcher; he contributed to all aspects of the work and was responsible for analysing and writing the process evaluation chapter and contributing to the final report.

Michael McKay (STAMPP Trial Manager, Liverpool John Moores University) was the trial manager; he oversaw teacher training, was the liaison lead with schools and stakeholders, led the field research and contributed to the conception and overall design of the study, as well as to the writing and editing of the final report.

Lynn Murphy (Manager, NICTU) contributed to the conception and overall design of the study, and managed the contribution of the NICTU.

Andrew Percy (Senior Lecturer, Queen’s University Belfast) contributed to the conception and overall design of the study, undertook the statistical analysis and contributed to the writing and editing of the results chapter and the final report.

Publications

McBride N, Mckay M, Sumnall HR. SHAHRP: School Health and Alcohol Harm Reduction Project – developments in Australia and the UK. Educ Health 2013;31:79-83.

McKay MT, Cole JC, Percy A, Worrell F, Mello ZR. Reliability and factorial validity of Adolescent Time Inventory-Time Attitude (ATI-TA) scores in Scottish and Northern Irish adolescents. Pers Individual Diff 2015;86:412–16.

Morgan GB, Wells KE, Andretta JR, McKay MT. Temporal attitudes profile transition among adolescents: a longitudinal examination using mover-stayer latent transition analysis [published online ahead of print 5 September 2016]. Psychol Assess 2016.

Harvey SA, McKay MT, Sumnall HR. Adolescents’ reflections on school-based alcohol education in the UK: education as usual compared with a structured harm reduction intervention. J Subst Use 2016;21:640–5.

McKay MT, Percy A, Cole JC, Worrell FC, Andretta JR. The relationship between time attitudes profiles and self-efficacy, sensation seeking, and alcohol use: an exploratory study. Pers Individual Diff 2016;97:203–9.

Protocol

Available from www.nets.nihr.ac.uk/projects/phr/10300209 (last accessed 6 April 2016).

Data sharing statement

Anonymised data from this study can be obtained from the corresponding author.

Disclaimers

This report presents independent research funded by the National Institute for Health Research (NIHR). The views and opinions expressed by authors in this publication are those of the authors and do not necessarily reflect those of the NHS, the NIHR, NETSCC, the PHR programme or the Department of Health. If there are verbatim quotations included in this publication the views and opinions expressed by the interviewees are those of the interviewees and do not necessarily reflect those of the authors, those of the NHS, the NIHR, NETSCC, the PHR programme or the Department of Health.

Parental permission for child participation in STAMPP

(PDF download)

Student consent to participate in STAMPP

(PDF download)

Primary outcome 2: questions assessing harms resulting from own use of alcohol at 33 months after baseline

Pupils

Head teachers/principals

Public health, education and other stakeholders in trial geographies

1. Is STAMPP similar to or distinct from other approaches to alcohol use in young people in the region/city?

2. What do you know about the aims, objectives and expected outcomes of STAMPP?

3. How does STAMPP fit in with your alcohol/health policy or strategic priorities for young people?

4. In your role, do you have commissioning responsibility for interventions such as STAMPP?

5. With respect to addressing alcohol-related issues, what’s your opinion on the relative contribution, impact and complementarity of alcohol education activities compared with other types of responses? (prompt: e.g. licensing, regulation, taxation, etc.)

The next questions assume that the SHAHRP/STAMPP is shown to be effective:

1. If shown to be effective, do you think it is likely that future implementation of STAMPP/SHAHRP would be supported by colleagues and other stakeholders, policy actions, money and material resources?

   1. If yes, are there any additional considerations that need to be taken into account before your organisation might support STAMPP/SHAHRP? [prompt: identifying providers, ensuring self-sustainability (e.g. teachers training colleagues in school on how to deliver), monitoring outcomes, etc.]

   2. In your organisation/department, are there key individuals who you think would be able to promote the intervention as part of their role?

   3. If no, why not? (prompt: ask them to think about topics such as policy priorities, money and other resources, priority of alcohol education vs. other alcohol actions, autonomy of schools, health and social well-being promotion in general vs. specific alcohol actions)

2. STAMPP has been delivered as part of a research trial (RCT). How do you think, if at all, implementation in the ‘real world’ would differ from delivery as part of a RCT?

3. If shown to be effective, do you think schools will want to deliver an intervention such as STAMPP/SHAHRP? Please explain the reasons for your answer.

4. Is there a role for other types of professionals/services in school alcohol education? How might these different roles work together? Do you think it is possible to develop a ‘whole-school’ approach to alcohol that might include interventions, curriculums, screening, and referral and school policies?