Preventing alcohol misuse in young people: an exploratory cluster randomised controlled trial of the Kids, Adults Together (KAT) programme

Public health context

Alcohol misuse has high personal, social and economic costs, and widens inequalities in health. 2 Harmful drinking has risen steeply in the UK during the last 20 years3 and the annual cost of alcohol misuse in the UK has been estimated at £25B, of which £1B was spent in Wales. 4,5 Misuse of alcohol by young people has raised particular concerns about the number who initiate alcohol consumption at a young age, and the high levels of regular and harmful alcohol use. 6,7 For example, 5% of 11-year-old boys in Wales report drinking alcohol at least once a week; the proportion increases to 35% at 15 years, by which age nearly half report having been drunk at least twice. 8 Alcohol misuse in young people has a range of health and social impacts, including disorderly and violent behaviour, risky sexual behaviour,9 accidental injury, and poor school attendance and achievement. 10,11 In the longer term, early initiation of alcohol consumption increases the risk of alcohol-related problems in later life. 12–15 There is also evidence to suggest that alcohol misuse in young people clusters with other risk behaviours. For instance, the 2011 survey of drug use, smoking and drinking among 11- to 15-year-olds in England found strong associations between past-year drug taking and alcohol use, while people who smoked were roughly twice as likely to have consumed alcohol in the previous 7 days. 16

Interventions to prevent alcohol misuse in children and young people

Schools have long been considered an important setting for delivering health behaviour interventions to young people, including those addressing alcohol misuse. In May 2013, four electronic databases [MEDLINE, Allied and Complementary Medicine (AMED), EMBASE and PsycINFO] were searched from 2010–13 to identify recent evaluations of school-based alcohol misuse programmes for children and young adolescents (see Appendix 1). Search terms for alcohol misuse (alcohol, alcoholic, binge drinking) were combined with a term for school based (school*) and terms for programme (interven*, prevent*, promot*, program*), with results limited to English-language publications. Evaluations identified by the search, including those that were community rather than school based, are presented in Appendix 1 and illustrate the breadth of approaches being investigated and the scarcity of programmes focused on younger children. Rather than focusing exclusively on individual behaviour, school-based interventions provide an opportunity to draw on the socioecological health promotion framework, thereby acknowledging and exploiting the wider influences on health behaviours, such as friendship groups and organisational (school) system influences. 17,18 Furthermore, schools have near-to-complete coverage of the target population, and so intervention reach is potentially very high, and they also have an expanding function as health-promoting institutions. 19–21 Health-promoting schools work within a framework based on the World Health Organization’s Ottawa Charter for Health Promotion22 and promote health through the whole school environment, not just through health education curricula. 23 Health-promoting schools, therefore, strive to integrate their curriculum teaching with the school’s physical, social and policy environment, and the wider school community including parents.

Teaching about alcohol is a component of the personal and social education (PSE) curriculum from Key Stage 2 (8- to 11-year-olds) onwards in Wales,24 and so classroom-based alcohol misuse interventions do not necessarily place additional strain on the curriculum and can help to fulfil key requirements. Since 2003, the All-Wales School Liaison Core Programme, funded by the Welsh Government,25 has provided lessons on alcohol and other drugs to pupils at Key Stages 1–4, delivered by police school liaison officers. While the programme has been well received by pupils and schools26,27 and is currently delivered in approximately 97% of schools in Wales,28 it has not been subjected to a rigorous outcome evaluation. Many studies of similar classroom-based programmes suggest that preventing alcohol misuse requires an approach which may be more complex than that used to teach subjects in which students are required to demonstrate knowledge, rather than to adopt and observe a behavioural principle. 29–34 Such complex interventions might have multiple interacting components and target different levels of the socioecological framework. 35 Effective alcohol-misuse prevention programmes for young people have been found to share characteristics which generally fall into one or more of three main themes outlined below: (1) use of a clear theoretical basis in programme design; (2) interactive delivery style; and (3) community (including family and parental) involvement. 29–34 We summarise each in turn.

The social development model

In terms of aetiological theory, the SDM supports the view that parental involvement should be a key principle of intervention theory. In addition to the general model, the SDM incorporates four key age periods, showing how social influences expand over time beyond the family environment to include school and peer influences and legal sanctions through the preschool period and (US) elementary, middle and high school periods. 73 By high school, peer norms, classroom management, school policies and legal sanctions are influential in addition to the family.

The SDM has predicted alcohol misuse in young people74 and interventions such as the Seattle Social Development Project in the USA and Preparing for the Drug Free Years, which operationalise the model, have achieved reductions in alcohol misuse. 75,76 The SDM proposes that young people learn social behaviour through interactions with others, resulting in the formation of attachments which, if strong, can have a lasting effect on behaviour through supporting the acquisition of skills and influencing norms and values. 77 Attachment to others who offer opportunities for and reward prosocial behaviour is a protective factor against antisocial behaviour. 73,78,79 Thus, involvement of both parents and children in interventions may increase the quality and frequency of parent–child interactions.

Pre-adolescent children who are still highly dependent on their parents will usually have more opportunities for interacting with and forming attachments to their parents than when they enter adolescence and develop a social life outside the home. Social influences expand over time beyond the family environment to include school and peer influences and legal sanctions. 73 Thus, a further implication of the SDM for intervention theory is that programmes involving parents and children will be more likely to succeed if they are based in primary rather than secondary schools. Intervention earlier in the life course is also supported by evidence that programme effectiveness is enhanced by involving young people who have not yet adopted the targeted behaviour(s). 39,53

The SDM also explains why interactive delivery methods and other features of the social influence model have been identified as elements in more effective preventative programmes. According to the SDM, initiation of social interactions depends on people perceiving that there is an opportunity for them to get involved with someone or some activity around them. That is, they see that interactions are relevant to, and intended for, themselves, and that they are competent to take part. Prevention programmes using interactive methods will boost opportunities for prosocial interaction. Perception of opportunities for social interaction is influenced by social structural factors (socioeconomic status, age, sex and race78), suggesting that programmes should include features which are acceptable to all social groups in the target population and which enhance participants’ confidence and skills.

Strength of the evidence base

Gaps remain in the knowledge base for alcohol misuse prevention, particularly in countries outside the USA, where most programme development and evaluation has been conducted. This was borne out in a recent Cochrane Library systematic review of school-based programmes which identified 53 randomised trials, 41 of which were conducted in North America and none in the UK. 50 Promising programmes from the USA may require adaptation and further evaluation when used elsewhere,80,81 to ensure that both aetiological and intervention theories are appropriate for participants with different cultural values and customs. While reviewers are clear that several programme components are needed and that components work together to increase effectiveness, the large number and variety of programmes, the relative rarity of long-term outcome evaluation (e.g. see the range of interventions and follow-up periods in Appendix 1) and systematic adaptation complicate efforts to understand the essence of effective prevention. 50 Lastly, most evidence relates to older age groups because relatively few interventions have been based in primary schools,31,82 despite the SDM indicating that programmes for younger children may be efficacious. Furthermore, most primary school programme evaluations use aggressive behaviour as a precursor to alcohol use as their primary outcome, as their follow-up of participants is not long enough to measure impact on alcohol use in adolescence. 82

An urgent priority in improving the evidence base is to identify and develop effective methods of engaging parents in prevention programmes. As outlined above, while the importance of family-based protective factors for alcohol misuse is widely recognised,29,32,52 knowledge remains limited regarding effective mechanisms for engaging parents in prevention programmes, particularly those that are school based, and differences in programme reach and acceptability between different socioeconomic groups. 83–86

Kids, Adults Together

Anecdotal evidence that some prevention programmes have attracted large numbers of parent participants is not well supported by detailed accounts of percentages or mechanisms of engagement. One such programme was the Parents, Adults, Kids Together (PAKT) programme in Victoria, Australia. 87 PAKT was designed as a family drug education forum prepared by children aged 10–12 years in class and presented to their parents after school. The forum was delivered in addition to existing school drug-education work. A police community safety officer from Wales visited Australia to see PAKT in action and subsequently adapted it for use in south-east Wales, specifically for prevention of alcohol misuse. An advisory group developed a teacher’s pack for the classroom work and Gwent police commissioned a digital versatile disc (DVD) for distribution to participants. The adapted programme was called Kids, Adults Together Family Forum, subsequently shortened to Kids, Adults Together or KAT, and piloted in two primary schools in Gwent, with the police officer presenting the family events.

Kids, Adults Together integrates specially designed classroom activities with a family education evening and a DVD to promote prosocial communication. The programme design addresses key factors affecting parental engagement and is promoted to parents as an opportunity for them to learn about the work their children have been doing in class, rather than as an educational evening about alcohol misuse. To encourage take-up by schools and parents, KAT is of much shorter duration and intensity than other such interventions; the Strengthening Families Programme 10–14 (SFP10–14), for example, requires parents to attend seven weekly 2-hour sessions. 88 A shorter approach is supported by evidence that brief interventions can be as effective as longer interventions in older adolescents. 89

An evaluation at the development stage90 found that KAT had important features previously identified in more effective interventions. Interactive delivery methods are used throughout, and it is based in primary schools, where most children have not yet become regular drinkers. The early timing of programme delivery and the programme content were acceptable to parents, children and school staff. Figure 1 illustrates how KAT incorporates three crucial aspects of the causal pathways to prosocial behaviour contained within the SDM: (1) the creation of opportunities for prosocial interaction between and within families; (2) the strengthening of the necessary skills which parents and young people need to communicate about alcohol-related issues; and (3) the encouraging of parents to reward and reinforce prosocial behaviour and attitudes in relation to alcohol. 91 Crucially, the programme succeeded in involving 40–50 adult family members in the family events at both schools.

FIGURE 1.

How KAT maps on to the SDM. Reproduced with permission from Segrott et al. 1

Study design

An exploratory cluster RCT was used to evaluate KAT. A RCT is the most robust design available to obtain an unbiased estimate of a potential effect size, even with a complex intervention in a ‘real-world’ setting such as schools. 106 As an intervention delivered in the classroom, randomisation at the level of the individual child was not appropriate, and so schools were chosen as the unit of randomisation.

Because it was originally anticipated that programme delivery as part of the trial would need to take place before the outcome of a funding decision on our grant application, we did not request funding to cover recruitment of schools, parents and pupils, or baseline data collection. Recruitment and baseline data collection, therefore, took place before the start of the funded study. While the main trial was funded by the National Institute for Health Research (NIHR), a nested process evaluation was conducted by a PhD student from the Centre for the Development and Evaluation of Complex Interventions for Public Health Improvement (DECIPHer). The process evaluation assessed implementation processes, fidelity and acceptability to families and schools. Qualitative methods, described in Chapter 2, were used to capture the experiences and perspectives of those delivering and receiving the intervention. A summary of key findings from an interim analysis is presented in Chapter 3.

Development

The first pilot study90 theorised the function of each KAT component using Valente’s framework of exposure, knowledge and attitudes, and practice. 107 There was evidence that each component had set off family communication which could mediate later drinking behaviour through parental regulatory practices (Figure 3). Viewing the DVD at home (and, as originally intended, on national television) had the potential to sustain, over a longer period, the immediate impacts of a short intervention. All of the components were thought to be necessary because of their cumulative effect: the classroom preparation led into the family event; the goody bags with the KAT badging and contents were physical reminders of the programme in children’s homes; and the DVD was particularly important for impact in the longer term.

FIGURE 3.

How KAT is expected to achieve its long-term aim. TV, television.

The first pilot study also established the importance of the classwork component in attracting parents to the family events. Children’s eagerness to show parents their work and parents’ status at the family events as proud supporters of their children were powerful factors in securing parental participation in the programme and were likely to be constant across different school contexts.

A further pilot study in 2011 focused on refining the materials provided to schools. An independent health education consultant employed by the research team modified the teachers’ handbook which had been developed for the earlier pilot study, clarifying the aims, learning outcomes and links to the curriculum for each lesson/activity and providing supporting information such as a timetable for the family event and a list of suggested questions for a quiz. As part of this work, the programme was implemented in two schools outside the trial area (one school which had previously implemented the programme, and one with no prior involvement) and feedback from staff, pupils and parents drawn upon. The number of suggestions for classroom activities in the teachers’ handbook was reduced and teachers were encouraged to choose their own way of achieving programme aims. That is, they had freedom (within limits) to alter the form of the activities, as long these still performed the same function. 108 The handbook included contact details of alcohol support services, and schools were encouraged to request one of them to attend the family event to facilitate contact between agencies and families needing support.

Implementation

The intervention was administered at cluster level (schools) and consisted of the KAT programme in addition to any existing alcohol-related lessons/school activities.

Kids, Adults Together requires approximately 1 week’s total classroom contact time (or around 20 hours). Classroom preparation for children was planned to take place over at least 1 week, with flexibility for schools to take longer according to, for example, the timing of the family event and the needs of the class. The KAT family event lasts about 1 hour and the contents of the ‘goody bag’ may be used by children and their families over an indefinite period.

Following baseline data collection in all schools, the health education consultant asked to visit all intervention schools to train teachers whose classes would be involved and to provide information about the programme to any other relevant members of staff. Staff training covered the key points in the KAT handbook and staff were also given a Microsoft PowerPoint (Microsoft Corporation, Redmond, WA, USA) presentation for use at the family event; appropriate information from websites, downloaded and photocopied; and Tacade’s Keys to Alcohol for Children Aged 7 to 11 Years Old resource. 109 Teachers were asked to ensure that support would be available for children who might be distressed by the topic (although this should usually have been in place to back up the usual alcohol-related curriculum content).

Evidence gathered during the pilot110 suggested that school staff would modify KAT in order to achieve programme aims and objectives in ways they thought would be more appropriate in the context of their own schools. The process evaluation included observation of classroom work so that analysis of any such adaptations could inform future programme development and evaluation.

The four schools randomised to the control group did not receive KAT, but continued with their usual activities, including any classroom work/school activities on alcohol. Process evaluation interviews with staff in control schools identified normal practice in relation to both substance education and structures for involving parents in school life.

Inclusion/exclusion criteria

Primary schools which included Years 5 and 6 and taught through the medium of English were eligible to take part in the research. Welsh-medium schools and those with infant classes only were excluded. In each participating school, all children in Year 5 and Year 6 classes were eligible. Head teachers were encouraged to involve as many classes as possible, but were allowed to select which classes should take part, because we were interested in understanding how schools’ preferences would shape the likely cluster sizes as part of a future effectiveness trial. Table 2 gives details of numbers of classes taking part in intervention and control schools. A number of the schools had mixed year-group classes, and we allowed those with Year 4/5 classes to participate in the trial.

Where parents/carers or children refused consent for children’s participation, these children did not participate in the trial. Children who were absent at both baseline and follow-up data collections and parents who were unable to communicate in English or who did not return their contact details did not participate in the trial. In intervention schools, all children in participating classes received KAT whether or not they participated in the trial. KAT programme activities were integrated into their normal classroom work and their parents/carers were invited to attend the KAT family events. Head teachers in all participating schools, and parents and teachers of children in the relevant classes at intervention schools, were invited to take part in process evaluation interviews. Children in intervention schools also took part in focus groups.

Effect sizes detected in previous studies

We undertook a search for previous studies which had used these selected outcome measures, to identify what size of effect had been detected in evaluations of interventions comparable with KAT. We present the results of this search in Chapter 3 [see Effect sizes detected in previous studies (secondary outcomes)].

Assessment of the reliability of questionnaire measures

Assessment of reliability was carried out using Cronbach’s alpha for internal consistency and factor analysis for unidimensionality. Summed scores were created for each outcome scale. With previously validated scales, the outcome was used as directed in the manuals or scale references – either categorical (using validated cut-offs) or used as a continuous score. Where no guidance was given, the continuous data were used in the primary analysis and, if necessary, the categorical outcome was examined as a secondary analysis.

Analysis of future primary outcomes

All analyses were on an intention-to-treat (ITT) basis using all randomised participants in the groups to which they were randomised, regardless of the intervention received. However, missing data were not imputed, and so all analyses were carried out on complete cases. The primary outcomes were alcohol consumption and levels of harmful drinking and drunkenness at follow-up. Alcohol outcomes were analysed using two-level generalised logistic models. Responses from pupils were nested within schools fitted using models adjusting for baseline data where appropriate. The distribution of frequency of drinking and drunkenness in the last month was examined and it was determined that these ordinal categorical data were most appropriately collapsed into binary outcomes due to sparse data in higher categories. Covariates included in the models were those that were used to balance the randomisation (school size and FSM entitlement). While we acknowledge that a household-level cluster is present (siblings within a school), the number of these per school was very small and was not incorporated. As well as any differences between trial arms, estimates of ICCs at school level were reported.

Analysis of future secondary outcomes

Questionnaire outcome score data from pupils were analysed using two-level generalised linear models, with responses from pupils nested within schools fitted using models adjusting for baseline data. Covariates included in the model include those that were balanced on at randomisation (school size and FSM entitlement). As well as any differences between trial arms, estimates of ICCs and an indication of effect sizes are reported.

Outcome measures from parents were also collected but only a small number of parents returned data and, therefore, only descriptive analyses of these are provided.

No formal subgroup analyses were carried out; however, as there were differences in the ratio of Year 5 and 6 pupils between arms, the effects of age were investigated further, as it was important to identify any patterns in the data relating to these groups which might be present in a future effectiveness trial.

Methods

This study used classroom observations, semistructured interviews and focus groups to develop the theoretical framework and explore implementation fidelity and acceptability of KAT. Semistructured schedules allowed the researcher to probe and explore key themes and issues while allowing participants to discuss issues, experiences and opinions about KAT that had not been anticipated by the interviewer. Observation, and interview and focus-group schedules are included in Appendix 8.

Training and roles and responsibilities

Training was offered to staff in intervention schools but some teachers in school 6 did not attend training sessions, mostly due to pressure of other work or because it was impossible for the school to bring together all relevant staff at one time. Consequently, the Year 6 teacher and Year 5 deputy head did not appear to be familiar with the contents of the handbook or to be aware of other information they would have received through training. In addition, although school staff arranged the date and venue for the family event and contacted parents, they did not get the hall ready or take charge of the activities. Teachers in the other two schools were on the whole satisfied that the KAT handbook provided ‘a good basis to work from’ (Year 6 teacher, head, school 4), but it was apparent that the success of the intervention was dependent on senior manager buy-in and support.

Classwork component

The programme handbook identified five key elements which teachers were asked to deliver as part of the classroom work:

• establish pupil knowledge

• emphasise healthy use of alcohol (not negatives)

• develop responsible attitudes to alcohol use

• improve communication skills – group/pair work

• produce work for the family event including:

  • posters to advertise event

  • personal invitations

  • take work home to finish.

Classroom observation suggested that the majority of these elements were implemented with high fidelity in all three schools. Observation and interviews showed teachers followed the handbook closely, including the suggestions for assessing children’s knowledge. Interviews with teachers and focus groups with pupils demonstrated a good understanding of the ’not too much, not too soon’ healthy approach to alcohol, with teachers highlighting that in order for children to develop enough confidence to raise the topic at home, it was important for them to realise that alcohol was not ’all bad’ and that parents who drank alcohol were not doing wrong. However, it was recognised by some teachers and parents that in families where there was significant alcohol misuse, the balanced message might not reflect children’s experiences. In these cases, it was thought that children would benefit from more intensive support outside the family. If such children do talk to school staff or other adults, then KAT could maintain its reach to high-risk families, with children forming an attachment to a teacher or other supportive adult.

Group/pair work had high levels of fidelity and was something that pupils responded positively to, with clear examples of how interactive teaching and learning could be used to good effect in encouraging responsible attitudes to alcohol use, for example the ‘drinkers and thinkers’ role play rehearsal at school 4:

The drinkers were calling the thinkers ‘un-cool’, ‘boring’, ‘chicken’ etc., for not drinking. The thinkers were saying things like, ‘they had an exam tomorrow, they didn’t want smelly breath’, ‘didn’t want to get into trouble with police’, or ‘be ill’. The drinkers were being sick and falling over.

Observation 4, class 1

Group work was also thought to play a significant role in helping children to talk about alcohol at home because they had ‘almost practised saying it’ (Year 4/5 teacher, school 3).

Finally, while all classes prepared work for display or performance at the family event, including posters (though it is not clear that these were used to advertise the event), personal invitations were produced at only one school and there was no evidence from any school that children, parents or teachers thought that they were particularly important. Homework activities also had poor implementation, with school policies or existing practices resulting in low fidelity.

Kids, Adults Together family events

Key elements of KAT family events were identified from the manual as:

• a presentation by pupils

• a non-judgemental approach

• an emphasis on healthy use of alcohol, not negatives

• an aim for all pupils to take part in the event.

The handbook suggested how the event could be organised, with details of a suggested format and instructions on how to run two activities aimed at encouraging communication about alcohol: a quiz/treasure hunt and an activity called ‘Agree or Disagree’. This information, together with answers to the quiz, was repeated in a PowerPoint presentation which accompanied the handbook. The three intervention school events included both activities suggested in the handbook, alongside presentations by the children. Table 8 lists the presentations children made at each school.

Observation at all three schools found that they achieved a non-judgemental approach, and, overall, parents who participated in research interviews agreed with this. Interviews with teachers also suggest that they tried to ensure that the approach was ’light-hearted’ and humorous. There was evidence from all three schools that the family events promoted a healthy approach to alcohol use, with only a minority of parents indicating that their children were more focused on the dangers of alcohol than healthy use. There was less success in involving all children in the event, with two of the schools not appearing to have followed the guidance in the handbook.

Parents/carers

Initially, there were a very small number of objections to KAT from parents/carers (schools 3 and 4) and these were withdrawn after discussions with school staff. In all three schools, parents/carers were positive about the topic of alcohol being addressed at primary school, with parents generally seeing it as the ’perfect age I think, 10. I wish I had it when I was in school’ (parent interview 6).

All parents interviewed across all schools said they had enjoyed the family event, and this was confirmed by children’s focus groups who reported positive feedback from their family and observations. Teachers at all three schools also reported having received positive feedback from parents about the event, directly and/or through the children.

In the school which withdrew from the study after baseline (school 8), it is not clear how big a part KAT’s acceptability to predominantly Muslim parents played in the circumstances leading to the school’s withdrawal. The teacher of one class made it clear that he did not think KAT would benefit either the school or the children in his class and had expressed concern from the outset about delivering KAT to children from mostly Muslim families. Because the school decided to withdraw completely from the study, we were not able to conduct interviews with parents/carers in this school in order to ascertain to what extent the programme was acceptable to them.

Children

Observation suggested that the vast majority of pupils enjoyed KAT and were fully engaged in it. There was no evidence that large numbers of children had been bored or alienated by the project. This was supported by focus groups, with the participants stating that they had enjoyed the classwork and the family events; parents and teachers at all three schools also reported that the children had enjoyed KAT.

A few small issues were noted which slightly impaired the overall acceptability of the programme for children. Although KAT is intended for children in Years 5 and 6, Year 4 children were involved because they were in mixed classes, and they found the group work and writing tasks challenging. Observation records of the family event at school 6 where the school staff took no responsibility for organising or introducing activities showed some levels of boredom and restlessness among pupils. However, three parents from school 6 said that their own children had enjoyed the events, and general feedback from teachers and children was positive.

School staff

The vast majority of teachers thought that the intervention was age-appropriate, the ’perfect age’ to deal with this topic. Teachers were also pleased with the way KAT fitted into the curriculum and there was no evidence that teachers lacked the confidence or skills to deliver the classroom work. The Year 6 teacher at school 3 said that children had disclosed sensitive issues in class and it was not a problem for her: ’At this age they need to be talking about stuff like that’. Staff in all schools liked the way KAT facilitated parent–school contact (although there was some disquiet among teachers at the level of organisation at school 6), with some head teachers keen to use the same format for other topics.

Classwork

Table 9 shows that the majority (70.9%) of pupils who completed follow-up questionnaires reported covering KAT classroom work in full, and most of the remaining pupils (22.8%) had done part of it. These patterns held true across all three intervention schools which completed KAT, although there was significant variation in the proportion of pupils who completed KAT in full or in part. A greater proportion of pupils in school 4 had completed all the work (86%) than in schools 3 and 6 (64.7% and 63%, respectively).

Family event

Table 10 shows the participation rates at the family event reported by pupils. An average of 59% of pupils reported attending the family events and 50% with their parents/carers. For the individual schools, the proportions of pupils whose families were represented by at least one adult member were 45.1%, 46.6% and 65.7%. For attendance by pupils themselves, the figures were 47.1%, 58.9% and 79.4%.

Reach across socioeconomic groups and localities

Table 11 provides the KAT attendance rates categorised by Family Affluence Scale score (low, medium and high). When Family Affluence Scale scores are taken into account, pupils with high scores were more likely to have an adult member of their family present at the family event than pupils who had low/medium scores (56.6% vs. 41.4%). However, there was no such pattern in relation to whether or not pupils themselves attended (59% vs. 60.3%).

Table 12 displays rates of participation in the family event for pupils and parents/carers for each school, and includes the FSM rate for each school. It shows clearly that the school with the highest FSM score (37.3.%) also had the highest proportion of families represented by at least one adult (65.7%). The school with the lowest FSM had the lowest proportion of families represented, and the school with the middle-ranking representation rates also had the middle-ranking FSM score.

At schools 3 and 4, staff reported that attendance by parents was higher than usual, with one teacher (school 3) ‘shocked’ when he or she saw the queue waiting to get in. At school 6, parental attendance was ‘on a par’ with the turnout for class assemblies. This appeared to be a positive assessment because the Year 5 deputy head said that the school had been making a special effort to get parents involved with their children in events at the school and that attendance at the KAT event had followed the recent trend.

Analysis of the process evaluation data suggested that there were three key reasons why parents/carers attended the family events: to support their children, to support the school and to satisfy children who put pressure on them to go. Also mentioned frequently by parents and children was that parents wanted to see their children’s work; and some had a general rule of always attending school events (school 4, parent 5, 6 and 7; school 6, parent 3). At school 3, the free tea and coffee may also have been influential (Year 4 focus group, Year 6 teacher, school 3).

Children whose parents do not attend the family event

Organising the event for family groups could be difficult for children whose parents do not attend. It was possible to count the number of children who had gone to the family event unaccompanied by members of their family using two questions from the children’s follow-up questionnaire: ‘Did you go to the KAT fun event at your school?’ and ‘Did any of your family go to the KAT fun event at your school?’. Table 13 shows that out of 94 children who said they had attended, 18 (19%) had gone without a member of the family.

Schools 3 and 6 held their family events immediately following on from the school day, which might have led to more children staying on for the event without an adult. School 4 had started their event later so that children would have returned home after school and come back to school later in the evening. Responses for each school from the 18 unaccompanied children are shown in Table 14.

Although proportions of unaccompanied children are slightly higher in the schools which ran the events straight after the school day, there is not much difference between the three schools. It should also be noted that children may have attended the events without having completed questionnaires, and so total rates of attendance by unaccompanied children could be higher or lower than described above.

Classwork

In each school, both parents and children reported that children told their parents about what they were doing in class. Topics included the kinds of work they were doing (posters, etc.); the physical effects of alcohol; dangers of alcohol; safe drinking; parental alcohol use; the nature of the KAT project; reasons why people drink; and units of alcohol. There were a few reports that children had not talked about it (focus-group participants A and C, school 4; parent 5, school 4; parent 1, school 6) and two children withheld information so that their parents would have a surprise at the family event (focus-group participant school 3, parent 6, school 6).

Homework

The KAT programme manual encourages teachers to ensure that there are opportunities for pupils to take some of the classroom work home to finish, ‘to generate interest amongst family members’. No homework was set in school 6, but at school 4 and school 3 two teachers set homework with the specific aim of stimulating conversation between parents and children (second Year 6 teacher, school 4; Year 6 teacher, school 3). In both schools, there were more reports that parents helped with homework than not. Topics covered included support groups (parent 1, school 4), physical harm (parent 2 Year 4, school 3) and consequences of misuse (parent 3, school 4). This teacher thought that girls were more willing than boys to take work home and talk to their parents.

Family event

Parents at school 4 and school 3 enjoyed talking to other parents at the events (parent 5 and parent 7, school 4; parent 1, school 3). Most parents maintained that they were ’quite open’ about alcohol, that they talked about it at home anyway, and that attending the family event had not made any difference to that. However, some went on to say things which suggested that there had actually been at least small changes: one said that their child would ’chip in’ more when alcohol was discussed (parent 1, school 4); another parent had been shocked at some of the information and had become more aware of the implications of her own drinking behaviour (parent 7, school 4). Parent 4 (school 4) seemed to reflect that the KAT project as a whole had led to more conversation, and parent 5 (school 4) said that while there possibly had been more conversation after the family event, this was just repeating what had been said at home before.

A significant number of children at each school said that they had discussed alcohol with their families after the event, though it was not clear (except for boy D, Year 6, school 3) whether or not families had discussed it more than usual. At all schools, some parents and children had told other members of their family and friends about the KAT activities and what they had learnt. Teachers at all of the intervention schools said that KAT provided an opening for discussions, or more serious discussions, about alcohol, and drew on evidence of children having conversations not only with their parents but also with friends, classmates, siblings and other relatives, and suggested that during the project, other children became informed and informative participants in a wide-ranging dialogue about alcohol.

Schools

All 39 eligible schools were contacted by letter, and invited to participate in the study, and telephone calls were made to all schools until sufficient schools were recruited. Meetings were held with nine schools which expressed an interest in the study. One head teacher was willing for the school to participate but only if it was not in the intervention group because there was no capacity within the school timetable to deliver KAT. The school was not, therefore, eligible to participate in the trial. However, they offered to assist with the piloting of the study questionnaires. The other eight schools with whom the principal investigator had meetings all went on to participate in the trial (a participation rate of 20.5%). Of the 31 eligible schools which did not participate at this point, 19 did not respond to letters or telephone calls, six indicated to the research team that they were not interested, four were too busy and two schools declined because they were already delivering projects similar to KAT.

Table 15 shows the number of pupils and percentages entitled to FSM in each of the recruited schools, alongside the average, median and range for all 39 eligible schools. It can be seen that the FSM rates in the recruited schools ranged from 1% to 37.2%. Six schools were above the county median, and three below.

Figure 5 summarises the recruitment process. One intervention school (school A) withdrew from the study before baseline data collection. The school’s main reason for withdrawing from the study was parents’ concern about the research topic, and appeared to be linked to the fact that, due to an administrative error, main information letters and reminder letters were switched in order. However, given the fact that the reminder letter explained how parents could access the original letter if for some reason they had not received a copy (and a link to the information online was also provided), there may also have been other reasons. We were not able to obtain more detailed information from the school, though we requested this on several occasions.

FIGURE 5.

Recruitment of schools. White, recruitment activities; pale green, intervention group; dark green, control group.

Following the withdrawal of this school, we wrote again in March 2012 to all eligible schools which were not already participating in the trial (n = 29) inviting them again to participate in the trial as an intervention school, but indicating that to take part they would need to be able to deliver the programme in the summer term of 2012. One school responded to this letter and agreed to participate in the trial. This school withdrew from the trial after baseline data collection had taken place, and shortly after programme delivery had commenced. Although the required timing and workload associated with the programme had been clearly explained to the school before they agreed to participate (see Appendix 9), their main reason for withdrawing from the study concerned these issues, and was mostly related to programme implementation and the need for multiple members of the study and programme support team to contact the school. However, it would appear that other issues, including one of the teacher’s concerns about the appropriateness of KAT for Muslim families, also played a part.

In total, therefore, we recruited nine schools from the 39 eligible (participation rate of 23.5%). We retained all four control group schools for the life of the trial, and three of the intervention schools, with two withdrawing as described above.

Pupils

Determining research trial participation rates

We originally aimed to recruit 50% of eligible children into the research trial order to achieve the sample size target (though this was based on using ‘opt-in’ parental consent for children’s participation, whereas we eventually used ‘opt-out’ parental consent). Table 16 shows the proportion of eligible pupils (at baseline) who provided questionnaire data. At baseline, this was 74% (intervention group) and 81% (control group). At follow-up, these figures were 68% and 74% respectively.

TABLE 16 - Summary of numbers and percentages of children completing KAT questionnaires, using different denominators

N/A, not applicable.

Time point	Intervention				Control
	n	% eligible, four schools	% eligible, three schools	% present	n	% eligible, four schools	% present
Baseline	172	74	N/A	78	152	81	88
Follow-up	158	68	85	92	138	74	77

Table 16 also shows response rates using two other denominators. First, the proportion of children present in class on the day when questionnaires were completed, which in every case was higher than the proportion of eligible children (which included those absent from school). Second, for the intervention group schools, we provide the response rate at follow-up based only on the three schools which were still in the study (we could not collect any follow-up data in the school which withdrew). Using this calculation, the proportion of children eligible at baseline who went on to complete follow-up questionnaires was 85% (compared with 68% when the school which withdrew is included in this figure).

All eight schools at which data collections were conducted were asked to confirm numbers of pupils on the registers of classes involved, in order to ensure that participant information sheets were distributed to all parents/carers and pupils, and to allow preparation of sufficient copies of the questionnaire. We also asked schools for information on numbers of parental refusals (to ensure that no data were collected from the relevant pupils), and pupils present on the day we visited. Some of the information supplied by the schools appeared to be inaccurate; for example, the number of questionnaires completed was greater than the number of pupils that schools said were on the register. Where discrepancies were identified, we contacted schools to check the information.

Figure 6 displays the participation rates in the intervention and control groups, and reasons for non-completion of questionnaires. Absence from school or the classroom were the most likely reasons for non-completion, and rates of parental refusal for participation were very low. In one intervention school, a large number of children came from families who did not speak English as a first language, and were from Muslim families. When we visited the school to collect baseline data, some of the children were unsure whether or not their parents were happy for them to take part (though their parents had not contacted the school to refuse permission). To make absolutely sure that we did not collect any data from pupils whose parents were not happy for them to participate, we advised 15 children at this school not to complete a questionnaire. There were no other possible/definite parental refusals in the intervention schools, and only two in the control group schools, indicating that the vast majority of parents were happy for their child(ren) to participate in the trial and intervention.

FIGURE 6.

Kids, Adults Together participation rates: percentage of children eligible at baseline who completed questionnaires. a, All of these pupils were from one school where many pupils and their families did not speak English as a first language, and many came from Muslim homes. When we spoke to the children at the baseline visit, some were uncertain whether or not their parents would be happy for them to take part, and so we advised that these children should not complete questionnaires; b, uncertainty over the exact number. Reproduced with permission from Segrott et al. 1

The pupils shown in as ‘Absent from classroom’ are those we cannot reasonably account for. This is the difference between the total number reportedly on the register, and the sum of those who completed questionnaires; absentees from school; and those who did not assent or whose parents withheld consent. In the intervention group, this number was 21 pupils (from 231 who were eligible), while in the control group 19 pupils (from 187 eligible) fell into this category. Figure 7 shows questionnaire completion rates as a percentage of those present in school on the day of data collection. Participation rates for the intervention group were 78% (baseline) and 92% (follow-up). For the control group, the figures were 88% and 77% respectively. The intervention school which withdrew after baseline (school 8) included a large number of pupils from ethnic minority groups, many belonging to Muslim families, and recent immigrants who did not speak or understand English very well. Letters and information sheets for parents and children were translated but because of the late recruitment of the school and the short time remaining for programme delivery, we did not adapt the information-giving process as completely as we would have liked, and school staff were unable to accept our proposals for providing extra information and explanations for participants. Consequently, when we spoke to the children at the data collection visit, we said that those who were at all uncertain about their parents’ wishes should not take part, although their parents had not actually contacted the school to refuse consent. This partly accounts for the lower numbers participating in this school and the impact on the overall recruitment rate.

FIGURE 7.

Kids, Adults Together participation rates: percentage of children present at baseline and follow-up who completed questionnaires. a, All of these pupils were from one school where many pupils and their families did not speak English as a first language, and many came from Muslim homes. When we spoke to the children at the baseline visit, some were uncertain whether or not their parents would be happy for them to take part, and so we advised that these children should not complete questionnaires; b, uncertainty over the exact number.

Figure 7 shows that 21 pupils in the intervention group are recorded as ‘Absent from classroom’ at baseline. If all of these had provided data on return visits, the percentage of those present who completed questionnaires at baseline would have been a little higher than the response rate of 92% at follow-up. In the control group, returning to recruit the 19 (baseline) and 25 (follow-up) pupils recorded as possibly absent from the classroom could also have increased response rates.

Figure 8 gives details of participation for only the three intervention schools which completed the programme, showing that 85% (baseline) and 85% (follow-up) completed questionnaires. In an effectiveness trial using an ITT analysis, the participants in school 8 would have been followed up.

FIGURE 8.

Kids, Adults Together participation rates: percentage of children eligible at baseline in intervention schools who completed questionnaires, excluding the school which withdrew after baseline. a, Uncertainty over the exact number.

Table 17 shows the proportion of pupil participants with low, medium and high Family Affluence Scale scores in each school, and also shows each school’s FSM rate. Participants with high Family Affluence Scale scores formed the majority of participants in each school. It is not possible to determine to what extent the distribution of Family Affluence Scale scores among the recruited sample reflects the total eligible population of classes.

TABLE 17 - Numbers and percentages of children who provided data for Family Affluence Scale, in each intervention school

Source: adapted with permission from Segrott et al. 1

School	Family Affluence Scale				Total
	Low	Medium	High	Missing
3 (FSM = 37.2%), n (%)	8 (18.6)	9 (20.9)	16 (37.2)	10 (23.3)	43 (100)
4 (FSM = 2.3%), n (%)	0 (0)	12 (21.1)	43 (75.4)	2 (3.5)	57 (100)
6 (FSM = 27.9%), n (%)	5 (6.3)	31 (38.8)	38 (47.5)	6 (7.5)	80 (100)
8 (FSM = 21.6%), n (%)	0 (0)	6 (42.9)	7 (50.0)	1 (7.1)	14 (100)
Total, N (%)	13 (6.7)	58 (29.9)	104 (53.6)	19 (9.8)	194 (100)

Retention of children

Three hundred and twenty-four pupils completed the baseline questionnaire (152 control and 172 intervention), while 296 pupils completed the follow-up questionnaire (138 control and 158 intervention). An additional 39 pupils (22 control and 17 intervention) who did not complete baseline questionnaires completed follow-up questionnaires. The study retention rate in the control arm is, therefore, (138 – 22)/152 or 76.3% and that in the intervention arm is (158 – 17)/172 or 82.0%, an overall retention rate of 79.9%. Figure 9 displays these figures in the form of a flow chart.

FIGURE 9.

Pupil flow chart.

Parents/carers

When schools were recruited (October 2011 to February 2012 and April 2012), we invited all parents/carers of children in participating classes to take part in telephone interviews (to collect outcome measure data at 6-month follow-up), and 52 volunteered. Interviews commenced in July 2012, and in an attempt to increase the number of participants, each participant was asked at the end of the interview if any other adult in the household would like to take part. One additional participant was recruited using this method. All interviews were completed by the end of September 2012 but interviewers continued their attempts to contact the remaining volunteers until the end of November 2012. Using an estimate that each eligible pupil would have only one parent/caregiver, the number of eligible parents was 418, of whom 12% volunteered to take part.

Figure 10 shows that numbers of parents from intervention schools who expressed interest in the research were much larger than in control schools. However, the larger drop-out rate from the intervention arm meant that approximately equal proportions of parents from each group provided data. Demographic data supplied by the 41 respondents were examined in case they were all alike in one or more respects but this was not the case (see Appendix 10). As the data set was so small, no further analyses were conducted.

FIGURE 10.

Parent recruitment for telephone interviews.

Secondary outcomes validation

There were five secondary outcomes in the pupils’ questionnaire: the Family Activity Scale, KIDSCREEN-52, TCPPAS, PCCS, and the FCS. All outcomes were modified and/or adapted for KAT, and therefore needed validation. For the outcomes measured at baseline and follow-up (Family Activity Scale, KIDSCREEN-52, TCPPAS), validation was carried out using baseline data, while the outcomes used only at follow-up (PCCS and FCS) were validated using the follow-up data set.

Table 27 shows that the reliability of the secondary outcomes was demonstrated using Cronbach’s alpha values. All were greater than 0.7, indicating high internal consistency of items. Factor analysis confirmed that each scale consisted of one factor, indicating that using a summed score for each is a valid interpretation of the data. Summed responses were investigated for normality and demonstrated negative skew. Positively skewed data can be log-transformed so that they become normally distributed before analysis. However, negatively skewed data are difficult to normalise using transformation. We have, therefore, left the data untransformed, as analysis of variance (ANOVA) and analysis of covariance (ANCOVA) are relatively robust to non-normality and the extension to two-level linear models should not invalidate the interpretation of the likely intervention effects. However, caution should be exercised when examining these data and reanalysing these scales as categorical outcomes rather than continuous scores may be preferable for investigating these data in a larger definitive trial.

Rates of missing data

Of the 358 pupils who provided their demographic variables, three failed to provide their year group. Among the other 355 pupils, 28 were in Year 4, 179 in Year 5, and 148 in Year 6. Table 28 shows how many pupils in each year provided data for secondary outcome measures.

Excluding the FCS, rates of missing data ranged from 14.1% to 23.5%. Rates of missing data were, thus, higher for the secondary outcomes than for the alcohol-related questions discussed above. For the measures used at both baseline and follow-up (the Family Activity Scale and KIDSCREEN-52 subscale), rates of missing data were higher at follow-up than at baseline, except for KIDSCREEN-52 in Year 6, where there was a small decrease in the rate from 14.9% at baseline to 14.2% at follow-up. For the FCS, there was a much lower rate of missing data in Year 6, 17%, compared with 41.9% in Year 5 and 42.9% in Year 4. One factor contributing to the highest rates of missing data found for the FCS (42.9% for Year 4) is that on 28 questionnaires, one item was accidentally omitted from the scale; otherwise, there was no discernible pattern to the unanswered items. Another possible reason is that the FCS was counted as missing if any one item out of 10 was not answered because the score for this measure is summed from the individual responses. However, we cannot rule out the possibility that the questions may have been unsuitable for children in all year groups. For the other scales, comparisons of rates across year groups do not support an age-related explanation for any differences, and the very small numbers of Year 4 pupils limit the scope for interpretation.

Table 29 shows the scores for the secondary outcomes. At follow-up, the average scores on the Family Activity Scale and PCCS were marginally higher for the intervention group. There was no difference between the average scores for the two groups on KIDSCREEN-52. The average score for the control group was marginally higher for the control group on the FCS. None of these effects reaches levels of conventional statistical significance and confidence intervals are wide. Of the four measures, two (the FCS and the PCCS) were used only at follow-up, while the other two were used at both baseline and follow-up.

Determining the primary outcome for an effectiveness trial

Our data suggest some problems with the reliability of pupils’ recall of lifetime drinking. These may have related to the young age of the children, or heightened awareness of alcohol issues as a result of questionnaire completion/intervention receipt, though we do not have evidence to confirm or refute this. If follow-up took place at 2 years past baseline in a future trial, the participants would be 2 years older than they were in this study. Age-related issues around comprehension might, therefore, be less of a problem. However, basing the primary outcome on whether or not participants have had a drink in the last 30 days would provide a good indicator of recent drinking behaviour, and may avoid the problems described above concerning recall over a child’s entire life. Our sample size calculations for an effectiveness trial are, therefore, based on detecting differences in past-month drinking (as a binary outcome) between intervention and control. However, should any future trial be planned it would also aim to measure other aspects of alcohol behaviour (lifetime drinking, and aspects of drunkenness) as key secondary outcomes.

Rates of drinking behaviours in the Health Behaviour in School-aged Children survey and randomised controlled trial of the Strengthening Families Programme 10–14

To estimate the prevalence of past-month drinking rates, we examined the 2009 HBSC survey, which collected data on this outcome. We also examined baseline data from an ongoing trial of another alcohol misuse prevention programme (SFP10–14). That trial does not collect data on past-month drinking at baseline, but we were able to use the other alcohol-related data to provide some additional context on drinking behaviours among 11- to 13-year-olds (the age which participants would be at main follow-up in an effectiveness trial of KAT). The SFP10–14 trial is using past-month drinking as one of its twin primary outcomes, and these data will be available in 2014 (and, therefore, could aid further work on developing a sample size for any future effectiveness trial of KAT). 8

Table 30 shows data from the HBSC for participants born in 1997, 1998 and 1999 (i.e. 11- to 13-year-olds at the time of data collection) relating to the number of times that they consumed alcohol during the last 30 days.

The data in Table 30 show an overall prevalence rate for past-month drinking of 24.5% (773/3151 = 24.5%), and the majority of those who have consumed alcohol have done so on only a few occasions.

Table 31 shows data from the baseline questionnaire used in the SFP10–14 trial. It indicates that for the 13-year-olds in the sample, around 21% drink alcohol monthly or more frequently. This is slightly lower than, but broadly comparable to, the HBSC data.

Table 32 below displays data for the question on whether or not participants in the SFP10–14 trial had ever had a drink of alcohol. At age 11, only 15.2% of participants report ever having had a drink of alcohol, but this increases to 52% by age 13.

Sample size calculation

Our sample size calculations use the prevalence rate of 24.5% for past-month drinking, derived from the HBSC data above. Tables 33 and 34 present sample size calculations based on KAT reducing past-month drinking by 2.5%, 5% and 7.5% respectively, using the ICC of 0.021 from the HBSC data for 11- to 13-year-olds’ drinking in the last 30 days. We are not able to specify with precision what effect KAT would achieve, and have therefore included a range of sample size calculations (based on 2.5%, 5% and 7.5% reductions in past-month drinking). However, the brief duration of the KAT intervention and the evidence from this study, including the lack of evidence of effect on hypothesised mediators, suggest that any effect would be small, and a 2.5% reduction may be the most appropriate of the three we present below.

The total sample size estimated from Table 33 must then be inflated for clustering. It should also be inflated for varying cluster size and pupil drop out.

Table 34 shows that for an effectiveness trial designed to detect a 2.5% difference between the two trial arms, the necessary sample size would be 9849 participants, and 263 schools. To detect a difference of 5% and 7.5% would require 127 and 54 schools, respectively.

The estimated sample sizes with 90% power for the same range of differences in drinking prevalence are given in Tables 35 and 36.

The total sample size estimated from Table 35 must then be inflated for clustering (see Table 36). It should also be inflated for varying cluster size and pupil drop-out.

Table 36 indicates that based on 90% power, a total of 351 schools would be needed to detect a 2.5% reduction in drinking in the intervention group. For 5%, the figure would 169 schools, and for 7.5% a total of 72 schools would need to be recruited.

Table 37 shows the total number of eligible schools (English-medium primaries) in each of the 22 local authorities in Wales, and the number of schools which could be recruited from each of these areas if 23% agreed to participate an effectiveness trial (the same proportion as in the current study). Using these calculations, it can be seen that if all eligible schools across Wales were invited to participate in the trial using the methods employed in the current study, 198 schools could be recruited, assuming that 23% agreed to participate. Presuming that a future effectiveness trial would use 80% power and would seek to detect a reduction of 2.5% in past-month drinking, this figure falls well short of the 263 schools which would be needed.

Key cost domains

To enable an economic evaluation to be conducted as part of any future effectiveness trial, we sought to map, in as much detail as possible, the key cost domains.

The costs associated with running KAT can be mapped on to three levels, namely families, schools, and external organisations. We deal with each in turn. For families, the main costs incurred relate to the opportunity cost of attending the family event and the expenses incurred through travelling to and from school. Some parents/carers might also incur childcare costs for young siblings of children involved in KAT, though the family events are open to all family members.

Schools’ costs fall into three clear categories. First, there is the cost of staff time devoted to the programme. This comprises teaching staff, teaching assistants, and non-teaching staff such as caretakers. Second, there is a cost attached to the production of materials for the family event. These are mainly paper and stationery – there is no requirement for schools to buy equipment as part of their classroom preparation. Third, by delivering KAT, schools incur an opportunity cost – i.e. the time spent on the programme by staff and pupils could have been utilised for other curricular or non-curricular activities.

During this exploratory trial we were also able to map out which external organisations were involved in programme delivery, the nature and extent of their input, and how this might evolve as part of the systems and structures established for any future effectiveness trial. In the current study, the following agencies supported the implementation of KAT:

• A Healthy Schools Scheme officer, who assisted with recruitment of schools, and also attended some of the training events for school staff and the family events.

• An educational consultant who provided training and support for school staff and also assisted with the running of the family events. The cost of her employment was covered by a grant from the Welsh Government.

• A local drugs agency which provided an information stand at all of the KAT events.

• A local secondary school which loaned its ‘smoothie bike’ free of charge for use at the family event in all of the intervention schools.

A number of non-staff costs were also covered by external agencies (principally, by a grant from the Welsh Government). These were the updating and printing of the programme manual and the purchase of additional educational resources; the cost of providing refreshments at all three schools which ran family events; the purchase of additional display boards (which were used in multiple schools); and the cost of van hire to transport the ‘smoothie bike’ between schools. The ‘goody bags’ and their contents (including the DVD) had already been produced and paid for by Gwent Police before the start of the trial as part of the earlier phase of programme delivery; Gwent Police made these items freely available.

Questionnaires completed by pupils in this study included questions on whether or not they completed the KAT classroom work, and rates of attendance at the family event (for pupils, families, and the number of adults present from each family). This information on programme reach could be used alongside data on programme costs to ascertain the cost per participant/family.

Translating inputs into financial costs

Based on our experience in this exploratory trial, we believe that it should be possible to translate all of the key inputs into financial costs which could then be used as part of an economic evaluation. A section of the programme handbook could be used to ask staff to record (in a pro forma in the book) how many hours they spend on KAT (pupil contact time and other preparation time). It should be possible for a process evaluation within any future effectiveness trial to record total pupil contact time in a subsample of the intervention schools, so that teacher self-reports and researcher observations can be compared.

Although we did not ask schools to calculate the cost of teacher input into the programme, we believe that it would be feasible to ask head teachers either to provide this information or to share sufficient information for us calculate an hourly cost.

In this exploratory trial, the cost of refreshments, transportation of the ‘smoothie bike’, and other non-staff costs were paid directly by the research team from a centrally held programme grant. In any future effectiveness trial, we would expect schools to organise and pay for refreshments at their family event, and to submit invoices for these and other costs (such as those associated with the use of ‘smoothie bikes’) to a central programme co-ordinator. These data could, therefore, be used to provide accurate information on the non-staff costs which schools need to recoup.

Process evaluation observation of classroom preparation activities allowed us to generate an overall impression of the amount of consumable materials (e.g. paper, pens and posters) that schools used. Given the difficulty of producing exact costs for these materials, the most efficient way of estimating the costs would be to ask each school to provide an estimate of all of the non-staff expenditure items they provide. This information could be requested at the same time as schools submit invoices for reimbursement of costs paid by a central programme grant. It was feasible to separate research-related and programme delivery-related costs in this exploratory trial. A health economic evaluation as part of any effectiveness trial would also need to ensure that all costs could be allocated to the research process or the delivery of the KAT programme.

Suitability of primary and secondary outcomes for an economic evaluation

In a future effectiveness trial it should be possible to analyse primary outcomes (relating to alcohol consumption) and secondary/tertiary outcomes (relating to family communication) alongside data on costs relating to programme delivery. We would be likely to adopt a similar approach to that currently being used in an ongoing trial of the SFP10–14, in which a cost–consequences analysis is being used. However, given the problems experienced with recruitment of parents into the current trial, it is unlikely that we would be able to access data from parents on service utilisation, which is a component of the data collection from parents in the SFP10–14 trial. Traumatic, rather than medical, problems are common consequences of adolescent alcohol misuse, and so estimating service use would focus on accident and emergency admissions 2 years post intervention (or after other specified follow-up intervals).

Feasibility of the data collection instruments

Our experience in this exploratory trial suggests that it is feasible to collect comprehensive and reliable data concerning programme delivery costs, and intervention reach and engagement. Low rates of missing data for the questionnaire items in this study which would form the primary and secondary outcomes in any future effectiveness trial suggest that it would be feasible to link these data with programme cost data. However, it is unlikely that we could collect service utilisation data from large numbers of parents using the methods employed in this exploratory trial, given the low recruitment rates encountered.

Systems for identifying eligible pupils

Although we asked schools for accurate and up-to-date information on numbers of pupils eligible to participate in the trial, and the number present in school on the days when data collection took place, we could have developed better systems for capturing and managing this information. To some extent, this weakness in our systems happened because recruitment and baseline data collection took place before the funded trial, and when there were no dedicated resources. However, we could have developed more accurate and comprehensive systems for determining total numbers of eligible pupils, how many pupils were present in school, and present in class, during data collections. More attention should have been given to this aspect of recruitment and data collection during initial discussions with schools, and accurate class lists should have been obtained at the earliest point possible.

Withdrawal of two schools from the study

During the trial we lost two intervention group schools, which decided to withdraw. The first of these did so because parents raised objections after main information and follow-up letters were mixed up in error. It is difficult to identify to what extent the parents’ objections were based on the nature of the study, or the switching of the letters, but this experience underlines the critical importance of systems to ensure that nothing is done by the research team which might unnecessarily cause concern to parents.

The second school withdrew on the basis of two main issues: the timing of, and the workload generated by, the KAT programme, and the burden created by contact from multiple members of the research team. In this case it is harder to identify ways in which we could have managed the process better, given that we spelled out very clearly on multiple occasions the timing and workload which participation in the trial would involve. In this second school, at least one member of staff appeared to have concerns about the applicability and acceptability of the KAT programme to Muslim families, though the school did not present this as a reason for their withdrawal. We knew that implementation of the programme and research in a school with large numbers of ethnic minority families, including a number in which adults did not speak English, was likely to be challenging. However, we discussed these issues in depth with the school, who felt they would not be problematic, and, because the school met our inclusion criteria and a key aim of the study was to assess the feasibility of the trial in different school contexts, we felt it was right to include them in our study. However, there may have been ways in which we could have reduced the perceived burden of multiple contacts from the research team.

Our data on the acceptability and feasibility of KAT to schools, parents/carers and pupils need to be viewed in the context of the withdrawal of these two schools. In particular, there are two specific limitations which should be noted. First, we are not entirely clear on the precise reasons why the two schools which withdrew did so, and the extent to which their withdrawals related to (1) the general acceptability of the research/KAT; (2) the manner in which we carried out the research; and (3) characteristics of the school, and the population of parents/carers which it served. Second, it must be acknowledged that for key aspects of programme implementation, we have data from only the three schools which remained in the study. It is possible that the balance or content of the themes present in the data might have been different had these two schools remained in the study, or if we had been able to collect more data subsequent to their withdrawal from the study.

Feasibility of collecting follow-up data in secondary schools

In our application for funding we did not plan to conduct long-term follow-up of pupils (as would happen in a future effectiveness trial), as this would be unfeasibly costly for an exploratory trial, and would delay any decision on the future of the programme beyond that which was likely to be acceptable to policy and practice partners. Our focus in this trial was on the overall acceptability of the KAT programme to schools, parents and pupils, and participation in a randomised trial, and the feasibility of implementing KAT as part of such a trial.

During recruitment of primary schools we decided to test the acceptability of using ‘opt-out’ parental consent for pupils’ involvement in the research component of the trial. We did this so that we would know whether or not such an approach would be feasible in a future effectiveness trial. All schools (including those which later withdrew from the study) were happy for us to use this method. However, the use of ‘opt-out’ parental consent at recruitment does have important implications for our ability to follow up children 2 years later, and after they have moved to secondary schools. In another ongoing trial, we have been able to successfully engage secondary schools in allowing the research team to conduct follow-up interviews in school with young people who were recruited either in the community or via primary schools. However, in that trial we have explicit written consent from parents (collected at baseline) for all aspects of the data collection involving their child(ren), including follow-up interviews.

Secondary schools may be less likely to provide assistance to us if we do not have written consent from parents. Therefore, although we have relatively strong data on rates of recruitment for primary schools and pupils at baseline, there is still significant uncertainty about the extent to which we could achieve adequately high response rates at long-term follow-up via secondary schools. In retrospect, we should have included some exploratory work with secondary schools to assess the acceptability to them of collecting follow-up data from pupils, and what requirements they were likely to have.

Assessment of cost-effectiveness

This study has provided a description of key cost and outcomes domains, and the feasibility of monitoring them on a wider scale. Based on our experience in this exploratory trial, we believe that it should be possible to identify all key costs and to translate programme inputs into financial costs which could then be used as part of an economic evaluation in any future effectiveness trial. Low rates of missing data in pupil questionnaires mean that it in any future effectiveness trial it should be possible to use primary and secondary outcomes alongside programme cost data. Given the low rates of parental recruitment into the trial, it would probably not be possible to collect service utilisation data from parents/carers, though it could be feasible to access data for young people on alcohol-related harms (e.g. A&E attendance). We recognise that our exploratory work for a future cost-effectiveness evaluation has some limitations, and that a more detailed economic evaluation setting programme costs against potential future savings would have provided a better basis for decisions about investment in any future RCT.

Implementation feasibility

Three schools completed programme delivery. One intervention school withdrew without delivering any part of the intervention, and one withdrew from the study after starting programme delivery, but had only done a small part of the classroom component (and had not held a family event). The three schools which completed programme delivery appeared to deliver most or all of the main components, and to retain the main underlying messages of KAT. In at least one class, there was a tendency for the negative aspects of alcohol to dominate rather than a more balanced set of messages on the healthy use of alcohol (‘not too soon, not too much’). Schools appeared to be generally happy with the KAT programme manual and the training provided by the health education consultant. However, some teachers had not read the programme manual, or were not aware of key guidance/instructions in it, and a number of teachers suggested that the manual could provide more detailed guidance. Of the three schools which completed programme delivery, only one asked pupils to write invitations to the family evening for their parents/carers. In one of the schools, the teaching staff appeared to believe that the health education consultant and research team would lead the KAT family event. This highlights the importance of ensuring that senior managers and other staff in schools take ownership and support the implementation of the programme (Table 38).

Intervention acceptability

The three schools which completed delivery of the KAT programme were happy with its content and structure, and were willing to deliver all of the main components. In these schools, very few parents objected to the study or intervention, or refused permission for their children to take part because they objected to the alcohol-related content. Most pupils enjoyed undertaking the KAT programme and few, if any, issues were raised about the content (Table 39). Process evaluation data support the idea that most parents who attended the KAT fun evening or who discussed the programme with their children found it acceptable (Table 40). In the first school which withdrew from the study, parents did raise concerns about the topic of the research/KAT programme to the school. It is difficult to ascertain whether this was mainly because the information and follow-up letters sent to parents/carers had been switched in order; as a result of particular aspects of the school population; a combination of these reasons; or due to another set of factors of which we were unaware. One of the reasons given by the second school which withdrew from the study was the time commitment needed by the programme and the timing of intervention delivery. However, one of the class teachers tasked with delivering the programme had concerns about its acceptability to the large number of Muslim parents in the school. Although process evaluation data indicated that some parents had asked questions about the KAT programme, we were not able to determine whether or not there were widespread concerns among parents at the school about either the research or the KAT intervention (Table 41).

Although we were not able to interview parents in school 8 (which withdrew after baseline data collection), the interview with one of the class teachers involved indicated that they had concerns about the relevance and acceptability of KAT to Muslim families. This was mainly related to the fact that because these families did not drink, an intervention which examined healthy behaviours and positive family communication in the home concerning alcohol would not be relevant. However, the programme does not assume that all families will drink alcohol, or that its content will be applicable only to those who do. The programme recognises that children will be exposed to alcohol-related behaviours, advertising and media portrayals, all of which take place outside the home setting. The programme aims to promote positive communication within families about alcohol, but the exact content of this communication is not intended, necessarily, to be based on family drinking. If KAT was to be run in schools with similar populations it would be important to fully address these issues.

Programme reach

The KAT programme demonstrated high levels of reach among both pupils and parents (Tables 42 and 43). In the three intervention schools which delivered KAT, 70.9% of pupils reported doing all of the KAT work in school, and a further 22.8% said that they had done part of it; 59.5% of pupils reported having attended the KAT fun evening at their school. For the three intervention schools as a whole, an average of 50% of children reported that at least one adult member of their family had attended the family event. While in two of the schools the figures were just below 50%, in the third school 65.7% of those families invited were represented by at least one adult. The ability of KAT to engage half of all the families invited to participate in the intervention is a significant achievement. When the three schools are ranked in order of the proportion of the families that attended, there is a positive association with FSM levels (Table 44). In the intervention group as a whole, pupils with higher Family Affluence Scale scores were more likely to have a parent who attended, suggesting that there is social patterning of family attendance at the school level. However, when rates of pupil attendance at the family event are examined, the proportions of pupils reporting low/medium and high Family Affluence Scale scores who attended the family event were almost identical (Table 45). A significant minority of pupils reported attending the KAT family event without a member of their family. It may be that these children remained in school (particularly where the fun evening was directly after the end of the school day) and were supervised by school staff, or that they came with other children’s parents. It is also possible that some children provided incorrect responses to the questions on attendance. Although the process evaluation researcher observed the family events, it was not possible to accurately identify the total number of pupils from the specific classes involved in the programme, and the precise number of families which were represented. In summary, KAT was able to engage with families from a range of socioeconomic backgrounds, and there is evidence to suggest that the programme could operate in a variety of school contexts.

There was some evidence from the study’s process evaluation that the ‘balanced approach’ advocated by KAT did not necessarily fit well with the experiences of families who had experienced alcohol-related problems. Although the training given to teachers as part of the KAT implementation dealt with this issue, the findings from the process evaluation may need to be further explored if KAT is implemented in the future. Although much of KAT’s content maps closely onto the school curriculum, some children may need additional support during the programme. Family communication processes are central to the programme’s intended mechanism of action. Children’s experiences of the programme may be negative if their parents/carers are not willing to engage in the intended prosocial communication, or do so in ways which are at odds with the core values of the programme.

Data from the study’s process evaluation – particularly from focus groups with pupils – highlighted that children were more likely to want to attend the family evening if their parents accompanied them. Ensuring that as many parents/carers as possible attend the family event would appear to be very important in ensuring that both pupils and parents receive the majority of the intervention, and that key intended family communication processes are set in motion. Were we recommending a future trial, we would need to maximise rates of participation by parents/carers, rather than aim for figures comparable with those achieved in this exploratory trial. As discussed above, a small but significant minority of pupils attended the KAT family event unaccompanied by members of their own families. It is not clear whether these pupils were with other participants’ families (and, therefore, may have directly experienced some of the key communication processes intended to occur) or if they sat by themselves or with friends.

Schools

This study tested the feasibility of recruiting schools for a future effectiveness trial and assessed the acceptability of randomisation to schools. From a total of 39 eligible schools, nine were recruited into the study (23%), though the recruitment process took longer than initially anticipated beacuse of delays in identifying the final school needed prior to randomisation. We purposefully selected recruitment strategies which could be scaled up as part of a future effectiveness trial, and tested their feasibility. We wrote to all schools in the study county inviting them to take part, and then made follow-up calls to all schools, until sufficient schools had been recruited. Therefore, we can be reasonably confident that the schools recruited into this exploratory trial might be broadly comparable with those likely to participate in a future effectiveness trial, and that the recruitment rates achieved in the current study could be achieved across multiple counties.

School recruitment could probably have been improved by inviting schools to participate during the academic year preceding a trial when it would have been easier for schools to include KAT in planning. However, evidence from two UK reports suggests that school staff generally may feel that their core function of delivering high-quality education to pupils may be compromised because they are overwhelmed with new initiatives. 151,152 Thus, it may not be realistic to expect better timing to result in a big increase in schools willing to participate in any further evaluation. McCrone et al. 151 note that involving staff is important to bringing about and maintaining change, suggesting a possible explanation for school 8’s withdrawal from the study.

Although KAT required input and commitment from head teachers, it also depended on the engagement of individual members of teaching staff, who were asked to deliver the intervention and facilitate data collection. Our findings, and those of our previous evaluation of KAT’s implementation,90 suggest that the quality and completeness of delivery of the intervention does depend to a large extent on the commitment and enthusiasm of individual teaching staff. A key issue here is perhaps the need to ensure that programmes such as KAT, which have a primary focus on health and well-being, can demonstrate how they might address educational outcomes for pupils, and thus help teachers to deliver their core role as educators. If such teachers can see the direct relevance and value of KAT, and its suitability for the pupils and families they work with, they may be more likely to fully engage with it. Individualised on-site coaching involving demonstrations, which show how interventions can be delivered in the ‘real world’ of the classroom, may be more effective ways of gaining teachers’ understanding of and commitment to an intervention than simply providing initial training alongside a handbook and support on request. 153 In this study, we sometimes found it very difficult to involve class teachers in initial meetings to discuss participation in KAT, and subsequent training and briefing sessions. Had we been able to achieve their involvement across all schools at initial programme set-up meetings, it might have been possible to identify and deal more effectively with some of the challenges which arose, in relation to both delivery of the intervention (e.g. the failure to include child invitations to parents in some schools), and teachers’ concerns about the appropriateness of the programme for families from ethnic minority backgrounds.

We recruited schools with a range of FSM entitlement rates, suggesting that, were we recommending a future trial, it would be possible to reach schools with families from a range of different socioeconomic backgrounds. In Newport, FSM rates range from 1% to 48.3% (median 18.4). FSM rates of the schools which participated in the trial were 1%, 2.3%, 21.6%, 27.9% and 37.2% (intervention), and 11.4%, 23.5%, 23.5% and 31.9% (control schools). The support of the local Healthy Schools Team (one of whose officers accompanied the research team on visits to schools to seek formal agreement to take part in the study) appeared to help communicate the value and importance of the study (Table 47).

From a small sample of nine schools, it is difficult to ascertain to what extent the recruitment and withdrawal rates in this trial might be replicated in a much larger effectiveness trial. For a larger effectiveness study, greater resources would be needed to recruit schools and develop better systems from the start, including contacting schools well in advance of the planned programme delivery period (thus minimising barriers to engagement in the trial). Because follow-up data would be collected from pupils in secondary schools, the loss of primary schools after baseline data collection (as happened in one of the cases in this study) would not have a major impact on the ability to collect follow-up data.

Although some of the schools which participated in the study expressed a clear preference to receive the KAT programme, randomisation was acceptable to all schools, and all those in the control group remained in the study and facilitated all data collection. Were we recommending a future trial, we would plan to collect baseline data before randomisation of schools took place, so it is unlikely that randomisation allocation would create significant risks to the viability of the trial. However, as noted above, the main follow-up data collections would take place in secondary schools, and not in the primary schools in which pupils were initially recruited. We did not include this aspect of data collection in our exploratory trial. If an effectiveness trial were to be considered in future, it would be important to undertake further work to assess the feasibility and acceptability of collecting follow-up data in secondary schools which had not been involved in the study from the start. In the trial itself, it would be important to engage with these secondary schools early in the research process, probably at the same time as the initial recruitment of feeder primary schools in which the study was taking place. As part of a current RCT of the SFP10–14, we have successfully engaged with secondary schools (most of whom have not had any role hitherto in the trial or the programme being evaluated) in order to conduct questionnaire completion with pupils in school. In these schools, we have written to the parents of the pupils concerned to inform them that we would like to conduct follow-up interviews with their child in school, and have given them the option to refuse permission. This system has been acceptable to most schools and enabled a significant number of interviews to be conducted in school, with very few parents refusing permission. Similar methods could be adopted for other school-based trials and also draw upon the network of contacts developed as a result. For instance, in our trial of SFP10–14, we have engaged the services of a retired head teacher, who has helped to negotiate entry into a large number of secondary schools. However, in the SFP10–14 study, ‘opt-in’ parental consent is being collected at baseline. Were we recommending a future trial of KAT, we would need to be clear right from the start about how many schools would be willing to allow follow-up interviews to take place with children without this ‘opt-in’ consent in place. Because pupils in KAT are recruited exclusively through the schools we would also be likely to need to interview more pupils per school at follow-up than is the case for the SFP10–14 study.

Pupil recruitment

Of those pupils eligible to participate in the study at baseline, response rates were 74% for the intervention group and 81% for the control group. Follow-up rates were 68% for the intervention group and 74% for the control group. If the school which completed baseline data collection but then withdrew from the study is removed (and from whose pupils no follow-up data could be collected) from the completion rates at follow-up, the figure for the intervention group rises from 68% to 85%. The main reason for non-completion of the questionnaires was absence from school or the classroom. In the control group, no eligible participants refused to complete questionnaires at baseline, and in the intervention group only 13 did so.

For both baseline and follow-up data collection, we paid a single visit to each school to collect data. At baseline, there were a total of 24 pupils absent from school on the day of data collection, and a further 40 who may have been present in school but not present in the classroom during data collection. Of those present in the classroom at baseline, 78% (intervention) and 88% (control) completed questionnaires – higher than the completion rates if the denominator used is those eligible. At follow-up, completion rates as a proportion of those present were 92% (intervention) and 77% (control). Had we made a second visit to each school to collect questionnaire data from pupils who were previously absent from school/their classroom, our recruitment and retention rates could have been increased. It should also be possible to optimise provision of support for pupils with low literacy levels, and we were able to achieve this to some extent when we conducted follow-up data collection. Literacy levels, rather than the questionnaire content, appeared to be the main reason for pupils refusing to complete questionnaires at baseline in at least one of the schools, based on feedback from teaching staff. A total of 39 pupils who did not complete questionnaires at baseline went on to complete follow-up questionnaires. Although we cannot be certain of the reasons for this, we feel it may be due in part to the provision of more members of research staff at follow-up data collection, and asking schools in advance how many pupils were likely to need assistance with reading questions. One suggestion made in one of the stakeholder group meetings (attended by key practice and policy collaborators) is that the use of online computer-based questionnaires could be considered. Internet-based activities and use of laptops is increasingly common in primary schools, though these might not be available in all schools. An internet-based questionnaire might be able to incorporate support for pupils with low literacy or offer questionnaires in languages other than English, and could also provide a means for pupils who were absent during data collection visits to complete the questionnaire at a later time point.

Our response rates also show some patterning according to sex and year group. More boys, and pupils in Year 5, completed questionnaires at baseline only. This may relate to differences in the proportion of girls and boys reaching expected literacy levels,154 but we cannot be certain as to the exact reasons. Were we recommending a future trial, it might be useful to consider in more detail whether there are aspects of data collection which are less appealing or more off-putting to boys, and how these could be modified. There were also sex differences in completion rates for our secondary outcomes (see Feasibility and acceptability of secondary outcomes).

Although rates of recruitment of pupils are important, the use of a cluster RCT design means that they need to be considered alongside rates of school recruitment (which is the unit of randomisation). Were we recommending an effectiveness trial, rates of school recruitment and retention would be very important in achieving the necessary statistical power.

Parents/carers

The number of eligible parents/carers who agreed to complete questionnaires was very low. Only 6.5% of eligible parents/carers participated in the questionnaire aspect of the study, and this is based on an assumption that there was only one adult in each household, so the true figure may be even lower. There are a number of factors which may explain this low rate. First, owing to data protection regulations, the research team was not able to access the names and contact details of eligible parents/carers from schools. We were only able to send a standard letter (which began ‘Dear parent/carer’) to parents via schools, and there was no way of identifying recipients who had not expressed an interest in participating in the research. Second, in this letter we asked parents to contact the school if they did not wish their child(ren) to participate in the study. If they were content for their child to participate, they did not need to make contact with school. In the same letter, parents were asked to complete a reply slip and return it to the research team if they themselves wished to take part in the study. It is possible that having two different consenting methods for children and parents in the same letter created confusion, or that, having established that no action was required of them in relation to their child’s participation, parents/carers were less inclined to respond to the remaining information. The method of providing parents with the option to withdraw their child from the study, rather than asking them to give explicit consent for their participation, is likely to have increased the rates of pupil involvement, and we chose this strategy deliberately as in an effectiveness trial primary outcome data would be collected from children. However, the relatively high rates of pupil questionnaire completion may have been at the expense of parental involvement. Third, in this exploratory trial we sought to use the best recruitment strategies which would be feasible and affordable if scaled up as part of a large effectiveness trial. With investment of more resources we could potentially have increased the rate of parental involvement in questionnaire completion (e.g. through holding information events, and spending more time in school, and placing less emphasis on letters). However, these procedures would be hard to sustain if an effectiveness trial involved a large number of schools. We were also concerned that a significant package of parental engagement work by the study team in schools could influence parental engagement in the KAT programme and school work, making it hard to disentangle programme processes from the research recruitment strategies. It is unlikely that we would be able to resolve all of these key challenges, and we return to this issue in our conclusion to suggest ways in which parents could be involved in effectiveness research.

Alcohol-related questions

Were we recommending an effectiveness trial, the primary outcome would be based on alcohol consumption. Thus, in this exploratory trial, we were concerned with assessing the acceptability and feasibility of collecting these data from primary school age children. Low rates of missing data (maximum 3.4%) suggest that most pupils felt comfortable with answering questions on alcohol consumption and understood their meaning. Rates of missing data were slightly lower at follow-up than at baseline.

There was, however, some evidence of misclassification, with pupils providing inconsistent answers. The main example of this was that a number of pupils answered ‘yes’ to having ‘ever had a drink’ at baseline, but answered ‘no’ to the same question at follow-up. It may be that children in this age group find it difficult to accurately remember such events over their entire life, especially if they do not have strong salience for the individual concerned,157 or that the measures are unreliable in other ways. Another possibility is that the baseline/follow-up questionnaires, the KAT intervention, or a combination of the two caused pupils to change their perception of their past drinking behaviours, or created a desire to provide socially acceptable answers. There were discrepancies between rates of ‘ever drinking’ and ‘past-month’ drinking at individual data collection points (i.e. there were more reports of ‘past-month’ drinking than ‘ever drinking’). The wording of the questions on past-month drinking was modified at follow-up based on feedback from pupils at baseline that the original phrasing was confusing, but this did not appear to improve the reliability of the data.

Our findings suggest that in effectiveness studies with children of a similar age, it may be preferable to measure past-month drinking rather than lifetime drinking as a primary outcome, and to use ’ever use’ as a marker of reliability. Past-month drinking should provide a relatively accurate picture of whether or not children have started to drink regularly. Were we recommending an effectiveness trial of KAT, additional secondary outcomes related to drinking (e.g. age of first drink, ever drink, drunkenness) would have formed important aspects of the overall design and analysis as well as the choice of a primary outcome (on which to base sample sizes and other statistical calculations).

In this study, data on alcohol-related behaviours were collected from young people at a point when most of them had yet to start drinking. Thus, for some of our data points (particularly past-month and lifetime drunkenness), prevalence rates are very low. Were we recommending an effectiveness trial, these data would have been collected after an interval of several years (at age 11–13 years for most participants). Because the children would be older, issues of questionnaire comprehension might be less apparent; it is also likely that rates of drinking and drunkenness would be higher (given the normal increase in drinking behaviours as children grow older), meaning that any issues with sparse data present in this trial would not be present. Clearly, age adjustment would be important and assessment of drinking trajectories between intervention and control group might be required.

In this exploratory trial, we did not aim to assess the effectiveness of KAT on alcohol-related behaviours, and there were a number of methodological and theoretical reasons for this. First, the focus of the trial was to examine the feasibility of an effectiveness trial of KAT, rather than to conduct a ‘mini trial’ of the intervention’s effectiveness. Second, especially given its delivery to children of primary school age in order to prevent early alcohol consumption, and among whom rates of drinking are very low, any impacts of KAT on alcohol behaviours would need to be measured several years later once the natural prevalence of such behaviours had increased. Third, the logic model for KAT clearly hypothesises that prevention of alcohol misuse is mediated via the strengthening of family communication and attachment, and, thus, any short-term assessment of the programme should focus on these intermediate outcomes. In this trial, we collected alcohol-related data mainly to assess potential completion rates, reliability, and the acceptability of the individual questions to participants. The trial was not, therefore, designed or statistically powered to detect differences in alcohol-related behaviours between trial arms. Given these caveats, it is not possible to use the data presented in Chapter 3 to draw conclusions as to the effectiveness of KAT in preventing alcohol misuse. Our data on rates of drinking and drunkenness at follow-up are mixed, and there is evidence of inconsistent reporting by pupils. However, taking into consideration the above caveats, some of our results suggest that KAT may have potentially harmful effects (particularly whether or not pupils report ever having had a drink of alcohol). There may be a number of reasons for these results, including actual harmful effects of the intervention, the imbalance in the trial arms at baseline in terms of age, etc., or the reliability of the measures used for this age group. It is also possible that the intervention content may have influenced how children answered questions at follow-up, though we do not know if this is the case.

Existing research provides numerous examples of alcohol, drug and other prevention interventions which have produced iatrogenic effects (including increasing substance use), even when carefully designed and well implemented. 158,159 It is important to understand whether such effects arise from a particular intervention component (or negative effects for a particular subgroup), an overall failure of intervention theory, or inadequate implementation which undermines the intervention’s theoretical basis. 159

Feasibility and acceptability of secondary outcomes

We faced considerable difficulties in identifying suitable and validated measures of family communication which were designed for children aged 9–11 years, and this was particularly the case for measures of family communication in relation to alcohol. These difficulties have significance beyond our particular study, as many other researchers may potentially need to measures outcomes similar to ours among primary school-aged children. Given the emphasis on delivering family-based prevention interventions to children before they start drinking, and while parents are still a primary point of attachment (i.e. prior to the teenage years),31,53,73 we were surprised at the paucity of validated measures in this area for primary school-aged children.

From the very small number of candidate measures that we identified, most were aimed at older age groups of children (and outside the UK), and were not always easily understood by participants in our study. Another weakness of many of the measures we considered using was that they assumed that children lived with either one or two parents at home, and were difficult to answer for some pupils who spent time with different non-cohabiting parents, or lived with other adult figures. While the measures we eventually used had strong internal consistency, it was not always possible to determine how the measures had originally been validated by their authors. In most cases, the measures had been used in cross-sectional or longitudinal studies, and there were few examples of their use in trials, and, thus, of what kinds of effect sizes might be expected from interventions similar to KAT.

The measures used were validated on our data set and showed good levels of internal reliability. Factor analysis showed that single summed scores were an appropriate way to summarise the data. However, the distributions of the scores indicated that linear modelling may not be the most appropriate method of analysis for these scores if used in an effectiveness trial, and that categorising the scores and using ordinal or logistic methods would be more appropriate.

Rates of missing data for secondary outcomes were higher than for the alcohol-related questions. For the Family Activity Scale, KIDSCREEN-52 and the PCCS, the completion rates all lay between approximately 75% and 90%. The FCS had higher rates of missing data than the other secondary outcome measures, particularly for Year 4 and 5 participants, but this is probably due to the way in which missing data were categorised, rather than because there were more questions unanswered. For all of these measures, missing data rates were higher for boys than for girls. Missing data rates appeared to be higher at follow-up than at baseline in Years 4 and 5, though there was no clear pattern for Year 6 pupils. The reasons for non-completion of the measures (or individual questions within them) could relate to their acceptability, the phrasing of the questions not being clear, or the fact that participants found them hard to answer. Analysis of the patterning of missing data suggested that ‘missingness’ was broadly equal across individual questions within the measures, rather than due to a small number of questions consistently being left unanswered (Table 49).

Identifying potential effect sizes

In this exploratory trial, our main aim in collecting data on alcohol-related behaviours was to assess the acceptability and feasibility of doing so in a future effectiveness trial, and we used data from 11- to 13-year-olds in the HBSC data set to calculate the sample size which would be needed for a 2-year follow-up.

As an exploratory trial, this study did not aim to measure the effectiveness of the KAT programme on predicted short-term outcomes. It is important to note that the trial was not powered to detect differences between the two groups, nor was it designed to ensure that intervention and control groups were equivalent in relation to key dimensions at baseline. In an exploratory trial such as this one, effect sizes provide only a broad estimation of what might be expected in an effectiveness trial in relation to intermediate outcomes on family communication. Analysis of these secondary outcomes indicated no evidence of any intervention effect (Tables 50 and 51). There are a number of possible explanations for these results. One is that KAT simply does not work, and that the results accurately reflect a lack of intervention effect. A second possibility is that KAT does, or may, have small short-term effects, but that our study has not detected them. This could be due to the imbalance of the intervention groups in important respects, the fact that the study was not large enough and not powered to detect effects, the unreliability of the measures used, or a combination of these factors. As noted above, we faced considerable difficulty in identifying suitable measures of family communication, and, even after adaptation by us, they were not always easily understood by participants, which may have influenced how they were completed. A third possibility is that our logic model for the programme (and, therefore, the dimensions we chose to measure) is incorrect or incomplete, and KAT operates through processes which we did not assess.

There were very few examples of previous studies which had used our secondary outcome measures. None were directly comparable to the current study, as they had used the measures with older age groups, in countries other than the UK, and had not used the measures to compare trial arm outcomes in RCTs. However, the average scores in our study sat within the range of scores previously recorded for these measures (see Table 50).

Assessment of Kids, Adults Together delivery costs

In this trial, we were able to identify key cost domains relating to the delivery of KAT and to capture the financial costs of most of these domains. Had we been recommending an effectiveness trial, it should have been possible to measure the cost of programme delivery, including those costs (such as staff time) incurred by schools. However, our inability to recruit parents/carers into this exploratory trial means that it would have been difficult to capture data on variations in service utilisation between trial arms. This would be a significant barrier to conducting a cost-consequence analysis (Table 52).

School-based alcohol misuse prevention programmes

Our review of recent studies identified that, while schools are generally acknowledged to be an important setting for the delivery of alcohol misuse prevention interventions and activities, there are a number of significant gaps in the literature. The majority of research on school-based alcohol misuse prevention interventions has been conducted in the USA, and the evidence base for UK programmes is, therefore, limited. 50 There is relatively little research on prevention interventions for primary school-age children,31,82 despite the recognition that this age group is the most appropriate development point for family-based programmes. 31 Long-term follow-up of participants is relatively rare. Although the involvement of parents and other community members has been shown to enhance intervention effectiveness, many school-based alcohol misuse prevention do not include them.

Our exploratory trial of KAT, therefore, makes an original contribution to the evidence base in these areas. KAT demonstrates features which have been shown to characterise effective prevention interventions: a theory-based design, interactive delivery styles and community involvement. 29–34 We deal with each of these in turn.

Issues which may have general relevance for school-based trials

1. Although rates of parent/carer involvement in the KAT programme were very high, we were not able to engage large numbers in the research element of the trial, and questionnaire completion rates were very low. Based on these findings, it would not be feasible to collect data on secondary outcome and service utilisation from parents in an effectiveness trial using the methods employed in this exploratory trial. Given that data for primary outcomes would be collected from children only, and that secondary outcomes would also be measured in children’s questionnaires, one option would be to remove all parent-based data collection from the trial. Another option would be to invest significant resources into increasing parent questionnaire completion rates, for example by offering a range of alternative response modes. However, the cost of this would probably be prohibitive and, for studies such as KAT that try to promote parental engagement, could have the effect of mimicking some of the intermediate processes, which would be problematic. It would also be very important not to alter parent recruitment methods in ways which could reduce the number of children who take part in the study. A study’s process evaluation could be designed in such a way that sufficient interviews with parents were included to explore potential intervention pathways. These data collection strategies would not provide an estimate of differences between the trial arms in relation to parent-derived outcomes, but they would allow greater understanding of potential mechanisms of action and provide information about the characteristics of parents who engaged with the programme.

2. In designing school-based trials, our findings suggest that it would be essential to build in sufficient time to recruit schools, and to approach them early in the life of the trial, so that intervention group schools had sufficient time to plan for programme delivery. However, while contacting schools well in advance may reduce the likelihood of schools refusing to take part because of time constraints or other projects, it could increase the risk of post-agreement/randomisation drop-out.

3. School recruitment was aided by having the support of a Healthy Schools Team practitioner. Recruitment and retention in school-based trials is likely to be increased by identification of influential figures such as healthy schools practitioners who can encourage schools to participate and remain in a study.

4. Offering incentives to schools to compensate them for the staff time and general inconvenience/disruption caused by trial participation (and staging these throughout a school’s involvement in the research process) may help to increase recruitment and retention rates.