School-based relationship and sexuality education intervention engaging adolescent boys for the reductions of teenage pregnancy: the JACK cluster RCT

Article history

The research reported in this issue of the journal was funded by the PHR programme as project number 15/181/01. The contractual start date was in January 2017. The final report began editorial review in September 2021 and was accepted for publication in May 2022. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The PHR editors and production house have tried to ensure the accuracy of the authors’ report and would like to thank the reviewers for their constructive comments on the final report document. However, they do not accept liability for damages or losses arising from material published in this report.

Copyright statement

Copyright © 2023 Lohan et al. This work was produced by Lohan et al. under the terms of a commissioning contract issued by the Secretary of State for Health and Social Care. This is an Open Access publication distributed under the terms of the Creative Commons Attribution CC BY 4.0 licence, which permits unrestricted use, distribution, reproduction and adaptation in any medium and for any purpose provided that it is properly attributed. See: https://creativecommons.org/licenses/by/4.0/. For attribution the title, original author(s), the publication source – NIHR Journals Library, and the DOI of the publication must be cited.

2023 Lohan et al.

Structure of this report

This report presents the findings from a cluster randomised controlled trial (RCT) evaluation of the If I Were Jack programme. This chapter provides the background for the study and a description of the programme. Patient and public involvement (PPI) in the study is described in Chapter 2. The major focus of the PPI involvement was in relation to the optimisation of the programme that occurred within this study prior to the trial. The methodology for the trial and process evaluation is outlined in Chapter 3. The results of the recruitment and data collection processes are outlined in Chapter 4. The quantitative findings from the trial regarding the impact of the programme on student outcomes are reported in Chapter 5, and the findings from the accompanying qualitative process study are set out in Chapter 6. Chapter 7 details findings of the economic evaluation, and key issues and conclusions emerging from the findings are set out in Chapter 8.

The public health problem addressed: reducing unintended teenage pregnancy and promoting positive sexual health

Relationship and sexuality education (RSE) for young people is a challenging, complex, controversial and critical worldwide public health issue. High-quality comprehensive RSE for young people is not only essential to achieving better health and health care for adolescents, but is also key to ensuring women’s rights, gender equality, and sound demographic and economic development for future generations. As such, high-quality RSE addresses three 2030 sustainable development goals: to ensure quality education, gender equality, and good health and well-being. 1,2

Although teenage pregnancy is not universally negative,1 reducing teenage pregnancy has the potential to reduce a myriad of negative medical outcomes, such as low birthweight and undernutrition, as well as the negative consequences for educational and social well-being of young people and their babies over the life course globally. 2–6 Teenage pregnancy is widely understood as not just a cause, but also a consequence, of under-privilege, which is why addressing teenage pregnancy through developing young people’s agency to improve their own lives is so important.

The UK still has the highest rate of teenage pregnancy in Western Europe. 7–9 Despite the fact that conception rates for women aged < 18 years have been falling (halving in the last decade), the rate in England and Wales in 2018 was 16.8 per 1000 population. 10,11 Approximately 25,000 teenage women become pregnant in England and Wales annually, and approximately half of these pregnancies end in legal abortion. 12 The teenage pregnancy rate in Scotland was 30.2 per 1000 in 2017. 13 In Northern Ireland (NI), abortion was illegal until October 201914,15 and was considered lawful only in exceptional circumstances in which the life of the pregnant woman was at immediate risk, or if there was a risk of serious injury to her physical or mental health. Reflecting the different legal framework (pre October 2019), government targets around reducing teenage pregnancies in NI relate to births and not conceptions. In NI, the birth rate to teenage mothers per 1000 young women aged 13–19 years was 11.3 in 2013. 16 In the same year, the teenage birth rate in the most deprived areas was 23.0 per 1000, nearly six times that of the least deprived areas (3.9 per 1000). 17

Preventing unintended conceptions implies preventing unprotected sex. Drawing from robust representative epidemiological data of school-aged children across the UK18 and the Young Persons’ Behaviour and Attitudes Survey NI,19 it is known that between 25% and 33% of 15-year-olds are having sex, with an associated rate of 2.8% reporting unprotected sex. Changing practices of unprotected sex and unintended pregnancy is complex, and cannot be achieved through RSE alone. 20–26 However, high-quality comprehensive RSE is an essential component in the process of reducing unintended pregnancy rates, as well as being a vital aspect of improving holistic sexual health and well-being. 27–33 The UK governments emphasise the policy importance of the implementation of RSE in schools in decreasing conception rates for those aged < 18 years and the promotion of positive sexual health among teenagers. 34–36

Current status of relationships and sexuality education

The current status of RSE provision in schools in the UK is as follows: educational policy, including relationship and sexuality education, is devolved to the nations of the UK. Hence, the terminology to describe the education young people receive relating to sex and relationships varies in each nation as follows. RSE guidance in NI is predominantly structured under the personal development (PD) strand of the Learning for Life and Work curriculum. In England, sex and relationship education (SRE) is provided under the umbrella term personal, social, health and economic education (PSHE). Pupils in Scotland receive relationships, sexual health and parenthood education as part of their Curriculum for Excellence framework. In Wales, SRE forms one of the six Areas of Learning Experience. For the purposes of uniformity in this report, the appellation RSE is applied UK-wide. In England and Wales, RSE is currently undergoing significant reform. England introduced compulsory RSE in 2020, post data collection for this trial,37 and Wales is set to do so in 2022/3. 38 In England, however, parents will still be able to opt out their children when the topic of sex is being discussed. NI and Scotland have statutory guidance on what should be taught, although this is not compulsory, and many schools follow their own guidance or ethos concerning how the curriculum is delivered. 39,40 Therefore, throughout the UK there is no uniform guidance for the curriculum, and, where guidance exists, it is open to interpretation by schools in terms of how it should be implemented.

The If I Were Jack programme

Intervention components

The intervention components available on the project webpage, www.ifiwerejack.com/ (accessed 20 October 2022), include:

• A culturally sensitive computer-based IVD to immerse young people in a hypothetical scenario of a week in the life of Jack, a teenager who has just found out that his girlfriend is unexpectedly pregnant. The If I Were Jack interactive film asks pupils to put themselves in Jack’s situation and consider how they would feel and what they would do if they were Jack. (The film is interactive in the sense that it requires viewers to respond about how they would think or act as the narrative develops, but the film does not change in response to these choices.)

• Classroom materials for teachers containing detailed lesson plans with specific classroom-based and homework activities designed to build pupils’ skills to (1) obtain necessary information and (2) develop communication skills with peers and trusted adults. Teachers can deliver the intervention to pupils during four 50- to 60-minute or six 35- to 45-minute weekly classes.

• Ninety-minute training session for teachers implementing the intervention.

• Two short animated films to engage parents/guardians and help/encourage them to have a conversation with their teenager about avoiding unintended pregnancy.

• Detailed information brochures, letters for schools, letters for parents about the intervention and factsheets and wallet cards about unintended teenage pregnancy (UTP) in general for schools, teachers, teacher trainers, young people and parents/guardians.

The Template for Intervention Description and Replication (TIDieR) checklist54 is used to describe the intervention in more detail, and this longer description of the intervention is available in Report Supplementary Material 1.

Theory of change and how it is hypothesised to work

Presented in Figure 1 is the theory of change logic model of the intervention. Described on the left is the core public health problem: the relatively high rates of teenage pregnancy, as noted above. In addition, the public health need to engage young men as well as young women in addressing teenage pregnancy is stated. In brief, we hypothesise that by encouraging personal identification with the UTP scenario in the IVD, we engage pupils in an exercise of the imagination whereby they stop and think about the consequences that a UTP might have on their current life and future goals. This identification and reflection process is reinforced by providing knowledge about the risks and consequences of a UTP and ways to avoid it, and offering opportunities to practise communicating about UTP with peers and parents/guardians (activities that also increase awareness of peer norms, personal and familial values, and beliefs about sexual behaviour and unintended pregnancy). We hypothesise that by targeting these psychosocial factors, as well as inviting scrutiny and critical thinking in relation to wider social norms (gender norms and class norms), we can have an impact on teenagers’ sexual behaviour via pathways through their intention to avoid a UTP. The behavioural outcomes we seek to achieve are delaying sex until young people feel prepared and avoidance of unprotected sex.

FIGURE 1.

Theory of change logic model of the intervention.

Evidence supporting intervention components and theory of change

The characteristics of effective RSE programmes that help to increase their impact on sexual risk-taking behaviours have been précised in a number of systematic reviews. 25,55–62 These impactful characteristics include the use of theoretically based interventions targeting sexual and psychosocial mediating variables such as knowledge, attitudes, self-efficacy, intentions, perceptions of risk and perceptions of peer norms that are linked to sexual behaviour change; the use of culturally sensitive and gender-sensitive interventions; the use of interactive modalities that promote personal identification with the educational issues and engagement of young people; the use of skills-building components; the involvement of parents in the RSE process; and facilitating linkages with support services. The If I Were Jack intervention represents an innovative combination of all these different elements and is therefore predicted to decrease young people’s sexual risk-taking behaviour in relation to avoiding teenage pregnancy. The evidence for each of the components is further broken down below.

Importance of comprehensive relationship and sexuality education despite limitations of impact on health outcomes

Above, we have described the evidence for the programme characteristics of high-quality comprehensive RSE. Before we embark on describing the aims, objectives and results of this study, it is important to also sound a note of caution in terms of the limitations of comprehensive RSE alone in impacting on complex health behaviours, such as preventing unprotected sex and consequently preventing teenage pregnancy and reducing sexually transmitted infections (STIs). Another significant motivator for this study, however, was the importance of CSE as an international human right of young people and as an expressed need by young people.

By way of definition, CSE teaches about abstinence as part of the foundation for avoiding STIs and unintended pregnancy, and also teaches about condoms and contraception to reduce the risk of unintended pregnancy and of infection with STIs, including HIV. It also teaches interpersonal and communication skills and helps young people explore their own values, goals and options. 113 The United Nations Educational, Scientific and Cultural Organization (UNESCO) specifies that CSE should be scientifically accurate, incremental, curriculum based, based on a human rights approach, based on gender equality, culturally relevant and contextually appropriate, transformative, and able to aid the development of life skills needed to support healthy choices. 33 This can be contrasted with the abstinence-only-until-marriage (AOUM) approach. The AOUM approach teaches abstinence as the only morally correct option of sexual expression for teenagers. It usually censors information about contraception and condoms for the prevention of STIs and unintended pregnancy. 113 If I Were Jack is part of CSE.

First, we acknowledge that two large cluster RCTs of CSE programmes24,114 in the UK did not show a significant effect on health outcomes. The SHARE (Sexual Health and Relationships Education) programme, trialled in Scotland, did not reduce conceptions or terminations by the age of 20 in comparison with conventional provision. 24 The Randomised Intervention of PuPil-Led sex Education (RIPPLE) trial was a peer-led sex education programme conducted in south-east England. The final results showed that the programme was not associated with change in teenage abortions but may have led to fewer teenage births. 114

Second, overall reviews of evidence have also detailed the limitations of CSE alone in affecting health outcomes, although they can have important impacts on knowledge, attitudes and skills. A NIHR-funded systematic review25 of the effect of interventions aiming to encourage young people to adopt safer sexual behaviour found that school-based interventions that provide information and teach young people sexual health negotiation skills can bring about improvements in behaviour-mediating outcomes such as knowledge, attitudes and self-efficacy but did not find significant effects on health outcomes. The review noted that these variables are, however, no less valuable than behavioural health outcome variables because they provide young people with a solid foundation on which to make sexual decisions. Similar findings have been noted in other reviews115–117> and meta-analyses, some of which are undertaken by organisations opposed to the implementation of CSE118,119 and favouring AOUM approaches. However, the AOUM approach not only has been found to be scientifically ineffective in helping adolescents to delay intercourse, but also is regarded as ethically flawed in terms of denying young people rights to information, endangering gender stereotypes and marginalising sexual minority youth. 120

Thus, despite some limitations noted in the scientific evidence for CSE alone in having an impact on ultimate health outcomes, such as teenage conceptions and teenage pregnancy terminations, the United Nations33 and WHO conclude from the evidence that ‘CSE can help adolescents to develop knowledge and understanding; positive values, including respect for gender equality, diversity and human rights; and attitudes and skills that contribute to safe, healthy and positive relationships’83 (WHO Recommendations on Adolescent Sexual and Reproductive Health and Rights. Geneva: World Health Organization; 2018. Licence: CC BY-NC-SA 3.0 IGO.). As the NIHR review also stated, these skills are no less important and reflect a holistic approach to RSE. 121–123

Moreover, young people have a right to high-quality comprehensive RSE. The United Nations Convention on the Rights of the Child,124 Articles 24 (health and health services) and 34 (sexual exploitation), stipulate that governments must provide education on health and well-being to enable children to protect their health, as well as implementing measures to protect children from sexual exploitation or violence. 124

Flowing from these fundamental rights, international human rights standards require that governments guarantee the rights of adolescents to health, life, education and non-discrimination by providing them with CSE in primary and secondary schools that is scientifically accurate and objective, and free of prejudice and discrimination. 125

The Committee on the Rights of the Child has further indicated that nation states should equip adolescents with relevant information in relation to contraception and avoidance of unintended pregnancy and STIs, including HIV/AIDS. 126

The right to sexual and reproductive health is also protected under the ‘right to the highest attainable standard of physical and mental health (https://www.refworld.org/docid/4538838d0.html; accessed 16 June 2020), enshrined in Article 12 of the International Covenant on Economic, Social and Cultural Rights. 127 Finally, the right information and education to promote SRHR is also linked to the United Nations Sustainable Development Goals (targets 3.7 and 5.6). 128

However, few countries follow human rights standards129 and include CSE curricula to be part of the mandatory school curriculum, or implement and sustain large-scale CSE programmes. 83 Hence, the Guttmacher–Lancet Commission on Sexual and Reproductive Health and Rights85 recognises the need for all countries to establish national curricula for comprehensive sexuality education based on evidence and drawing from international technical guidance such as that provided by the international policy community. 33 Furthermore, the Commission noted that, to be comprehensive, sex education must include strategies to increase gender equality and holistic health more broadly.

Finally, comprehensive RSE is also not just a right but a need, as expressed by young people themselves,41,45,96,130–133 and human rights standards state that the curricula should be developed with young people’s input, such as in the case of If I Were Jack (described more fully in the next chapter). With that balance of views on scientific evidence and children’s rights in mind, we now describe the aims and objectives of this study.

Aims and objectives of the JACK trial

The overall aim was to carry out the first UK-wide cluster RCT of a comprehensive relationship and sexuality education intervention using a gender-transformative approach to specifically engage with young men and young women to address teenage pregnancy and promote positive sexual health.

The objectives of the study were to:

1. assess the effectiveness of the intervention in preventing unprotected sex at 15 years of age among teenage boys and girls in a cluster RCT across the UK

2. assess the impact of the intervention on secondary outcome measures of knowledge, attitudes, skills and intentions to avoid teenage pregnancy, as well as additional behavioural outcomes of engagement in sexual intercourse, contraception use and STIs

3. examine any differential impacts for teenage boys and girls as well as for different socioeconomic groups and nations of the UK

4. conduct an economic evaluation of the intervention compared with current practice

5. conduct a process evaluation, examining reasons for participation and non-participation; intervention delivery and fidelity in intervention schools; RSE provision in all participating schools; and self-reported perceptions of effectiveness and moderating influences in intervention schools among a sample of pupils, teachers and school principals and parents.

The full protocol for this trial was published in June 2018,48 and the protocol and any updates to the protocol (described in Chapter 3, Deviations and rationale) can be found at the NIHR Evaluation, Trials and Studies website.

Introduction

Engagement of stakeholders, including policy-makers and commissioners, as well as teachers, young people and parents, has been central throughout this study. The purpose was to ensure that stakeholders’ views and opinions were used to inform key aspects of the intervention optimisation (stage 1 of this study), as well as study design processes, data collection procedures and dissemination. We begin this chapter with an overview of stakeholder involvement and then move to a fuller description of the process and results of stakeholder involvement as part of stage 1 of this study: intervention refinement and optimisation.

Overview of purpose and mechanisms of stakeholder engagement

The purpose and mechanisms of stakeholder engagement were as follows.

Stakeholder involvement in stage 1: intervention refinement and optimisation

Conclusion

The involvement of stakeholders has been a key method of ensuring the quality and acceptability of the intervention and trial methods. Stakeholder voices and opinions informed intervention development and optimisation, study design processes and dissemination. The dedicated intervention refinement phase and pilot study allowed optimisation of the intervention materials prior to progressing to the stage 2 trial. Close consultation with stakeholders throughout was key to the success of stage 1 of the project and laid solid stakeholder-informed foundations on which to build and implement processes and procedures for stage 2 of the study. All resulting materials were made available on the If I Were Jack website post trial.

Study design

The JACK trial was a Phase III134 multicentre, parallel-group, cluster RCT with two treatment groups: the If I Were Jack intervention versus schools’ usual RSE provision. Schools were the unit of randomisation, with a 1 : 1 allocation. An embedded process evaluation and economic evaluation were conducted (economic evaluation methods are detailed in Chapter 7). The trial protocol was published. 48

Ethics approval and research governance

The trial was conducted in line with the ESRC Framework for Research Ethics and received a full ethics review by the School of Nursing and Midwifery (QUB) Research Ethics Committee in July 2017 (Ref: 11.MLohan.05.17.M6.V1), which independently assessed our compliance with the ESRC Framework. This approval covered data collection in each partner site. A trial steering group was convened to oversee the trial (see Report Supplementary Material 3). QUB acted as the main sponsor of the research and ensured that governance and indemnity procedures were in place. The project was registered on the Human Subject Projects database in QUB and prospectively registered in an international register of trials (ISRCTN99459996).

Participants

The study sought to recruit schools in the four nations of the UK, and to recruit teachers and pupils within these schools. Parents were also recruited but participated only in the process evaluation.

Recruitment procedure

Informed consent

Written, informed consent for random allocation was obtained from a member of the school senior leadership team, and from students, parents, teachers and intervention delivery staff for data collection. Parents were also informed about the trial and offered the opportunity to withdraw their child from the trial (although not the intervention).

Particular attention was paid to enhancing the process of informed consent for students within a school environment through the following measures:

• Information and informed consent forms were administered and collected by the research team. Although teachers may have been present during this process, it was emphasised to pupils that their participation in the study was voluntary and that, should they decline to participate, they would have the opportunity to continue with independent study during data collection periods.

• The research team also produced a short animation video informed by the pupil members of the TSG to convey what the study involved and their right to consent, as well as their right to withdraw their consent at any time without giving any reasons. This was shown to pupils in schools prior to asking for written informed consent.

Randomisation, concealment and blinding

When the recruitment quota for schools had been filled prior to baseline data collection, schools within each nation and socioeconomic stratum were randomly allocated (1 : 1 concealed allocation) to a trial group by an independent Northern Ireland Clinical Trials Unit (NICTU) statistician. The statistician produced eight randomisation schedules (using unique identifiers for schools), one for each FSM stratum for each nation, using random permuted blocks of mixed size, generated using Dotmatics nQuery Advisor 7.0 (San Diego, CA, USA). The NICTU was not involved with recruitment and released the randomisation code only when all schools were recruited and baseline data collection was completed, ensuring allocation concealment.

The statistician sent teacher trainers the allocation schedule for all schools. The teacher trainers were part of the intervention delivery team, and independent of the external research team. School allocation was communicated to each school by the nation-specific teacher trainer following baseline data collection within the school. Research teams were unaware of school allocation until after schools were informed of their allocation. (In a trial of this type, it was not possible to blind participants to allocation.) At randomisation, the NICTU Trial Statistician also randomly selected two intervention group schools from each nation as case-study schools for the process evaluation (see Process evaluation methods).

Treatment group allocation

Data collection and management

Data collection

Baseline

At baseline (prior to commencement of intervention delivery), fieldworkers administered paper-based questionnaires to participating pupils, which they completed during the school day under exam conditions. Teachers were asked to stay at the front of the room to maintain order while also alleviating pupils’ concerns that teachers could see their answers. Fieldworkers supported pupils requiring extra help and ensured that questionnaires were completed confidentially.

Follow-up

Participating pupils were in the study for approximately 18 months, and completed follow-up questionnaires between 12 and 14 months post intervention using procedures identical to those employed at baseline. The data collection timeline is shown in Figure 2.

FIGURE 2.

Data collection timeline. PSE, Personal and Social Education.

Outcomes and measures

The outcomes measured in this trial are based on the logic model (see Figure 1) and described in Table 3. A full copy of the baseline questionnaire is available in Report Supplementary Material 3.

Sample size

This trial was powered to detect a 50% reduction in the incidence of unprotected sex (from an expected rate of 2.8% to 1.4%) by 15 years of age. A difference of 1.3% in unprotected sex has been shown to have a meaningful impact on pregnancy rates. 24,146–148 The between-group difference in the incidence of unprotected sex of 1.3% [95% confidence interval (CI) 0.5% to 2.2%] by 9 months in our feasibility trial149 demonstrated that such an effect size is plausible and is consistent with effect sizes seen in the literature. 146 This study was designed to also take account of clustering. In the feasibility trial data, the intraclass correlation coefficient (ICC) was 0.01. 44 As pilot studies can provide imprecise estimates of ICCs,150 we re-estimated using ICCs from three sources: (1) the RIPPLE cluster RCT,148 (2) the WHO Health Behaviour in School-aged Children Survey (HBSC) GB18 and (3) the Young Persons’ Behaviour and Attitudes Survey 2011. 151 The data from the WHO and Northern Ireland Statistics and Research Agency (NISRA) studies were combined. Both the RIPPLE and combined WHO and NISRA studies found an ICC of 0.004. Based on (1) the ICC of 0.01 (95% CI 0.01 to 0.04), and assuming 120 students per year group in school, and (2) a 7% rate of attrition (based on the conservative attrition rate found in the feasibility study plus two additional schools), the sample size calculation showed that a trial involving 33 schools per group would provide 80% power at a 5% significance level. The alternative ICC of 0.004 was calculated to provide 93% power.

Statistical analysis

Process evaluation methods

Data analysis

All audio files were transferred securely to QUB and transcribed verbatim (interviews and focus groups) or typed up by researchers (observational field notes and other secondary source data). Data were organised using NVivo 12 (QSR International, Warrington, UK) software and analysed systematically and thematically based on the six steps proposed by Braun and Clarke155 to enable identification and analysis of patterns within the data by moving iteratively between theoretical understandings and the new data.

Methodological rigour was ensured by establishing credibility, transferability, dependability and confirmability using techniques suggested by Lincoln and Guba. 156 These inductively and deductively derived codes were first compiled as a code book and then applied to the data. These codes as applied to the data were discussed with the researchers from each of the four nations to check the validity of emerging themes and bring out some of the complexity in the data. All transcripts were independently coded by two members of the research team (10% coded by seven members). Meaningful quotations from participants were extracted to support and exemplify identified themes/subthemes. The data were analysed to form overarching themes emerging from each of the participant groups outlined above and also across these groups, but in all cases paying attention to deviant data. Open-ended questions from the SEQ and parent survey were transferred to NVivo 12 and thematically coded as per other qualitative data.

Implementation fidelity data were entered into Excel files in each site and transferred to QUB for summary descriptions. Data from the parents’ survey and SEQ were imported into Excel and SPSS 27.0 (Statistical Product and Service Solutions; SPSS Inc., Chicago, IL, USA) and tabulated as summary statistics. Descriptive data analysis was conducted to identify general patterns in the data.

FIGURE 3.

Flow diagram showing how the primary outcome was assessed. 0, Avoided unprotected sex; 1, unprotected sex.

Deviations and rationale

The protocol for this evaluation was published in June 2018 and amended in August 2019. It is available at the NIHR Evaluation, Trials and Studies website (https://www.southampton.ac.uk/netscc/index.page; accessed 20 June 2022). The original aims and objectives of the evaluation have not altered, and the overall approach to the research design in relation to the cluster randomised trial, the process evaluation and the cost-effectiveness evaluation has also remained unchanged. A small number of minor deviations to the original protocol published in June 2018 have been made, and these are detailed below.

Aims

The purpose of this chapter is to report the recruitment and data collection achieved in the cluster RCT and in the process evaluation. We also report attrition at a cluster and individual level, and the reasons for withdrawal.

School recruitment

A total of 803 schools were initially identified from educational authority records in each of the four nations as being eligible for participation based on inclusion criteria. These criteria included schools in defined geographic areas, as set out in the recruitment protocol (see Chapter 3). Independent, private schools, special schools, Irish-/Welsh-medium and Scottish Gaelic schools (but not those with an embedded Irish-/Welsh-medium component) and schools with < 30 pupils per year group (year 11 in NI, S3 in Scotland and year 10 in England and Wales) were excluded. Schools involved in previous studies involving the If I Were Jack intervention (n = 15) were also not eligible to participate in the current study. Eligible schools were stratified by socioeconomic status based on eligibility for FSM, resulting in two lists identifying schools above and below the nation-level median percentage of pupils eligible for FSM. These lists were then sent to NICTU and shuffled randomly before schools were approached (see Report Supplementary Material 3 for recruitment protocol). From these lists, 263 schools were contacted (172 in the above the median FSM stratum and 91 in the below the median stratum) and 66 were recruited, giving an overall recruitment rate of 25%. Rates of recruitment by nation were 29% NI, 11% England, 60% Scotland and 40% Wales. Reasons for recruitment and failure to recruit are discussed in the process evaluation results (see Chapter 7).

Flow of participants in the trial

Consolidated Standards of Reporting Trials diagram

In total, 66 schools (n = 8216 pupils) were recruited to the JACK trial, with 33 (n = 4100 pupils) allocated to the intervention group and 33 (n = 4116 pupils) allocated to the usual RSE practice control group. The Consolidated Standards of Reporting Trials (CONSORT) flow diagram is shown in Figure 4, summarising school eligibility, number approached, recruitment and randomisation, and retention through the trial, along with median cluster size. Figure 4 additionally reports the number of schools that declined to participate prior to randomisation and withdrew following randomisation (n = 1) or were lost to follow-up (n = 3). It also reports the number of pupils who opted out of participation within schools and those lost to follow-up. Losses to follow-up at the school and pupil level are discussed further below.

FIGURE 4.

The CONSORT extension for cluster trials 2012 flow diagram (nation breakdown included). a, Sites were allocated to intervention or control post baseline data collection. E, England; S, Scotland; W, Wales.

Baseline comparability

Process evaluation

In this section, we report the data collection achieved in the process evaluation and outline the removal of process evaluation data collected from schools that withdrew from the trial.

Baseline data

The values for the primary outcome and secondary outcomes at baseline are summarised in Table 10. There were no notable differences between the intervention and control groups in relation to the primary or secondary outcomes at baseline. There were also no differences between the two groups in relation to missing data at baseline.

Primary outcome

The primary and secondary outcomes are summarised in Table 11.

The number of young people in intervention schools who avoided unprotected sex (sexual abstinence or use of reliable contraception at last sex) was 2648 (86.6%), compared with 2768 (86.4%) in control schools [adjusted OR (aOR) 0.85, 95% CI 0.58 to 1.26; p = 0.42]. An exploratory post hoc analysis showed no effect on self-reported sexual abstinence at 12–14 months [2407 (78.30%) in the intervention group and 2511 (78.25%) in the control group; aOR 0.85, 95% CI 0.58 to 1.24; p = 0.39]. However, the exploratory post hoc analysis also showed that students in the intervention group were more likely than those in the control group to report using reliable contraception at last sex [42/106 (39.62%) in the intervention group vs. 29/110 (26.36%) in the control group; aOR 0.52, 95% CI 0.29 to 0.92; p = 0.025).

Secondary outcomes

Students in intervention schools were more knowledgeable about safe methods of contraception and avoiding unintended pregnancy [mean difference (MD) 0.18, 95% CI 0.024 to 0.34; p = 0.02], had improved attitudes towards progressive male gender roles (MD –0.33, 95% CI –0.64 to –0.02; p = 0.04) and had stronger intentions to avoid an unintended pregnancy (MD 0.61, 95% CI 0.16 to 1.07; p = 0.01) than students in control schools. There was no statistically significant difference in sexual self-efficacy or comfort communicating about avoiding unintended pregnancy.

Intentions to avoid unintended pregnancy included subscores on contraception, willingness, timeliness, readiness, norms and attitudes. Although the results indicate no effect of the intervention on the intentions subscales of willingness (MD 0.015, 95% CI –0.11 to 0.14; p = 0.81), timeliness (MD –0.025, 95% CI –0.075 to 0.026; p = 0.34) or readiness (MD 0.050, 95% CI –0.041 to 0.14; p = 0.28), students in the intervention schools had stronger intentions to use effective contraception (contraception subscale: MD 0.34, 95% CI 0.15 to 0.54; p < 0.001), to avoid peer pressure to have a sexual relationship (norms subscale: MD 0.24, 95% CI 0.012 to 0.47; p = 0.039) and not to have sex until ready and be prepared and to share responsibility for contraception with a partner (attitudes subscale: MD 0.31 95% CI 0.18 to 0.44; p < 0.001).

There were no differences between the intervention and control groups at the 12- to 14-month follow-up in relation to the number of students who reported ever having sex without contraception (intervention 42.57% vs. control 44.38%; OR 0.66, 95% CI 0.40 to 1.09; p = 0.10. However, at follow-up, students in the control group were more likely than those in the intervention group to report no use of contraception at last sex (intervention group 27.53% vs. control group 32.88%; OR 0.55, 95% CI 0.31 to 0.97; p = 0.04).

Subgroup analysis

The results of the subgroup analyses are summarised in Table 12. An exploratory post hoc subgroup analysis was conducted to look specifically at the effect of the intervention for those who were sexually active at baseline (average age 14 years). Among those sexually active at baseline, the proportion of students who had had unprotected sex by the 12- to 14-month follow-up was lower in intervention schools than in the control schools (49.38% vs. 60.92%; p for interaction = 0.06). The CIs in the adjusted analysis indicate benefit for nearly all who were sexually active at baseline (OR 0.56, 99% CI 0.29 to 1.09). Narrowing this down to look at the subgroup who had had unprotected sex by baseline, the results also favour the intervention but are not statistically significant (OR 0.64, 99% CI 0.28 to 1.46; p = 0.27).

Other subgroup analyses focused on sociodemographic characteristics and their effect on the primary outcome results. In relation to nation, there was evidence that the intervention decreased the rates of unprotected sex in NI, Scotland and England. The p-value for the test for interaction (0.05) indicates evidence of a difference in the effectiveness of the intervention between nations. The CIs in the adjusted analysis indicate benefit for nearly all participants in England (OR 0.46, 99% CI 0.18 to 1.13). In Wales, the direction of the effect of the intervention differed from that of the other nations but the CIs were especially wide and indicate no significant difference between the intervention and control groups in Wales (OR 1.68, 99% CI 0.72 to 3.91).

There were no differential effects of the intervention on the primary outcome according to participant socioeconomic status, as measured by the Family Affluence Scale (p = 0.60), participant sex (p = 0.81) or participant ethnicity (p = 0.16).

Sensitivity analyses and missing data

Standardised scores were calculated for the Male Role Attitudes Scale, Comfort Communicating Scale, and Intentions (including subscales) to deal with partial missing data arising because students responded to some but not all questions (see Chapter 4). The results of the secondary analyses (based on the standardised scores) for these scales suggest that the intervention was effective in increasing adolescents’ intentions to say no to sex until ready (MD 1.26, 95% CI 0.026 to 2.49; p = 0.045), which was not observed in the main analysis reported above. The results for the other scales were similar to those reported in the main analysis.

Follow-up data were not available for four schools (three intervention and one control; see Chapter 4). The sensitivity analyses made different assumptions on the best- and worst-case scenarios. The best-case scenario assumed that none of the students in the intervention schools had unprotected sex but that 13% of students in control schools did. The worst-case scenario assumed that none of the students in the control school had unprotected sex but that 13% of students in the intervention schools did. The rate of 13% was based on the school with highest rate of unprotected sex at baseline.

The results of the sensitivity analysis are presented in Table 13. In the best-case scenario, at the 12- to 14-month follow-up, fewer students in the intervention school than in control schools reported unprotected sex (12.05% vs. 14.53%, OR 0.59, 95% CI 0.34 to 1.03; p = 0.06). In the adjusted worst-case scenario, a slightly higher number of students in the intervention schools had unprotected sex than those in control schools (14.92% vs. 13.49%, OR 1.39, 95% CI 0.71 to 2.71; p = 0.34).

Serious adverse events

This was a low-risk study; therefore, no data monitoring was required. No adverse events were observed or reported.

Aims

The published aims48 of the process evaluation were informed by realist approaches to the evaluation of interventions134,153 as well as by MRC guidance154 for process evaluations to consider the context, implementation and mechanisms of the impact of interventions. Our aims were as follows:

• Context

  • Examine reasons for school participation and non-participation to inform the risk of bias in the trial as well as the long-term sustainability of the implementation of the intervention.

• Implementation

  • Examine intervention delivery and fidelity in the context of overall RSE provision in intervention schools.

  • Assess RSE provision in control schools and potential contamination caused by any changes to provision that could be due to participation in the trial.

• Mechanisms of impact

  • Explore self-reported perceptions of effectiveness and moderating influences in intervention schools among a sample of students, teachers and school principals/head teachers and parents.

Below, we address each of these aims, in turn detailing the data sources used to address each section, namely context, implementation and mechanisms of impact.

Context

Implementation fidelity

This section reports on intervention implementation fidelity to cover all aspects of implementation as follows:

1. Implementation of teacher training – assessed through a random selection of audio-recordings of the teacher training.

2. Implementation in the classroom – in terms of both the structure of the programme and content – assessed through 35 observations in classrooms in eight case-study schools; 128 teacher implementation logs; 29 individual interviews; five paired interviews (n = 10 participants); eight focus group interviews (n = 31 participants) with teachers; and eight focus group interviews with students (n = 58) in case-study schools.

3. Implementation of the parental component – assessed through an online survey with parents (n = 134 returned) and a SEQ (n = 3179).

Mechanisms of impact

In this section, we refer to the underpinning theory of change of the intervention based on social cognitive theory (theory of planned behaviour) along with gender transformative theory (which challenges gender inequalities) as discussed in Chapter 1. Based on the logic model, we focus on the end users’ self-perceptions of effectiveness in relation to some of the key active programming mechanisms of If I Were Jack. These are:

• engaging boys and gender transformation

• knowledge acquisition

• enhanced communication skills

• intentions for behaviour change in relation to avoiding unprotected sex (delaying sexual intercourse/practising safe sex).

By breaking it down in this way we offer a better sense of what was perceived to have worked best for whom, as well as an overall sense of perceived effectiveness.

The data for this section are drawn from:

• teacher and trial champion interviews (n = 29 individual interviews and n = 5 paired interviews, total n = 39 participants)

• teacher focus groups (eight groups, n = 31)

• student focus groups (eight groups, n = 58)

• SEQs (n = 3179).

Summary and conclusion

This chapter was structured to answer focused questions of the process evaluation as defined in our study protocol. 48

First, we asked why schools chose to participate or declined to participate in the trial. The data demonstrated the primary reasons for participation were an acknowledgement by schools that If I Were Jack was a high-quality, novel resource; a recognised need by schools to address RSE/holistic pastoral education; recognition that there was a gap in current materials in relation to engaging males; the opportunity for teacher training that could have wider benefits for future RSE programmes; and the financial incentive. The main reasons for not participating were research saturation or other commitments, a perception that there was adequate RSE provision already in place, and the additional time commitment required by the trial to co-operate with the research components and/or programme delivery.

Second, we asked whether or not the intervention was delivered with fidelity. The data demonstrated that the teacher training and classroom activities were generally delivered with high to medium fidelity, with some exceptions, but that the parental component, an optional element, was delivered with low fidelity. We noted, however, the potential for a high risk of bias in measuring implementation fidelity of the classroom-based activities on teacher self-reports in implementation logs, some of which was picked up through data triangulation with classroom observations, student focus groups and SEQs. Gathering data through implementation logs was problematic.

Relatedly, we sought understanding for the reasons for high and low implementation, especially of classroom activities and the parental component. High fidelity to classroom implementation was enhanced by teacher training and the manualised and user-friendly nature of the programme, following years of prior optimisation with teachers. The primary reasons reported by teachers for deviations in intervention delivery fidelity were insufficient time, perceived clash with school ethos and inadequate computer facilities. Implementation fidelity was also more generally impacted by teacher creativity, that is teachers changing/adapting materials to the perceived needs of students, including SEN students. Low fidelity to implementation of the parental component was explained by parents in terms of not receiving the material and embarrassment or fear of raising these issues with their child. Teachers reported barriers to implementation owing to student refusal and/or reluctance at school or teacher level to engage with parents on these issues. Students said it would be embarrassing and awkward. A proposed solution to mitigating these barriers was normalising these conversations between parents and school at an earlier stage in education, with continuity on the topic throughout all stages; in other words, avoiding the ‘big talk’ with schools and parents at age 11/12 years.

Third, we asked about the nature of RSE delivered in control schools and whether or not this had changed owing to participation in the trial as a marker of risk of bias. Judged against assessment criteria informed by a best practice tool of RSE implementation in the UK (Sex Education Forum, 2020),206 most control schools (n = 15) received ‘low’ ratings for their overall RSE provision, with eight receiving ‘medium’ ratings and only three receiving ‘high’ ratings. There was no indication that the direction of RSE had substantially changed as result of participation in the trial. There was also broad comparability in the quality of RSE provision in both control and intervention groups outside the provision of If I Were Jack. On the whole, the quality of RSE provision in both groups was rated as ‘low’ to ‘medium’.

Fourth, we asked about the perceptions of effectiveness of the intervention, from the perspectives of both teachers and students, especially probing key theory-driven programming mechanisms as depicted in the intervention logic model. The data demonstrated that the gender focus of the intervention, the engagement of males and the generation of empathy and understanding of both male and female perspectives was a key strength of the programme: ‘[i]t really got them communicating and talking about things and seeing things’ (T0314W). Potential limitations to effectiveness relating to this mechanism were a perception that the programme could overshadow female perspectives and an acknowledgement that a short programme may be insufficient to challenge deeply embedded gendered ideas around sexuality.

The data demonstrated that the intervention was an opportunity to gain new knowledge on sex and relationships as well as acquiring skills of where and how to find information and support. According to teachers, it ‘lifted heads’ and was an ‘eye opener’ (T0124NI). Students reported that learning where they could get ‘contraception and other things’ was ‘very helpful’ as well as learning ‘where you could get help if you need it’, and for males especially, ‘it was quite useful to know that you could go and do that’ (T0124NI). A key limitation in the programme to generating knowledge acquisition was its strong heteronormative bias and focus on teenage pregnancy, although some LGBTQ+ students thought that the programme was nonetheless relevant.

Teacher and student perceptions that the programme provided an opportunity to learn and practise communication skills were especially evident in the data. Teachers and students both noted increased confidence among students to communicate with peers about sex and relationships. According to one teacher, ‘we treated them with a mature topic, in a mature way, [and] they responded’ (T0112S). The interactive video seemed to be an especially important trigger for generating communication skills on relationships and sexuality in the classroom. This was because the IVD was regarded as centring the importance of young people’s thoughts and ideas, thereby laying the groundwork and building confidence for young people in the classroom to pick up and continue the conversation. There was a strong sense that these communication skills would endure, although limitations to this were also noted.

Finally, the perceived impact of effecting behavioural change, especially in terms of delaying sex until one is ready and practising safe sex, was considered. Teachers and students tended not to ‘over-promise’ the programme’s potential impact on behaviours: ‘it’s like early days’ (T0115E). However, there were ample data to suggest that the programme provided an opportunity to think about and plan sexual behaviours in ways that were positive and healthy, in terms of both the messages that teachers conveyed [‘I think, in every lesson, there was an opportunity for me to say that, “remember this is meant to be a positive experience” ’ (T0420NI)] and the messages that students recounted for themselves {‘[m]ore guys are thinking about [sex] way more than girls, so if they think, “oh, fine, I’ll just wait until my girlfriend’s ready”, then it might be better for both of us in the long term’ (Male, PU0229NI)}.

In the final discussion chapter, the process evaluation data, the trial data and health economics data will be considered in the round to draw together the final conclusions of the study.

Aim

The aim of the economic evaluation was to describe the costs and consequences of implementing If I Were Jack in UK schools to provide information to decision-makers on the implications of rolling out the intervention further. Two analyses were conducted:

1. Within-trial economic analysis. The aim of this analysis was to calculate the cost of teacher time to deliver If I Were Jack compared with current RSE provision (and hence the opportunity cost) based on teacher-completed resource use questionnaires. Descriptive statistics for sexual health-related health-care resource use and costs, and inputs to the model based on participant-completed data, were also calculated.

2. Decision analytic model. The aim of this analysis was to report the long-term cost and consequences of If I Were Jack based on a decision model, with input data collected from the trial in addition to relevant data from the published literature.

Methods

Long-term decision model

We designed a behaviour change-based decision model67 aimed at assessing the long-term cost-effectiveness of the If I Were Jack intervention versus RSE. The model was populated using evidence collected from the trial and published literature on different types of outcomes that are directly and indirectly related to pregnancies and STIs among young people. When no trial data were available, assumptions were made from pre-existing data and expert input. The model had a 20-year time horizon and calculated the expected costs and consequences for a hypothetical cohort of patients with similar characteristics to those enrolled in the trial. Twenty years was thought to be sufficient to capture the long-term benefits. People’s sexual behaviour changes when they are in their late 30s,166 and hence we did not model beyond this time-point. Both deterministic and probabilistic sensitivity analyses (PSAs) were conducted to assess the robustness of the results to alternative modelling assumptions. The long-term decision analysis model was conducted in R version 4.0.3 (The R Foundation for Statistical Computing, Vienna, Austria).

Design

A conceptual model developed to evaluate the long-term costs and consequences of the intervention, which incorporated theories of behaviour change, is illustrated in Figure 5. Reading from left to right, we started with the target population (young people aged < 16 years) who engaged with the If I Were Jack intervention and experienced an increase in the levels of the cognitive outcomes in topics related to sexual health (knowledge, attitude and skills). We hypothesised a change in the behaviour outcomes with an increase in the number of people who delayed vaginal sexual intercourse (abstinence), an increase in the instances of protected sex for those sexually active and a decrease in the instances of unprotected sex. This, in turn, led to a decrease in the number of unintended pregnancies and STIs in the target population.

FIGURE 5.

Health economic conceptual model of the If I Were Jack intervention.

Population

The target population was young people in post-primary schools in the UK, under 16 years of age, within the general population. An intervention that encourages the use of contraceptives is unlikely to prevent young people who hope to become pregnant from endeavouring to become pregnant. The model therefore assumes that the intervention is aimed at teenagers who want to prevent pregnancy.

Outcomes

The outcomes were reported in terms of the output of a cost–benefit or return on investment (ROI) analysis, whereby both costs and benefits are measured in monetary units. The ‘benefits’ were the cost savings that result from averted pregnancies. The ‘cost’ was the total amount of public money currently being spent for preventing pregnancies and STIs among young people.

Perspective

The analysis took a public sector perspective, which included costs incurred by the public sector. The analysis was presented with and without the inclusion of government-funded benefits. We reported aggregated costs by public sector (education, health care, welfare).

Modelling the number of pregnancies

To estimate the number of pregnancies averted, we used a mathematical model to define how different types of contraceptive methods were translated into cases of pregnancy:

Y = N f ( g c o n K c o n + g p i l K p i l + g e m e K e m e + g i n j K i n j + g d i a K d i a + g i m p K i m p + g i u d K i u d + g w i t K w i t + g n o c L )

The total number of pregnancies (Y) depended on the number of sexually active females in each group of the study (N_f), the probability of becoming pregnant within 1 year without contraception (L), the percentage of students using condoms (g_con), pill/patch/ring (g_pil), emergency pill (g_eme), injection (g_inj), diaphragm/cap/spermicide (g_dia), implant (g_imp), intrauterine device (g_iud), withdrawal (g_wit) or no contraception (g_noc), and the contraceptive failure rates for each contraceptive class (K_con, K_pil, K_eme, K_inj, K_dia, K_imp, K_iud, K_wit). Evidence on the failure rates was taken from the ROI analysis of improving access to contraception in the UK. 165 The failure rate was defined as the percentage of women experiencing unintended pregnancy within 1 year of typical use of that contraceptive method. The rates of contraception use were taken from responses to the follow-up questionnaire in the If I Were Jack trial. The impact that social disadvantage may have on the number of unplanned pregnancies was accounted for based on evidence from the third National Survey of Sexual Attitudes and Lifestyles (NATSAL)166 and the proportion of pupils receiving FSM, collected from the trial. 167 A sensitivity analysis was conducted to assess the impact that variations in the area-level deprivation effect and failure rates of the contraceptive methods may have on the final results.

The proportion of sexually active males and females aged < 16 years was estimated using the answers to the HBSC surveys for England,168 Wales169 and Scotland170 and the Young Life and Times171 (YLT) survey for NI. We took this frequency to be an approximation of the proportion of sexually active females and males in the target population in each country. We used National Office of Statistics (NOS) data about the population estimates of females and number of births by age in 2017 in England, Wales, Scotland and NI to obtain the total number of sexually active females in the target population in each country. 16,172–174 We then combined these data with the evidence from the answers to the NATSAL-3 survey166 to obtain estimates of the number of sexually active females in three different age groups: 11–13 years, 14 years and 15 years.

There were no directly measured data for the probability of becoming pregnant within 1 year without contraceptive use in the UK. For the US, evidence from the published literature was available to first calculate the 1-month pregnancy probability and convert this into a 1-year probability. 175 This evidence was adapted in the literature using data from the HBSC surveys for England, Wales and Scotland and the 2011 YLT survey for NI. We used the available evidence on the conceptions and population of females, baseline condom use and annual condom failure rate to derive an estimate of L for each country in the UK among young people aged < 16 years. 16,168–173,176 We included in the model different types of outcomes associated with the number of pregnancies averted using the structure shown in Figure 6.

FIGURE 6.

Model structure for the outcome associated with the number of pregnancies averted.

The probability of becoming pregnant in the target population was expressed, following conception, in terms of the probability of females giving birth, terminating the pregnancy, miscarrying or having an ectopic pregnancy or a stillbirth. The probability of birth, abortion, miscarriage/ectopic pregnancy/stillbirth was based on national government statistics for England, Wales and Scotland. 16,170–174 For NI, the model uses the information about the number of abortions in England and Wales for NI residents. 177 The probability of miscarriage/ectopic pregnancy/stillbirth is based on Hospital Episode Statistics (HES) data collected by the Information Centre for Health and Social Care. 178

Results

Long-term decision model

Over 20 years, per 100,000 young people receiving If I Were Jack compared with those receiving standard RSE, If I Were Jack resulted in 379 fewer unintended pregnancies, 680 fewer STIs and 10 additional QALYs at a cost saving of £969,876 (excluding government-funded benefits), or a cost saving of £9.70 per young person who received the intervention (Table 30). Table 31 reports the overall and treatment-specific base-case discounted (at 3.5%) cost estimates from the model disaggregated into the different cost components, and Figure 7 shows the total number of unintended pregnancies, split between number of abortions, miscarriage and births, and the total number of STIs, by STI type. The majority of cost savings, excluding benefits, were due to costs associated with birth avoided. This is as a result of the reduction in unintended pregnancies.

TABLE 30 - Base-case analysis results from the decision model associated with unintended pregnancies, STIs, QALYs loss and total costs, either including or excluding state benefits and lost earnings

Results	Control group	Intervention group	Averted outcomes/incremental costs
Unintended pregnancies, n	2531	2152	379
STIs, n	1853	1173	680
Discounted
QALYs loss	28	18	10
Total costs, £ (without benefits)	£20,459,742	£19,470,336	–£989,406
Total costs, £ (with benefits and lost earnings)	£191,457,170	£166,111,565	–£25,345,605

TABLE 31 - Cost components of the base-case analysis from the decision model (discounted)

Cost components	Control (£)	Intervention (£)	Incremental (£)
Contraception costs	4,832,457	5,982,093	1,145,703
Pregnancy costs	600,064	596,131	–3933
Abortion costs	987,846	842,759	–145,087
Miscarriage/ectopic pregnancy/stillbirth costs	99,396	81,888	–17,507
STI treatment	270,122	172,520	–97,602
Birth costs	13,669,857	11,794,945	–1,874,912
State benefits and lost earnings	170,997,429	146,641,228	–24,356,201

FIGURE 7.

Total number of teenage (< 16 years) unintended pregnancies and STIs per 100,000 young people over a time horizon of 20 years for standard RSE compared with the If I Were Jack intervention. (a) Unintended pregnancies; (b) STIs.

The largest gap was observed in the number of unintended pregnancies that resulted in abortions (1447 in the control group vs. 1229 in the intervention group), followed by births (644 in the control group vs. 559 in the intervention group) and miscarriages (272 in the control group vs. 220 in the intervention group). Similar trends were observed for most types of STIs, with the number of chlamydia cases being associated with the largest discrepancy between the two interventions (1090 in the control group vs. 635 in the intervention group), followed by genital warts (363 in the control group vs. 227 in the intervention group). Relatively small differences were observed in the cases of gonorrhoea (165 in the control group vs. 153 in the intervention group) and PID (females only; 94 in the control group vs. 80 in the intervention group), and no substantial difference was observed in the number of HIV cases (around 1 for both groups).

When nation-specific values were included in the model, including country-specific effects from the trial, the largest cost savings and greatest benefits were seen in England (see Appendix 2, Table 45).

Probabilistic sensitivity analysis

Tables 32 and 33 show the results from the PSA, in which the model parameters were randomly varied by drawing their values from associated probability distributions (see Appendix 2, Table 44, for detailed information about the choice of the distributions and parameter variations). In Table 32, 95% bootstrapped CIs for the total number of unintended pregnancies, STIs and QALYs and associated costs (either excluding or including government-funded benefits) by treatment group are reported alongside estimates obtained in the base-case analysis. In addition, 95% bootstrapped CIs obtained under the PSA for the averted results for each type of outcome and incremental costs between the two groups are reported and compared with corresponding base-case estimates. In Table 33, total cost results obtained under the PSA reported by type of cost component, namely the costs related to contraception, pregnancy, abortion, miscarriage/ectopic pregnancy/stillbirth, STI treatment, birth, and government-funded benefits and lost earnings. For a £20,000 and £30,000 cost-effectiveness threshold per QALY gained there is, respectively, a 96% and 99% probability that the If I Were Jack intervention is cost-effective compared with standard RSE.

TABLE 32 - Estimates (from the base-case analysis) and 95% bootstrapped CIs (from the PSA) based on 1000 iterations, including the number of outcomes (unintended pregnancies, STIs or QALYs) and total costs associated with each group, as well as the averted outcome and incremental cost results between the If I Were Jack intervention and standard RSE. Costs were calculated either excluding (without) or including (with) government-funded benefits and lost earnings

Outcome	n (95% CI)		Total costs without benefits, £ (95% CI)		Total costs with benefits, £ (95% CI)
	Control	Intervention	Control	Intervention	Control	Intervention
Unintended pregnancies	2,531 (2461 to 3100)	2152 (2078 to 2716)	20,459,742 (18,687,125 to 24,591,145)	19,470,336 (17,003,319 to 22,175,576)	191,457,170 (140,548,595 to 217,430,076)	166,111,565 (134,884,491 to 171,019,049)
STIs	1853 (1106 to 3400)	1173 (412 to 2467)
QALYs loss	28 (19 to 45)	18 (14 to 30)
Averted outcomes/incremental costs
Unintended pregnancies	379 (231 to 477)		–989,406 (–1,559,545 to –382,949)		–25,345,605 (–47,857,741 to 5,438,251)
STIs	680 (189 to 1467)
QALYs	10 (5 to 16)

TABLE 33 - Cost estimates (from the base-case analysis) and 95% bootstrapped CIs (from the PSA) based on 1000 iterations from the decision model disaggregated into seven components: contraception, pregnancy, abortion, miscarriage/ectopic pregnancy/stillbirth, STI treatment, birth, government-funded benefits and lost earnings. For each component, discounted cost estimates associated with the control, the intervention and the incremental results are displayed

These costs were calculated by adding up the costs associated with the application and obtainment of government-funded benefits for young mothers as well as state costs aimed at supporting the development of their children. These costs included costs for child health-care, costs of child school education, costs of child benefit, costs of child tax credit, costs of housing benefits and costs of income support. It also includes the costs of lost earnings.

Cost components	Control, estimate (£, 95% CI)	Intervention, estimate (£, 95% CI)	Incremental, estimate (£, 95% CI)
Discounted
Contraception costs	4,832,457 (4,645,046 to 5,045,220)	5,982,093 (5,791,743 to 6,190,293)	1,145,703 (1,126,841 to 1,216,578)
Pregnancy costs	600,064 (506,366 to 716,344)	596,131 (480,238 to 703,678)	–3933 (–50,181 to 42,480)
Abortion costs	987,846 (825,467 to 1,139,093)	842,759 (747,888 to 947,512)	–145,087 (–214,646 to –60,854)
Miscarriage/ectopic pregnancy/stillbirth costs	99,396 (73,811 to 141,232)	81,888 (54,695 to 104,823)	–17,507 (–36,428 to –2,020)
STI treatment	270,122 (149,359 to 693,619)	172,520 (132,954 to 486,693)	–97,602 (–207,776 to 150)
Birth costs	13,885,602 (11,576,613 to 16,041,941)	11,928,685 (9,454,455 to 13,888,264)	–1,956,917 (–3,299,170 to –413,975)
Government-funded benefits and lost earningsa	170,997,429 (156,208,678 to 183,834,114)	146,641,228 (140,286,029 to 153,431,931)	–24,356,201 (–32,711,305 to –15,922,648)

Incremental results obtained from the PSA, quantifying the impact of sampling uncertainty on base-case estimates, are graphically summarised in Figure 8 using separate cost-effectiveness planes for each type of outcome: pregnancies averted, STIs averted and QALYs gained. The majority of iterations (97.5% for pregnancies, 96.6% for STIs and 95.7% for QALYs) were located in the south-east quadrant as the If I Were Jack intervention resulted in better outcomes for a lower cost.

Probabilistic sensitivity analysis: cost-effectiveness planes

FIGURE 8.

Cost-effectiveness planes for the expected number of averted outcomes (unintended pregnancies, STIs and QALY losses) and the total costs increment (health service cost perspective) between the two treatment groups. Estimates are obtained based on a total of 1000 iterations for the PSA.

Primary and secondary results

The primary analysis found no significant differences between the intervention and control groups in relation to self-reported avoidance of unprotected sex, measured as a combination of sexual abstinence or use of reliable contraception at last sex. However, a post hoc exploratory analysis of these questions separately showed that the intervention was effective in increasing the use of reliable contraception at last sex for those who already were, or became, sexually active by the time of follow-up. There were no significant differences in rates of sexual abstinence between the intervention or control groups at the 12- to 14-month follow-up.

The If I Were Jack intervention led to significant improvements in a number of important secondary outcomes relating to our theory of change, notably knowledge, attitudes towards male gender roles and intentions to avoid an unintended pregnancy. There were also small and statistically non-significant improvements in sexual self-efficacy and comfort communicating about avoiding unintended pregnancy.

Significant improvements in knowledge included improved knowledge of reliable contraceptive methods, how to access contraception and the legal age of consent. Students in intervention schools had stronger intentions to avoid UTP. This included stronger intentions to use reliable contraception, avoid peer pressure to have sexual intercourse, not to have sex until ready and be prepared to share responsibility for contraception with a partner.

An additional planned sociodemographic subgroup analysis showed no significant differences between males and females in relation to the primary outcome. There were also no significant differences in relation to ethnicity, nation of the UK where delivery took place or socioeconomic status, as measured by the family affluence scale.

Intervention cost and cost-effectiveness

The costs for If I Were Jack fall within the ‘very low cost’ category (< £80 per student per year) for school-based interventions according to the Education Endowment Foundation guidance. 205 Taking account of staff time for training, lesson planning and delivery, in addition to health resource use, the If I Were Jack intervention cost less than £3 more per student than standard RSE.

The health economic analysis showed that If Were Jack is likely to be cost-effective when projected over a 20-year time horizon. Those in the intervention group were more likely to use reliable forms of contraception at last sex. This observed difference showed that, compared with standard RSE, the If I Were Jack intervention is likely to be cost-effective, as it would result in 379 (95% CI 231 to 477) fewer unintended pregnancies, 680 (95% CI 189 to 1647) fewer STIs and 10 (95% CI 5 to 16) QALYs gained per 100,000 young people for a cost saving of £9.89 (95% CI £3.83 to £15.60) per young person who receives it, with the majority of the cost savings related to health care. Approximately 96% of iterations are in the south-east quadrant of the cost-effectiveness plane, in that If I Were Jack results in better outcomes for a lower cost.

Process evaluation

The following issues were addressed: reasons for school participation; intervention delivery and fidelity in the context of overall RSE provision in intervention schools; RSE provision in control schools and potential contamination caused by any changes to provision that could be due to participation in the trial; and perceptions of effectiveness, especially relating to perceptions of the mechanisms of effectiveness at play in the intervention among students, teachers and school principals/head teachers.

Interpretation of the study findings in the light of previous research

Overall, the results relating to the primary and secondary outcomes are consistent with two systematic reviews of systematic reviews of school-based RSE in high-income countries. 207,208 These reviews of reviews indicate that, overall, school-based RSE interventions demonstrate effectiveness in enhancing knowledge, skills and intentions, but do not demonstrate efficacy in relation to biological outcomes such as reducing adolescent pregnancy and HIV, or behavioural outcomes such as reductions in unprotected sex. While emphasising the importance of the psychosocial outcomes, these reviews also suggest that the absence of a clear pattern of effectiveness in relation to biological and behavioural outcomes is because rates of engagement in sexual intercourse and rates of HIV and pregnancy are low among adolescents aged < 16 years in high-income countries. Denford et al. 196 suggest that interventions could nevertheless be generating positive changes in sexual behaviours that are not yet manifest. The positive effects observed for the psychosocial secondary outcomes in our study (i.e. knowledge, attitudes towards male gender norms and intentions to avoid unintended pregnancy) support this argument, as these outcomes relate to mechanisms that research28,56–61,63,94 shows are linked to a reduction in risk-taking behaviour once young people are sexually active. Consistent with this theory of change, we found that evidence of positive behavioural change to avoid unprotected sex through use of reliable contraception was manifest among those who were sexually active (at baseline) or became sexually active (at follow-up). This suggests that the intervention was having a positive effect on one of the key behavioural changes of the intervention logic model, namely sexual readiness, with evidence that young people in the intervention group were more adequately prepared for this transition in life, whereas there was no evidence that the intervention increased rates of sexual initiation among young people. This finding in the present study is important, as previous high-quality systematic reviews31,209 report that few school-based interventions in high-income countries are effective in increasing contraceptive use or condom use at last sexual intercourse among young people who are sexually active, and none of the previous UK-based randomised trials of RSE interventions had demonstrated effectiveness on this outcome. 148 A US-based study demonstrated the effectiveness of an after-schools-based intervention (CAS-Carrera)70 in reducing unprotected sex at last sex among those who were sexually active. However, this study found significant effects for females only, whereas our study found that the intervention was equally effective for males and females who were sexually active, suggesting the importance of male engagement.

The positive impact on gender role attitudes with a significant shift in the intervention group towards more gender-equitable roles is consistent with Haberland and Rogow’s86 systematic review, which reported that HIV and sexuality programmes that addressed gender or power were more effective than programmes that did not. Gender inequity is widely regarded in the literature as contributing especially to deficits in girls’ SRHR but also to inequalities among boys. 75,84,85 In some studies, gender attitudes are measured among males using only the gender-equitable men’s scale. 210 It is clear from the qualitative data that the gender focus of the If I Were Jack intervention was explicit to participants. Specifically, the process evaluation results provide evidence that the gendered focus on generating critical gender awareness and male responsibility was seen as being one of the most innovative, stimulating and enjoyable elements of the intervention. In the trial results, significant positive changes in the intervention group were identified in relation to intentions to share responsibility (among males and females) to use contraception to prevent unintended pregnancy.

Our finding of significant improvements in young people’s intentions to avoid unintended pregnancy, which includes their perceived ability to share responsibility for contraceptive decision-making with a partner and communicate consent for sexual intercourse, sexual readiness and sexual preferences regarding timing, are important impacts because they address a neglected research area of considering young people’s sexual desires and preferences as part of healthy relationships in RSE. 208 Although RSE specialists have called for the inclusion of a discourse of pleasure to enhance a more holistic view of sexual well-being and other aspects of positive sexuality,211,212 the inclusion of the measurements of these outcomes are ‘conspicuous by their absence’ in randomised trials of sex education. 208

The process evaluation provided valuable insights on what RSE is wanted and what is needed and implementable in the classroom, including in faith-based schools in the UK. The JACK trial confirms that young people want comprehensive, positive, inclusive and skills-based learning to enhance their ability to develop healthy, positive relationships throughout their lives, delivered using interactive, participatory, learner-centred or critical thinking approaches. 130,208,213–217 The results also confirm teachers’ need for high-quality, evidence-informed RSE resource materials with manualised programming that are implementable in the classroom and are designed to enable participatory and critical learning. 217–220 Adding to this, both student and teacher perspectives emphasise the need to enhance teacher training and the skill set of teachers in delivering school-based RSE.

Strengths and limitations of the study

We undertook a well-conducted, large cluster RCT of an intervention proven feasibility, which was developed with very substantial user co-design. During this study, and prior to trial commencement, we undertook a further optimisation of the intervention to enhance its cultural salience across the four nations of the UK through PPI. We brought together a YPAG from the four nations of the UK in a weekend residential event to advise on all aspects of the production of two new IVDs, one using actors from NI (for use in NI and Scotland) and one using actors from England (for use in England and Wales). Further consultation on all aspects of the intervention and key aspects of study design were continued with the YPAG and a RSE expert stakeholders group, again from across all of the UK, as well as PPI membership of TSC throughout the trial. RSE experts’ feedback suggested that the opportunity to learn from other RSE experts across the whole of the UK, as well as invited international experts from WHO and Rutgers, Netherlands, was a key dividend of their partnership on this trial. 221

The trial was undertaken in a group of schools that were ethnically, culturally, and socioeconomically diverse. Uniquely, in a trial of a RSE intervention, we included schools in the four nations of the UK and included schools that were faith-based (although they were all Christian) as well as those that were not. It is also one of the few trials that has disaggregated the results by sex146 and included an analysis of how the intervention might promote more equitable gender relationships.

Participating schools were representative of the 263 schools initially approached and only four schools were lost to follow-up, two of which were a result of school closures during the COVID-19 pandemic lockdown measures. Our follow-up was lengthy to allow time both for intervention effects to develop and to capture the largest number of students before the school leaving age of 16 years. Although a longer follow-up period would have afforded a greater possibility of capturing intervention effects, especially once students reach the legal age of sexual consent (16 years in the UK), it is very difficult to collect data once students leave school. (In the UK, students may leave school at age 16 or continue until age 18 years.) The follow-up period of 12–14 months also allowed for investigation of persistence of intervention effects after the end of the facilitated intervention. Student participation was high and parental withdrawal of students was low. Our outcome research team and intervention team remained independent throughout the trial, and blinding of lead researchers was maintained. Outcomes were assessed using age-appropriate validated instruments. Although self-report outcomes can be open to recall bias, baseline data were collected before randomisation, instruments were used with standardised recall periods and actions at the school level are very unlikely to have biased reporting between intervention and control groups. Although some studies222 have included age of sexual debut as a secondary outcome, owing to concerns expressed by the reviewing research ethics committee relating to responsibilities to report sexual intercourse under the age of 13 years in the UK, age at sexual debut was not included.

Our data are subject to a number of limitations. Our trial was carried out in a randomly selected sample of the whole of NI, the south of Wales, urban and peri-urban areas of the south-east of England and central Scotland only. Our process evaluation did not identify any factors that might suggest that implementation or effects would be different in schools in other areas, given the level of diversity included, but we do not claim that the findings are generalised to all parts of the UK.

The ICC in the current study was much larger (0.12) than expected (0.01), based on our feasibility study44 and previous research in this area. 18,148 The sample size was also reduced as a result of losses to follow-up, including the loss of two schools due to COVID-19-related closures. Therefore, the study may have been underpowered. Although missing data can introduce bias, our data were analysed using an intention-to-treat approach, and sensitivity analyses to account for missing data led to little or no change in our conclusions. Missing data can probably be attributed to the sensitivity of sex research among any population, but especially among young people who are not yet at the legal age of sexual consent.

The large number of secondary outcomes investigated necessitated multiple statistical testing. However, to mitigate this, we tested only pre-specified secondary outcomes and subgroup analyses (which were reported using 99% CIs). The subgroup analyses were likely to have been underpowered. Although providing the most useful data for the majority of outcomes, student self-report is not without limitations. Specifically, it is not feasible to validate young people’s self-report of sexual activity with an objective measure, and self-reports therefore offer the best method of assessing sexual activity. 223 However, as is standard in measures of sexual activity, we provided definitions for sexual activities and terminology to aid comprehension, and student questionnaires were administered under exam conditions to enhance confidentiality of responses. 146 All the secondary outcomes were based on validated measures.

Our process evaluation was both broad, in collecting data from all schools, and deep, in exploring some processes in randomly selected case-study intervention schools in each of the four nations alongside a SEQ in all intervention schools. Although response rates for process data collection were generally high, there were two areas of data collection, the teacher implementation logs capturing delivery of implementation and surveys of parents, that had lower response rates. In relation to TILs, our triangulated data collection approach (of interviews with students, teachers and trial champions) meant that we were still able to assess acceptability and fidelity of the intervention in most schools.

The economic evaluation also had some limitations. The costs and outcomes associated with school-based public health interventions fall across multiple government bodies. 224 Outcomes such as QALYs, which are generally recommended for economic evaluations of health-care interventions in the UK, may not be the most relevant outcome of interest when the costs fall on the education sector. To address this limitation, we reported on a range of different outcomes in our analysis. Costs are reported separately to allow different decision-makers to determine the relative cost-effectiveness of the intervention on health, education and social welfare. We have included government-funded benefits, sometimes referred to as ‘transfer costs’, as, although they are complex to include and it is generally recommended that they be excluded from economic evaluations as they are cost neutral to society,225 they are a key component of the cost of unintended pregnancies and have been included in previous165 ROI analyses of contraception effectiveness. They are also likely to be of interest to some decision-makers who might consider keeping government-funded benefits to a minimum, an important political outcome. Finally, a key outcome that we are interested in, unintended pregnancy, is a rare outcome in the target age group, with costs that occur over a longer time horizon. To address this, we used decision modelling to extrapolate the hypothesised impact of reducing instances of unprotected sex on unintended pregnancies. One of the shortcomings of this approach is that the model was based on a range of assumptions. However, we attempted to address this by subjecting the model to a range of probabilistic and deterministic sensitivity tests.

Conclusion

The need for RSE to engage with young men to promote positive sexual health for all and challenge the gender inequalities that underlie young women’s generally poorer sexual health outcomes (especially in relation to sexual violence, adolescent pregnancy and STIs) is widely endorsed by the WHO83 and UNESCO33 among others. 75,84,85 To our knowledge, the JACK trial is the first randomised trial, health economic and process evaluation of a school-based RSE intervention using a gender-transformative approach to especially engage young men, as well as young women, and promote joint responsibility in preventing teenage pregnancy (by delaying sex until ready or through use of effective contraceptives) and promoting positive sexual health and relationships. To our knowledge, the JACK trial is also the first UK-wide trial of school-based RSE in which it was demonstrated that it was possible to recruit and retain faith-based schools in a trial of a comprehensive RSE. Although the intervention did not reduce the rate of unprotected sex overall (through a combination of either sexual abstinence or use of reliable contraception), positive effects were observed in the sexually active population at follow-up, and especially among those whose sexual debut was early (sexually active at baseline, median age 14 years), as well as for those who became sexually active by the follow-up time point. These results are particularly promising, as they indicate that this relatively low dose intervention is most effective for groups at higher risk of unintended pregnancy owing to earlier sexual debut and is likely to reflect the particular relevance of contraceptive use in those who already were, or became, sexually active by the 12- to 14-month follow-up. The intervention therefore showed evidence of having an impact one of its key behavioural outcomes, that of sexual readiness, while not increasing rates of sexual initiation. Furthermore, If I Were Jack was effective in equipping all young people (whether sexually active or not) with the knowledge, attitudes and intentions required for safe and pleasurable relationships now and in the future.

The health economic analysis showed that, based on a 20-year time horizon, the If I Were Jack intervention is likely to be cost-effective, as it would result in fewer unintended pregnancies and STIs and a QALY gain for a cost saving of £9.89 per young person who receives the If I Were Jack intervention compared with standard RSE, with the majority of the cost savings related to health care. It is a low-dose, low-cost intervention with the cost per student falling well below the Educational Endowment Fund threshold for ‘very low-cost’ interventions. The school-based method of delivery in If I Were Jack provides a universal and efficient way of promoting sexual health and pregnancy prevention owing to the potential population-level reach, and school delivery providing as a sustainable platform for delivery.

Implications for decision-makers and practice

The year 2020 marked the 25th anniversary of the Cairo International Conference on Population and Development (ICPD) and Beijing Platform for Action Women’s Rights, which fundamentally shifted thinking on sexual and reproductive health towards the need to address gender inequalities that disproportionately affect girls and women’s sexual and reproductive health. Equally, it drew attention to the need to engage men and boys in addressing gender inequalities and to engage with men and boys as reproductive agents in their own right. However, the lack of intentional male engagement along with a focus on addressing gender inequalities within RSE, a key starting place in achieving these aims, has been highlighted as an important gap by UNESCO,226 the WHO,83 United Nations Population Fund84 and the European Society of Experts on Sex Education in their International Guidance on Sexuality Education,212 and again noted in the latest systematic review of reviews of RSE programme evaluations. 208

Closer to home, in 2021, a governmental agency [Office for Standards in Education, Children’s Services and Skills (Ofsted)] inspection of schools in England227 illuminated the gendered nature of deficits in sexual and reproductive health. The report found that 60% of young women in schools experienced contact forms of sexual harassment, compared with 25% of males. In relation to non-contact forms of sexual harassment, such as unwanted or inappropriate sexual comments, the report found that 90% of young women in schools experienced this, compared with 75% of males.

The Ofsted report227 recommends comprehensive RSE as a fundamental building block to addressing this problem in schools and specifically recommends improved teacher training in RSE. However, an important lesson from the JACK trial to add to this recommendation, and which builds on decades of practitioner learning encompassed in the ICPD conference and Beijing Platform for Women’s Rights resolutions, is that RSE programming should not be gender neutral and assume it will equally engage with males as well as females, or gender blind in relation to addressing gender inequalities. 228

The problems highlighted by the Ofsted report227 illuminate the urgent need to address problematic masculinities in school cultures in high-income countries such as the UK. To be successful in doing so, RSE should intentionally and positively engage with young men as well as young women in addressing gender inequalities in sexual and reproductive health. As one young female pupil in the Ofsted report was quoted as saying: ‘it should not be our responsibility to educate boys’ (contains public sector information licensed under the Open Government Licence v3.0).

The JACK trial provides practitioners and programmers with an acceptable, feasible and cost-effective gender-transformative programme. This programme invites young people to engage in young men’s perspectives while equally inviting young people to challenge the unequal gender norms associated with male sexual desire and female reproductive responsibility. It encourages communication and behavioural skills among young men and young women to prevent a teenage pregnancy and to know how to seek help. If I Were Jack acknowledges sexual pleasure and sexual intimacy in young people’s lives and asks young people to consider for themselves the balances between sexual pleasure and sexual responsibility in a gender-equitable manner. In addition, the programme seeks to address deficits in sex education for young men, particularly with respect to teenage pregnancy, identified in the scientific literature, which disadvantages adolescent men as well as adolescent women. 28,42,46,79,87–92

If I Were Jack is, however, a brief intervention of a low dose not designed to address all RSE needs. Although this study has demonstrated the added value of even such a brief intervention, in terms of increased knowledge and gender-equitable attitudes among young people for healthy and positive intimate relationships as well as a reduction of unprotected sex among young people who are already sexually active, perhaps the greater added value is the rigorous testing of intentional male engagement and gender-transformative components, which could be incorporated more broadly into RSE programming. Recognising that access to RSE is a human right under United Nations Rights of the Child124 and optimally delivered as other subjects are, over the whole school year across all year groups, in an age-appropriate manner,83,85,212,226 arguably the added value to advancing RSE practice in this trial is in demonstrating how and why male engagement and gender-transformative programming is important. 228

We recognise too, however, that more work could be done to enhance the gender-transformative components of the If I Were Jack intervention. Notably, the intervention could also be enhanced through components that challenge hegemonic heterosexuality to more holistically address gender inequalities that relate to LGBTQ+ young people as part of RSE and provide a broader range of knowledge, skills and access to services. Nonetheless, this study could provide an important reference point for practitioners to incorporate intentional engagement of young men and gender-transformative components in RSE that could be extended in new ways.

Implications for research

Although our study included schools with diverse socioeconomic characteristics, the study was conducted within the context of a high-income country and is not necessarily generalisable outside this context. We have plans under way to develop and trial adaptations of the If I Were Jack intervention in South America and Southern Africa, taking learning from the current study into account. We would also recommend that the feasibility and effectiveness of gender-transformative interventions should be explored further in trials across a range of diverse contexts and, in particular, low- and middle-income countries. In addition, given the promising results of this brief, gender-transformative intervention, designed to especially engage with males, we would recommend research that could test the integration of the intervention components, or similar components, as part of broader comprehensive RSE curricula. It is important too that we extend research on RSE beyond schools to research how best to meet the needs of those who have missed out on this education. Members of the team have co-designed and implemented an RSE intervention based on If I Were Jack, specifically for young offenders in custodial settings in Northern Ireland and Scotland, with promising results so far. 229

Although our study shows that it is feasible to recruit and maintain faith-based schools to a RSE intervention, all of the faith-based schools in the current study were Christian, which limits the generalisability of the study to non-Christian faith-based schools. Therefore, we would also recommend that future studies seek to recruit a diverse range of faith-based schools alongside non-faith-based schools (where applicable) to better understand the feasibility, acceptability and effectiveness of school-based RSE interventions in faith-based schools. We suggest (depending on context and prevalence of faith-based schools within each context) that this can be best achieved through active engagement with faith-based schools during the design or adaptation of the intervention to achieve early dialogue on acceptability.

We did not undertake any data linkage as part of the current study, which means that it is not possible to assess the longer-term impacts of the If I Were Jack intervention on the study outcomes or in relation to rates of unintended pregnancies, as such data are not routinely collected in NI. We recommend that future studies explore the feasibility of anonymous data linkage with routine data (if available) to explore the longer-term effects of interventions.

As highlighted by Oringanje et al. (2016),31 and as evidenced in the literature, there is a lot of variation in the way some behavioural outcomes related to sexual activity are reported in RSE intervention studies. This can make it difficult to estimate ICCs, and thus the required sample size for cluster RCTs, in this area. The ICC in this study was much larger (0.12) than expected (0.01), despite the estimates being based on our own feasibility study27 and previous research in this area. 28,29 In addition, few RCTs of RSE interventions pre-specify primary and secondary outcomes, opening the door to the possibility of outcome switching, hampering the quality of sample size calculations and study interpretation in the field. We have highlighted this with the hope of informing other researchers in planning and designing future studies.

The use of TILs to capture teachers’ implementation of the programme was not as successful in the current study as we had hoped. Given the importance of implementation data to the process evaluation, we would recommend that there should be a distinct financial incentive (for example a £100 voucher) built into trials for collecting this information from individual teachers in each school. This would offset the low response rate by individual teachers and the disproportionate effort by the research team in trying to collect these responses.

The low parental participation observed in our study is a common finding in the experimental research literature on RSE. 146 However, the results of the process evaluation suggest that a significant proportion of parents may not have received programme materials from their child’s school, which makes it difficult to draw definite conclusions about parental engagement in the current study. We recommend that future studies include training for teachers and school staff in relation to building confidence to engage with parents around RSE. Future studies might also consider closer liaison with schools to ensure that information is communicated to parents to invite their participation in the research. Although the use of digital media to engage parents in this study did increase participation when compared with our earlier feasibility trial,48 there is a need to further explore opportunities and mechanisms to improve parental engagement in RSE.

Further work is required to evaluate the suitability of different health economic outcomes for inclusion in economic evaluations of sexual health interventions aimed at young people. Extrapolating outcomes over longer-time horizons will capture a wider range of cost and outcomes, but is subject to a wide range of uncertainty.

Ethics approval

Ethics approval for the study was granted by the Research Ethics Committee of the School of Nursing and Midwifery, Queen’s University Belfast in July 2017 (reference number 11.MLohan.05.17.M6.V1).

This trial was registered with the ISRCTN (reference number 99459996).

Funding

Queen’s University Belfast agreed to act as sponsor for the research.

Contributions of authors

Maria Lohan (https://orcid.org/0000-0003-3525-1283) (Professor of Social Science and Health) was the chief investigator. She conceptualised the study and led design of the study, the research and writing of the manuscript. She conducted the trial’s progress review, including staff and resources, and ensured that the trial was administered in a financially responsible manner.

Kathryn Gillespie (https://orcid.org/0000-0001-5272-392X) (Research Fellow) was the lead researcher on the process evaluation data analysis and write-up. She contributed to data collection and to trial data interpretation and write-up.

Emily Warren (https://orcid.org/0000-0002-9117-844X), Ruth Lewis (https://orcid.org/0000-0002-6768-6188), Kelly Buckley (https://orcid.org/0000-0002-8862-3776), Theresa McShane (https://orcid.org/0000-0003-1284-1272) and Linda Adara (https://orcid.org/0000-0002-0015-3942) (Research Fellows) led school recruitment and study data collection, and contributed to process evaluation study design, data analysis and writing of the manuscript.

Áine Aventin (https://orcid.org/0000-0001-5849-0506) (Lecturer, Sexual and Reproductive Health, Co-investigator) was trial manager (year 1) and had day-to-day responsibility for the conduct of the trial and the operations of the research team in year 1. She also contributed to the conceptualisation and design of the study, optimisation of the intervention, process evaluation data analysis and writing of the manuscript.

Aisling Gough (https://orcid.org/0000-0002-4201-7316) (Research Fellow) and Susan Lagdon (https://orcid.org/0000-0003-0477-2297) [Lecturer, Psychology (Mental Health)] were trial managers during years 3 and 4 and year 2, respectively. During this time they had day-to-day responsibility for the conduct of the trial and the operations of the research team. They contributed to data collection and writing of the manuscript.

Aoibheann Brennan-Wilson (https://orcid.org/0000-0002-6652-6238) (Research Fellow) was closely involved in interpretation of the trial data and the writing of the manuscript.

Lisa McDaid (https://orcid.org/0000-0002-7711-8723) (Professor of Social Science and Health, Co-investigator) contributed to the conceptualisation and design of the study, led the Scotland study team, contributed to trial and process evaluation data interpretation, and contributed to writing of the manuscript.

Rebecca French (https://orcid.org/0000-0002-1962-5022) (Associate Professor of Sexual and Reproductive Health Research, Co-investigator) contributed to the conceptualisation and design of the study, led the England study team, contributed to trial and process evaluation data interpretation and writing of the manuscript.

Honor Young (https://orcid.org/0000-0003-0664-4002) (Senior Lecturer, Co-investigator) contributed to the conceptualisation and design of the study, led the Wales study team, contributed to the intervention optimisation, trial and process evaluation data interpretation, and writing of the manuscript.

Clíona McDowell (https://orcid.org/0000-0002-7644-7197) (Head of Statistics, NI Clinical Trials Unit, Co-investigator) led on the design of the statistical analysis plan and statistical analysis and contributed to the writing of the manuscript.

Danielle Logan (https://orcid.org/0000-0002-3932-3654) (Biostatistician) and Sorcha Toase (https://orcid.org/0000-0003-1799-5588) (Biostatistician) contributed to statistical analysis and to writing of the manuscript.

Rachael M Hunter (https://orcid.org/0000-0002-7447-8934) (Associate Professor – Health Economics, Head of the Health Economics Analysis and Research Methods Team, Priment Clinical Trials Unit, University College London, Co-investigator) and Andrea Gabrio (https://orcid.org/0000-0002-7650-4534) (Assistant Professor in Statistics) designed and conducted the health economic analysis.

Mike Clarke (https://orcid.org/0000-0002-2926-7257) (Professor and Director of Northern Ireland Clinical Trials Unit, Co-investigator) contributed to the conceptualisation and design of the study and trial data interpretation, and contributed to the writing of the manuscript.

Liam O’Hare (https://orcid.org/0000-0002-4453-2880) (Principal Research Fellow, Co-investigator) contributed to conceptualisation and design of the study and to the trial and process evaluation data interpretation, and contributed to the writing of the manuscript.

Chris Bonell (https://orcid.org/0000-0002-6253-6498) (Professor, Public Health Sociology, Co-investigator) contributed to the conceptualisation and design of the study and trial data interpretation, and contributed to the writing of the manuscript.

Julia V Bailey (https://orcid.org/0000-0002-5001-0122) (Associate Professor in Primary Care, Co-investigator) contributed to the conceptualisation and design of the study and intervention optimisation and to the writing of the manuscript.

James White (https://orcid.org/0000-0001-8371-8453) (Reader in Population Health, Co-investigator) contributed to the conceptualisation and design of the study, development of sample size calculation and recruitment strategy, statistical analysis plan, trial data interpretation and writing of the manuscript.

All co-authors (except Aisling Gough, who was unavailable) approved final manuscript.

Publications

Lohan M. Engaging Boys and Addressing Masculinities in Comprehensive Sex Education Reaps Benefits for Both Adolescent Boys and Girls: Results from the JACK Trial. 2022. URL: http://www.srhm.org/news/engaging-boys-and-addressing-masculinities-in-comprehensive-sex-education-reaps-benefits-for-both-adolescent-boys-and-girls-results-from-the-jack-trial/ (accessed 13 December 2022).

Lohan M, Brennan-Wilson A, Hunter R, Gabrio A, McDaid L, Young H, et al. Effects of gender transformative relationships and sexuality education to reduce adolescent pregnancy (the JACK trial): a cluster-randomised trial. Lancet 2022;7:e626–37.

Aventin Á, Gough A, McShane T, Gillespie K, O’Hare L, Young H, et al. Engaging parents in digital sexual and reproductive health education: evidence from the JACK trial. Reprod Health 2020;17:132.

Lohan M, Aventin Á, Clarke M, Curran RM, Maguire L, Hunter R, et al. JACK trial protocol: a phase III multicentre cluster randomised controlled trial of a school-based relationship and sexuality education intervention focusing on young male perspectives. BMJ Open 2018;8:e022128.

Data-sharing statement

All data requests should be submitted to the corresponding author for consideration. Any intention to use or share project data for purposes other than those outlined in the JACK trial project protocol or the JACK trial Data Management Protocol must be approved by the PI and Trial Management Group (consisting of representatives from all partner sites). Access to anonymised data may be granted following review.

Disclaimers

This report presents independent research funded by the National Institute for Health and Care Research (NIHR). The views and opinions expressed by authors in this publication are those of the authors and do not necessarily reflect those of the NHS, the NIHR, the PHR programme or the Department of Health and Social Care. If there are verbatim quotations included in this publication the views and opinions expressed by the interviewees are those of the interviewees and do not necessarily reflect those of the authors, those of the NHS, the NIHR, the PHR programme or the Department of Health and Social Care.

Control school relationship and sexuality education provision assessment tool

Activity 11: If I Were Jack homework survey

Modelling the health consequences of behavioural change

The baseline probability of becoming pregnant without contraception is derived from the available evidence in the literature. The procedure used to identify the base-case value for this parameter is described in the following text.

The proportion of sexually active males and females aged < 16 years was estimated using the answers to the HBSC surveys for England, Wales and Scotland168–170,230 and the YLT survey for NI. 171 We take this frequency to be an approximation of the proportion of sexually active females and males in the target population in each country, which is shown in Table 34.

In addition to these data, we used national official statistics about the population estimates of females and number of births by age in 2017 in England, Wales, Scotland and NI172–174 to obtain the total number of sexually active females in the target population in each country. We then combined these data with the evidence from the answers to the NATSAL-3 survey,230 about the age when respondents (aged 20–24 years) first had vaginal intercourse, to obtain estimates of the number of sexually active females in three different age groups: 11–13 years, 14 years and 15 years. The data to obtain this estimate are shown in Table 35.

There are no directly measured data for the probability of becoming pregnant within 1 year without contraceptive use in the UK. For the US, evidence from the published literature is available to first calculate the 1-month pregnancy probability and convert this into a 1-year probability,175 which may overestimate pregnancy risk. This evidence was adapted in the literature using data from about the HBSC surveys for England, Wales and Scotland168–170,230 and the 2011 YLT survey for NI. 171 We used the available evidence on the conceptions and population of females, baseline condom use and annual condom failure rate to derive an estimate of L for each country in the UK among the teenagers aged < 16 years. 176 Age-specific rates are used to avoid overestimation based on the failure rates of 15-year-old teenagers, whereas 2017 country-specific population data (see Table 35) were used to derive the base-case values for L among teenage females aged < 16 years, which are reported in Table 36.

It is important within any health economic analysis to compare the additional costs and benefits associated with an intervention. A pregnancy may result in a birth, an abortion or a miscarriage/ectopic pregnancy/stillbirth. The consequences of each of these outcomes need to be estimated in terms of costs and benefits. We therefore follow the approach used in the literature176,231 and include in the model the outcomes associated with the number of pregnancies averted.

The probability of birth, abortion, miscarriage/ectopic pregnancy/stillbirth is based on national government statistics for England, Wales and Scotland. 172,174 For NI, since abortion was not legal and official data were not available at the time of writing, the model uses the information about the number of abortions in England and Wales for NI residents. 173 This is likely to be an underestimate of the true value of the number of abortions for NI, and sensitivity analysis will be carried out to assess the impact of variations of this estimate on the results. The probability of miscarriage/ectopic pregnancy/stillbirth is reported by HES data collected by the Information Centre for Health and Social Care. 172–174,178 This suggests that 8.5% of all pregnancies result in miscarriage/ectopic pregnancy/stillbirth; however, this is not separated by age. One study was identified that reports outcomes of pregnancy by age and suggests that about 6% of all pregnancies of women aged < 20 years end in miscarriage, ectopic pregnancy or stillbirth. 232 This analysis was based within Denmark; however, it seems consistent with the UK HES data. It should be noted that these miscarriage statistics are based on the number of hospital stays and not all miscarriages require a hospital stay, meaning that the proportion of pregnancies ending in miscarriage, ectopic pregnancy or stillbirth may be underestimated. The impact of this on the model results is tested within a one-way sensitivity analysis. The probabilities of each outcome following conception are shown in Table 37.

A pregnancy may result in a birth, an abortion, a miscarriage, an ectopic pregnancy or a stillbirth. The consequences of each of these outcomes need to be estimated in terms of costs and benefits. There are numerous negative outcomes typically associated with a teenage pregnancy. First, young age at pregnancy is associated with a greater likelihood of abortion. 172,233,234 Second, if the pregnancy is continued, there might be a greater risk of having a very low-birthweight baby. Finally, there are longer-term negative outcomes for the mother, the father and the child typically associated with a teenage birth. 235,236 These include lower education, poorer employment status and an increased risk of a female child becoming a teenage mother herself.

The evidence from two UK-specific reports about the financial and economic impact of teenage pregnancy suggests that about 43% of teenage mothers have no qualifications. 199,233 Based on this evidence, an estimate was derived about the medium-term negative impacts on earnings in the UK for teenage mothers who fail to gain qualifications. Focusing on vocational qualification, the estimated annual levels of earnings disparity between young workers with qualifications versus young workers with no qualifications lie between £2771 (Level 2 qualifications – CSE/GCSE or equivalent) and £3447 (Level 3 qualifications – AS/A-levels or equivalent). In addition, estimates about the long-term economic impacts of teenage pregnancy in terms of the potential scale of the motherhood penalty in the UK for non-teenage mothers were derived based on the average earnings for women in age groups 22–29 years, 30–39 years and 40–49 years, where the potential size of the gap is estimated to lie between 5% and 10% of earnings. 201 Given the average level of earnings in each age group, the size of the gap is estimated to be in the order of £990–1980 per annum for a woman in her 20s, rising to £1223–2445 for a woman in her 40s. 201 The medium- and long-term cost outcomes associated with teenage pregnancy that are included in the model are shown in Table 43.

Three articles235,237,238 were identified within the literature, assessing outcomes for the father of a child born to a teenage mother. Evidence from two237,238 of these, which were based on a longitudinal study of young fathers in the UK, indicates that parenthood may lead young fathers towards a low-skilled labour trajectory, especially for those living in a social deprivation status. However, this was a qualitative study, and no impact on the long-term economic outcomes of young fatherhood was captured. Evidence from the third235 article suggests that becoming a young father (< 23 years) is not associated with substantial long-term negative outcomes, although this study does not control for unobservable characteristics. Given the limited and poor quality of the evidence found in the literature, no negative impact on the father has been incorporated into the model. The review identified one study6 comparing the long-term outcomes of children born to teenage mothers with those of children born to older mothers. This study suggests that children of teenage mothers have a slightly lower chance of high educational attainment, greater risks of inactivity and teenage childbearing, and a higher probability of lower earnings. However, this study did not adequately control for unobservable characteristics, which may influence the child’s outcomes, and hence it may overestimate the negative outcomes associated with a teenage birth. Given that other studies235,239 found that accounting for unobserved family background typically mitigates the estimated consequences of early childbearing, the model takes a conservative approach and does not incorporate any long-term negative implications of a teenage birth on the child.

Finally, from the perspective of the state, young age at motherhood is typically associated with an increase in claims for means-tested benefits. 165 Based on the evidence retrieved from the literature,176 the model assumes that 90% of teenage mothers will receive income support and associated benefits, which include child tax credits. In addition to the long-term outcomes of a teenage birth, we identified a study240 using UK-based cohort data that considers the impact of teenage birth on immediate birth outcomes, including very low birthweight. This study suggests that the probability of women who aged< 16 years having a very low-birthweight baby (< 1500 g) is 0.016. The parameters associated with the outcomes of a teenage birth are shown in Table 38.

Modelling the number of sexually transmitted infections

The STI model implemented in the analysis consists of several interacting submodels for each STI. All submodels are connected with the same sexual network to enable transmission and co-infection.

The model estimates the number of STI cases averted for HIV and also for chlamydia, gonorrhoea and genital warts, according to the risk of infection as shown below and the proportion of sexually active individuals who receive the intervention. The data and assumptions used to derive the model parameters are presented in the following sections.

Sexual behaviour

Evidence about the effectiveness of condoms for protecting against STIs was taken from the available literature. The effectiveness of condoms in reducing heterosexual HIV transmission was estimated to be 90%, based on evidence from the literature. 247 For genital warts, we used the evidence from a longitudinal study on young people,248 which suggested that consistent use of male condoms reduced the risk of genital HPV transmission by 70%. A Cochrane systematic review25 estimated that consistent use of condoms results in an 80% reduction in HIV incidence, and the authors also noted that there was insufficient evidence to estimate the effectiveness of condoms in preventing other STIs. We therefore assumed that condom effectiveness was 80% for HIV, 70% for genital warts and 90% for chlamydia and gonorrhoea. A summary of the base-case values for the parameters related to sexual behaviours in the target population is shown in Table 41.

Benefit and cost outcomes of pregnancies and sexually transmitted infections averted