A proportionate, universal parenting programme to enhance social-emotional well-being in infants and toddlers in England: the E-SEE Steps RCT

Article history

The research reported in this issue of the journal was funded by the PHR programme as project number 13/93/10. The contractual start date was in February 2015. The final report began editorial review in October 2020 and was accepted for publication in October 2021. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The PHR editors and production house have tried to ensure the accuracy of the authors’ report and would like to thank the reviewers for their constructive comments on the final report document. However, they do not accept liability for damages or losses arising from material published in this report.

Copyright statement

Copyright © 2022 Bywater et al. This work was produced by Bywater et al. under the terms of a commissioning contract issued by the Secretary of State for Health and Social Care. This is an Open Access publication distributed under the terms of the Creative Commons Attribution CC BY 4.0 licence, which permits unrestricted use, distribution, reproduction and adaption in any medium and for any purpose provided that it is properly attributed. See: https://creativecommons.org/licenses/by/4.0/. For attribution the title, original author(s), the publication source – NIHR Journals Library, and the DOI of the publication must be cited.

2022 Bywater et al.

Social and emotional well-being

Social and emotional well-being in childhood refers to the ability of a child to express and manage their emotions in socially and culturally appropriate ways and to develop positive relationships with their parents, other adult caregivers and peers. 1,2 The development of social and emotional well-being requires many foundational competencies, skills and characteristics, including, for example, inhibitory control, emotion regulation, self-confidence, perspective-taking and attachment. 1,3 Throughout infancy and toddlerhood, the typically developing child experiences rapid cognitive and maturational changes that support the accomplishment of increasingly complex developmental tasks in social and emotional domains of development. 2 By age 20 months, infants can respond to their own name, recognise themselves, display interest in other people (e.g. adults and peers) and are able to engage in co-ordinated interaction with them. In addition, infants at this age can form attachments with their caregivers and develop internal working models of relationships, are capable of expressing basic emotions and can also regulate their emotions through self-soothing with support from adults. 3 These social and emotional competencies form the foundations of healthy development and school readiness.

There are strong continuities between infant social and emotional well-being and later life outcomes. Impaired social and emotional well-being in the early years increases the risks of negative outcomes throughout childhood and into adulthood. For example, longitudinal studies document increased risks of poor mental health, as well as increased risks of antisocial behaviour and criminality, and poor educational and employment outcomes. 4–6 Conversely, positive social and emotional well-being in the early years is associated with good health and development outcomes throughout the life course and provides a basis for adaptive resilience to future adversities. 4,7,8 Evidence suggests that promoting social and emotional well-being in the early years is more effective, and less costly, than interventions delivered later in life once difficulties are more entrenched. 9

Parent–child relationships play a critical role in the development of social and emotional well-being. Parenting practices, styles, skills and knowledge, as well as parental mental health, can have an impact on the quality of parent–child relationships and attachment bonds. Unresponsive parenting (e.g. when a parent is under stress or experiencing depression) can lead to ineffective parenting strategies and (inadvertent) emotional neglect. 2 Although the majority of research has been conducted with mothers, there is a growing field also documenting the influential role of fathers and other co-parents, such as grandparents, in the development of children’s social and emotional well-being. 10,11

Parenting interventions to promote infant social and emotional well-being

Parenting programmes are effective in reducing behavioural, social and emotional difficulties in school-aged children,12,13 as well as improving parent psychosocial health. 14 However, as highlighted by the National Institute for Health and Care Excellence (NICE) in 2012, there is a gap in our understanding of the extent to which parenting and family-focused interventions are effective in enhancing social and emotional well-being specifically in infants. 15 An earlier systematic review found some preliminary support for the use of group-based parenting programmes to improve the emotional and behavioural adjustment of children aged < 3 years. 16 However, there was insufficient evidence to support any firm conclusions and further research on parenting programmes for younger age groups was recommended. 16 Although there is significant policy interest and increasing research in the area of early intervention and prevention [with the subsequent establishment of the Early Intervention Foundation (London, UK) and the 1001 Critical Days Manifesto movement], the evidence gap identified by NICE still exists.

The case of proportionate universal interventions

There has been a particular call for research on interventions provided to families within a framework of ‘proportionate universalism’. Proportionate universalism was first proposed in the Marmot review (2010)17 and refers to services designed to be delivered universally, but with a scale and intensity that is proportionate to the level of need or disadvantage. Proportionate universalism is a response to an overwhelming body of evidence highlighting social gradients of health, whereby people at the top have less threat to health than the deprived people at the lower end of the gradient. 29 A proportionate universalism approach argues that services will not reduce health inequalities if they are focused solely on the most disadvantaged in society; however, if we solely concentrate on the most disadvantaged then the health gradient will not decrease and will only tackle a small part of the problem’. 17

The Marmot review’s call to action included strategic objectives concerning the provision of proportionate universal parenting services. 17 Although there are numerous examples of parenting programmes for young children in the UK (see the Early Intervention Foundation’s Guidebook18), many such programmes are targeted at high-risk families or children who are already experiencing difficulties. Targeted approaches, although potentially effective, have been criticised for causing stigma towards parents experiencing disadvantage and, in turn, affecting the uptake of services. In addition, it has also been argued that targeted approaches often do not identify all children and families in need of additional support. 8 Proportionate universal approaches that embed interventions within existing universal services may improve uptake and interaction with, and access to, care. 8,16

Lack of proportionate universal interventions and evidence base

Despite clear potential, there is a large gap in the evidence base for proportionate universal interventions. A recent systematic review of universal parenting interventions for 0- to 2-year-olds reported a lack of evidence for effectiveness of interventions in the postpartum to 24-month period. 8 Although the quality of evidence was low, previous controlled studies reported no differences between intervention families and controls. The authors8 speculate that one reason for the lack of intervention effects may be a lack of behaviour change content in the programme theory for included interventions, despite their reliance on parental behaviour change as a mechanism for improving social and emotional well-being in infants. Hurt et al. 8 concluded that ‘there is an urgent need for robust evaluation of existing interventions, and to develop and evaluate novel interventions to enhance the offer to all families’.

The Incredible Years^® parenting programmes

The Incredible Years^® (IY) parenting programmes [URL: www.incredibleyears.com (accessed 6 January 2022)] are parent interventions that are underpinned by social learning theory and are designed to enhance the social and emotional well-being of children (aged 0–12 years). The programmes encourage rewards for behaviours in children parents want to see more of, while ignoring behaviours parents want to see less of. The IY programmes are manualised and delivered by trained leaders to groups of 10–12 parents for 2 hours a week for 10–14 weeks. The Incredible Years infant programme (IY-I) and Incredible Years toddler programme (IY-T) versions for 0- to 1-year-olds and 1- to 3-year-olds, respectively, build on decades of research evidence that demonstrate the effectiveness of the IY programmes for parents of children aged ≥ 3 years. Previously, the older-age IY programmes have demonstrated effectiveness, cost-effectiveness and transportability in independent trials across several countries/contexts. 12 Meta-analyses suggest that IY may be beneficial to younger children and their parents,19 and that families with severe depression and severe conduct problems demonstrate co-occurring changes for the better. 20 An analysis of data pooled from several IY trials suggested a large moderating effect for depression and that IY was more beneficial for children where parents were more depressed. 21

Evidence for the IY-I is slowly growing. 22 A small (n = 80) randomised non-targeted study in Wales, UK, showed that control mothers were significantly less sensitive during play with their baby at a 6-month follow-up. 23 In addition, a small trial in Denmark, delivered universally, found differential outcomes for the lowest and highest functioning families, suggesting that IY-I should be targeted (as originally designed). 24 Results from 12 IY-I groups (n = 79 group participants) showed parental benefits of improved mental health and parenting confidence post course (pre–post only, no comparator) and influenced a rural county in East Wales to scale delivery of IY-I. 25

Two IY-T trials, one in the UK26 and one in the USA,27 were inconclusive. The UK trial26 was a small community-based trial in Wales. The trial26 relied on geographical targeting to disadvantaged ‘Flying Start’ areas and did not always reach families that needed most support. The US trial27 delivered IY-T through primary care (i.e. paediatric practices), rather than community settings.

Both the IY-I and IY-T require further evaluation to establish their clinical effectiveness and cost-effectiveness and to inform the evidence base. Neither IY or similar parenting programmes [e.g. the Positive Parenting Program (Triple P)28] have, to date, been delivered or tested using a targeted proportionate universal approach delivered in community settings.

Introduction to the E-SEE Steps model and its components

The Enhancing Social–Emotional Health and Well-being in the Early Years (E-SEE) Steps model is a unique multilayer intervention delivery model that combines three elements of existing IY programmes in a proportionate universal approach. The universal dose is the Incredible Babies book (IY-B), with two targeted group-based programmes (i.e. IY-I and IY-T) providing support at a greater intensity for those with greater need at different points in their child’s development.

The IY age-appropriate programmes lend themselves well to a proportionate universal delivery model, as they can be delivered in cumulative ‘doses’ according to need. 29 The E-SEE Steps model has the potential to provide robust evidence and inform NICE guidance. Recent research30 on child outcomes using longitudinal data has demonstrated the usefulness of using ‘stacked’ early interventions across the early years of children’s lives to maximise impacts on child outcomes. In addition, a recent systematic review31 suggested that a proportionate universal approach, although underutilised, is useful for mental health interventions and, given previous evidence for older age IY programmes, this intervention may be useful when delivered in this way.

Explanation of rationale

Although there is significant policy interest, there is a lack of robust UK evidence for promoting social and emotional well-being and for programmes specifically designed to prevent later mental health issues developing in children aged ≤ 2 years. The early years are a critical period of development for children, during which empathic and responsive parenting promotes positive outcomes. However, the majority of parenting programmes are designed for older children for whom social, emotional and behavioural difficulties are identifiable. Recent UK policy and guidance has placed emphasis on a whole family approach (i.e. including fathers and grandparents in an integrated proportionate approach).

The proposed study will evaluate a preventative approach for parents of very young children at a time when the child may show no obvious signs of mental health or behavioural difficulties (or at least these signs are difficult to detect), although risk factors, such as parent or co-parent depression, may be present. The IY basic programme (for parents of children aged ≥ 3 years) has demonstrated substantial post-intervention improvements in a variety of parent and child outcomes and has a robust evidence base in the UK. 32 However, although IY-I and IY-T have been developed with the same successful format and support infrastructure as the basic programme, they have not yet been evaluated in a proportionate delivery model within a community-based trial.

Specific objectives

The E-SEE study comprised two phases: (1) a pilot trial and (2) a definitive randomised controlled trial (RCT). The pilot phase informed the main trial design and trial procedures. The main trial was designed to (1) establish the effectiveness of the E-SEE Steps model on clinical outcomes, (2) evaluate the processes around service delivery and (3) assess cost-effectiveness.

The primary research questions were:

• Does the E-SEE Steps model enhance child social and emotional well-being at 20 months of age when compared with services as usual (SAU)?

• Can IY be delivered as a proportionate universal model, and what are the organisational, or systems-level, barriers to and facilitators of delivering in this way, with fidelity?

• Is IY, and the proposed delivery model, cost-effective in enhancing child social and emotional well-being at 20 months when compared with SAU?

Alongside the outcome, process and economic studies designed to answer these questions (see Chapters 3–5, respectively), a series of substudies were planned (see Appendix 10), including (1) a study exploring the experiences of co-parents, (2) a study exploring hospital visits and admissions and (3) a study exploring comparisons with a similar trial conducted in Ireland.

Pilot phase

The study was funded as a two-phase RCT, comprising an internal 18-month pilot conducted in two local authority (LA) areas and followed by a 30-month pragmatic two-arm RCT conducted in four LAs. 35 The study met stop/go criteria to progress to a main trial phase; however, the pilot identified design changes needed to ensure viability [e.g. changes to programme materials, the addition of another screener and more flexibility for the delivery of the model by one organisation (i.e. health or LA) as opposed to the originally required two organisations (i.e. health and LA)].

Table 1 summarises the progression criteria, associated measures and assessment of the pilot, and consequent changes to study design and/or implementation.

A recalculation of sample size for the main trial was also undertaken using data collected during the pilot phase.

Although the overarching design remained the same, the pilot was reclassified as an external pilot. A full account of the pilot phase of the study and how the learning informed the main trial design is provided in Blower et al. 36

Main trial design

The trial was designed as a pragmatic two-arm RCT and economic appraisal, with an embedded process evaluation to examine the outcomes, implementation and cost-effectiveness of the intervention, and intervention uptake.

Participants and eligibility criteria

Eligible trial participants were parents (or primary caregivers) with the main parenting responsibility for the index child (aged ≤ 8 weeks) and their co-parent (if appropriate).

Primary caregiver is used here as an umbrella term to describe any person (e.g. biological parents, step-parents, foster parents, grandparents or legal guardians) who has the primary parental responsibilities of a child. English law states that if the child lives with their mother, then the mother is recognised as the primary carer.

Co-parent is a term used to describe any individual who may or may not be the ‘biological’ parent of the child, but who is involved in the upbringing of the child alongside the child’s primary caregiver (i.e. the father, or a parent by partnership or marriage to one of the child’s biological parents, such as stepmother). One co-parent could be recruited for each recruited primary carer.

Parents with a child aged ≤ 8 weeks were approached by health visitors or family and child service staff/services to see if they wished to hear more about the E-SEE study. For those who were interested, a form was completed, giving permission for the research team to receive their contact details, assess their eligibility and to contact the parent to make an appointment to visit them in their home (or an alternative venue to suit the parent) to discuss the trial further. Parents then decided if they wished to participate in the trial or not. Recruitment, therefore, was conducted within the home by the research team, with written informed consent being given by parents and co-parents wishing to participate. Parents could also self-refer if they had heard about the study via other channels (e.g. community groups and forums).

Non-eligible parents were provided with information about how to access local children centres and health service provision. Figure 1 illustrates the flow of participants throughout the trial.

FIGURE 1.

Flow of participants through the main trial. Note that ‘unwilling’ is defined as not wishing to attend the intervention and ‘unable’ is defined as unable to attend (e.g. because of lack of child care, returned to work or time the intervention was delivered).

Parents were eligible if they:

• had the main parental responsibility for a child aged ≤ 8 weeks at initial engagement

• were willing to participate in the research

• were willing to be randomised and, if allocated to intervention, were able to receive IY services offered

• were not enrolled in another group parenting programme at consent stage

• were fully competent to give consent.

Co-parents were eligible if the primary carer agreed to their involvement and if they lived with the index child or looked after them for at least three evenings each week.

The exclusion criteria were the opposite of the above and, in addition, parents were excluded if:

• the child had obvious organic or developmental difficulties or had been diagnosed with the same.

Settings and locations where data were collected

Expressions of interest, via completed proformas, were received from 16 potential sites in England. The proformas and initial discussions with sites were used to establish (1) levels of local deprivation, (2) sufficient live birth rates per year to allow recruitment of eligible and interested families, and to achieve an adequate randomisation sample with viable numbers for group delivery, and (3) willingness to support staff and intervention delivery costs and time. A number of interested sites were unable to participate because of limits on health visitor capacity/contracts or because sites were already delivering the IY-I or IY-T or were committed to running other parenting programmes. Four trial sites were selected that met the requirements. Two sites were in the north of England, both with council and NHS IY delivery, one site was in the Midlands of England, involving LA-only delivery (although health visitors also approached potential participants), and one site from the south of England, involving an NHS-only provider.

Intervention

The E-SEE Steps model adopts the IY series [URL: www.incredibleyears.com (accessed 7 January 2022)] of programmes for parents, children and teachers to enhance social and emotional well-being in children aged 0–12 years as content for the different levels of intervention. Two new programmes have been developed for parents of children aged 0–2 years: the IY-I and the IY-T (see Appendix 1, Figure 11, for the IY-I and IY-T logic model). Each programme is accompanied by a parenting book (i.e. the IY-B), reflecting the content of the programmes delivered in group sessions, with activity and journal pages. In addition, the E-SEE Steps model used the IY-B as a universal educational resource to increase parental knowledge and understanding about babies and toddlers’ social and emotional development. All intervention parents received this book immediately following randomisation to intervention and postal tracking numbers were used to establish receipt of the book. There is evidence37,38 to support a link between parents’ knowledge and their practice/parenting behaviour. Within the universal proportionate framework, three levels of the IY parenting programme were investigated in this RCT.

Universal proportionate delivery of the intervention

Following baseline data collection, the universal-level intervention dose (i.e. the IY-B) was posted to all intervention families to read and use at home. Postal tracking was used to establish receipt of the book. An active placebo (e.g. another type of book) was not specifically given to the control arm, as the control arm received SAU.

The IY-I and IY-T were targeted using two screener measures. Eligibility was based on the primary parent’s level of depression, as measured by the Patient Health Questionnaire-9 items (PHQ-9), and/or the child’s social and emotional well-being score, as measured by the Ages and Stages Questionnaire: Social and Emotional, 2nd edition (ASQ:SE-2). Eligibility for the IY-I and IY-T were assessed at follow-up 1 (2 months post baseline) (FU1) and follow-up 2 (9 months post baseline) (FU2), respectively, during home data collection visits. Eligible parents (and their co-parent if applicable) were invited to attend the IY-I and/or IY-T. The IY developer recommends home visits to eligible parents before the group sessions begin to better understand the family’s needs and to build rapport and engagement.

In an attempt to ensure viable group size (i.e. a minimum of five parents), non-research parents could be invited to the groups. These parents were invited by service staff based on professional judgement/assessment. We did not collect data for these parents, as they attended as they would for any other parenting intervention delivered at that site. This is accepted practice in types of research interventions that require viable group size. The intervention families could also access all SAU.

Setting for intervention delivery

Both IY-I and IY-T were delivered in convenient community venues and locations to reduce participant travel time, burden and drop-out. Travel and crèche facilities were provided by sites, where possible, and provision monitored. Each IY group was delivered by two IY-trained staff from health/LA staff, such as health visitors, infant mental health practitioners, children centre staff and nursery nurses. Group leaders attended separate 3-day training workshops for IY-I and IY-T that were delivered by accredited UK-based IY trainers. Sites were advised to deliver a ‘dry run’ practice of an IY-I or IY-T group prior to delivering the research groups. In addition, the IY developer recommended regular peer review and/or delivery supervision for facilitators for effective implementation and encouraged agencies to support accreditation of group leaders. Further details on training and supervision are provided in Report Supplementary Material 1.

Control treatment

Control parents/co-parents had access to SAU. IY-I and IY-T did not form SAU in participating sites, although other parenting programmes may have been available. We documented the nature of SAU in each locality and collected data on which health (and social) services families had accessed via completion of an adapted Client Service Receipt Inventory (CSRI). A waiting list control design was not feasible, as children of control group parents would exceed the IY-I and IY-T age range after intervention group completion. Implementation partners were asked to consider offering the IY-T book or IY basic programme to control parents at the trial end.

Outcomes and measures

Questionnaire packs were pre-tested with non-research parent representatives of the ethnic and socioeconomic profiles of the regions in the study [facilitated by the E-SEE Parent Advisory Committee (PAC)] to assess user-friendliness and comprehension of the questionnaire materials and length of time for completing them. Measure selection was also informed by a suite of systematic reviews of the measurement properties of outcome measures frequently used in RCTs of parenting programmes for children in the early years. 39,40 The child and parent measures presented were administered at all time points [i.e. baseline, FU1, FU2 and follow-up 3 (18 months post baseline) (FU3)] unless otherwise stated. Provenance, properties and rationale for all measures are presented in the trial protocol. 41 All measures were completed by parents, with the exception of the Infant CARE-Index observational measure and, in addition, researchers completed the CSRI and demographic form based on parent responses.

Data collection

Data collection took place in the participant’s own home. All data collectors were educated to postgraduate level (or with equivalent experience) and had prior experience working with children or families. All data collectors attended a study-specific training delivered by the E-SEE trial co-ordinator and E-SEE York trial manager prior to data collection. Data collectors also undertook safeguarding training and good clinical practice training. Further detail on training and data collection procedures can be found in the study protocol version 10. 41

Sample size

Sample size was calculated on the child primary outcome of social and emotional well-being using the ASQ:SE-2. The study was designed to evaluate the effectiveness of the whole E-SEE Steps model (i.e. ‘Do the scores of children in the E-SEE Steps arm, on average, stay below those scores for children in the SAU arm over the three follow-up measures?’). Parents were eligible to be offered the proportionate intervention if they scored ≥ 5 on the PHQ-9 or their child scored within the monitoring or cause for concern range on the ASQ:SE-2. We defined the minimal clinically important difference at FU3 to be 5 units on the ASQ:SE-2 in the E-SEE Steps arm compared with the SAU arm. We expected this effect to be consistently seen over the three follow-up points. Assuming a standard deviation (SD) of 18 units on the ASQ:SE-2 at FU3, the correlation between baseline and FU3 scores is 0.26, and between pairs of measurements after baseline is 0.40. For a design effect of 1.25 for the intervention arm, two-sided 5% significance level and 90% power, we would require the study to have retained 441 intervention participants and 92 control participants. Allowing for overall attrition of 12%, this would require 606 parents to be randomised with an allocation ratio of 5 : 1. The high allocation ratio was required to ensure that a sufficient number of parents were eligible and able to attend IY groups. Assuming the attrition rate of 12%, we expected 151 parents to be eligible for IY-I, with 50 parents attending, and 147 parents to be eligible for IY-T, with 48 parents attending, based on our pilot data.

Additional information about the ASQ:SE-2

The ASQ:SE-2 thresholds for concern are shown in Table 2. For interventions aimed at improving the mental health of parents, a minimal(ly) clinically important difference of 5 has been suggested; however, given that this is an intervention aimed at child behaviour, it is not appropriate here but is provided for reference.

Randomisation

Randomisation was at the individual level, using a web-based randomisation system developed by the Sheffield Clinical Trials Research Unit (Sheffield, UK) in collaboration with epiGenesys (Sheffield, UK) and using a randomisation sequence prepared by the trial statistician. E-SEE Step participants were randomised in a 5 : 1 ratio (intervention to control arms) stratified by sex of child, sex of carer, recruitment site and PHQ-9 baseline score or ASQ:SE-2 baseline score (with the last two variables representing level of need).

Randomisation occurred after eligibility had been established, informed consent obtained and baseline measures collected from parents to reduce initial attrition. The allocation schedule was concealed and the intervention arm was confirmed only once eligibility and consent were confirmed by researchers. A member of the research team inputted participant information to the online system to enable randomisation, with allocation results returned immediately. The trial co-ordinator informed families of allocation to condition.

Parent–child dyads were randomly allocated to intervention or control arms and stratified according to level of need based on the depression score of the parent with the main parenting responsibility or the child’s social and emotional well-being (assessed using ASQ:SE-2), sex of the child and primary parent, and recruitment site. The co-parent was automatically assigned the same allocation as the parent.

If the parent had missing depression data, measured using the PHQ-9, then randomisation was not possible. We did not randomise participants who had missed three or more of the main questions on the PHQ-9 measure at baseline.

Statistical methods

Information on the calculation of the sample size for the main trial and the clinical meaning of intervention effects has been provided in Sample size. Information on the analysis for the ancillary substudies is included in the study protocol, which is available along with the E-SEE main trial statistical analysis plan (SAP) and pilot SAP. 41

Effectiveness was assessed using intention-to-treat analysis, accounting for the repeated measurements of individuals at FU1, FU2 and FU3 using general estimating equations. Membership of the IY-I and IY-T groups is confounded with outcome at FU1 and FU2 and so programme group clustering was unaccounted for because it is not possible to do this appropriately without biasing the treatment effect estimate in the methods we have explored. This is also current practice in the wider community. 31 Simulations conducted during SAP development suggested that estimates from our chosen model are robust to intracluster coefficients (ICCs) < 0.2.

Ethics arrangements

This study was approved by NHS Wales Research Ethics Committee 5 (Bangor, UK) on 22 May 2015 (reference 15/WA/0178 and IRAS 173946). Health Research Authority approval, which was issued on the basis of an existing assessment of regulatory compliance, was received on 13 September 2016 (i.e. a letter of Health Research Authority approval for a study processed through pre-Health Research Authority approval systems). The chief investigator’s departmental ethics committee at the University of York required submission of project documentation and approval was given on 10 August 2015 (reference FC15/03).

Changes to protocol

Version 10 is the final version of the protocol. Please refer to the protocol for a full list of amendments. 41 Significant amendments (i.e. those not related to simple language and terminology) to the protocol in the main trial phase (i.e. versions 5–10) are detailed below. All amendments were approved by the Research Ethics Committee, E-SEE TSC and DMEC, and were accepted by the funder.

Patient and public involvement

The involvement of parents and caregivers of very young children has been integral to the E-SEE trial. In this section we outline the different ways that patient and public involvement has contributed to the E-SEE model, from the planning stages through to the dissemination of trial findings. Reflections on what worked well and not so well are also provided.

Main outcome measures

The primary outcome was child social and emotional well-being using the ASQ:SE-2. Higher values of ASQ:SE-2 indicate a higher degree of risk in terms of the social and emotional well-being of children. The key secondary outcome was the PHQ-9 depression module for the primary carer, subsequently referred to as PHQ-9. Higher values on the PHQ-9 indicate higher levels of depression for the parent. Therefore, for both the primary and key secondary outcome, we were looking for the difference in average scores between the treatment and control group to be negative for the results to indicate efficacy. We were powered on the primary outcome only and so present findings in terms of significance for ASQ:SE-2 and trends or confidence intervals (CIs) for PHQ-9.

There are age-appropriate versions of the ASQ:SE-2. Three versions were used in this trial.

Data completeness

The trial had very high levels of response to all surveys at all follow-up points both in terms of the number of surveys returned and the number of items completed on each survey (see Report Supplementary Material 2). Therefore, very little imputation was required.

Baseline covariates

No major imbalance between arms at baseline was found, both in terms of covariates (Table 3) and baseline outcome scores (see Appendix 3, Table 35). Mean ASQ:SE-2 score at baseline was moderately lower in the treatment group than in the control group, but this was adjusted for in the primary analysis, as baseline ASQ:SE-2 was a planned covariate.

Although we assessed imbalance between the arms, we did not assess representativeness of the sample to local or national population data. What was noticeable here was the large number of E-SEE parents in live-in relationships (90% and 89% of intervention and control parents, respectively, with 65% and 68% married, respectively). According to the Office for National Statistics,52 marriage prevalence is 50% and 60% of the population (aged ≥ 16 years) were living as a couple in 2019, the majority of whom were married. Only 3% and 4% of parents (intervention and control, respectively) said that they were not in a relationship, which is below the national average of 25% of families being lone-parent families. The numbers were too small to conduct subgroup analyses.

Primary analysis of primary outcome: ASQ:SE-2

The mean ASQ:SE-2 score was higher in the treatment group than in the control group at all follow-up points apart from FU3. As higher scores relate to poorer outcomes, this provides no evidence in the direction of efficacy (Figure 2).

FIGURE 2.

Summary of overall ASQ:SE-2 scores by follow-up point. (a) Baseline; (b) FU1; (c) FU2; and (d) FU3.

There was a borderline statistically significant positive difference in ASQ:SE-2 scores between arms after controlling for baseline covariates and stratification variables, suggesting that the intervention may have a detrimental effect (3.02, 95% CI –0.03 to 6.08) (Table 4).

Residuals from the primary analysis model were not perfectly normal, as they were symmetric around zero, with no outlying or overtly influential values (see Appendix 4, Figure 12). The validity of the primary analysis was, nevertheless, investigated using a sensitivity analysis because the ASQ:SE-2 itself has a positively skewed distribution and, together with the extreme imbalance in allocation the type I error, may be increased because there is a heightened risk that, by chance, we observe more extreme values in the larger treatment arm.

Sensitivity analysis of the primary outcome

In addition to the pre-planned sensitivity analysis outlined in the SAP [see URL: www.york.ac.uk/healthsciences/research/public-health/projects/e-see-trial/#tab-3 (accessed 10 January 2022)], unplanned additional sensitivity analyses were undertaken because of the two issues above.

The first planned sensitivity analysis refitted the primary analysis model after transforming ASQ:SE-2 to z-scores and percentage scores (where percentage score is the difference from minimum score expressed as a percentage of the difference between the minimum and maximum scores possible). This transformation, to some extent, also addressed the issue that three different (age-specific) ASQ:SE-2 versions were used over the follow-up periods. The second planned sensitivity analysis was MI. However, because of the skewed distribution of the outcome and concerns over inflated type I error rates due to unbalanced allocation, we felt that further sensitivity analyses were justified.

Four types of unplanned sensitivity analyses were conducted. First, we re-ran the primary analysis on square root-transformed primary outcome scores. Appendix 4, Table 36, shows that the residuals for this model are less skewed. Second, we re-ran the primary analysis model, assuming an unstructured rather than auto-correlated structure (see Appendix 4, Table 37). This allowed more of these data to be used because Stata omitted participants who do not have data for at least two adjacent follow-up points. Third, owing to the skewed raw data and imbalance in the size of each arm, a permutation test was carried out. We permuted the allocation to arm 1000 times, keeping all other variables unchanged, and calculated the proportion of times we observed an effect estimate greater than or equal to the actual effect estimate to calculate a permutation-based p-value.

Finally, MI was undertaken using the difference between each participant’s ASQ:SE-2 follow-up score and baseline ASQ:SE-2 score. MI on this ‘differenced’ data removed much of the skew in these data because participants with extreme ASQ:SE-2 values at follow-up typically had similarly extreme values for ASQ:SE-2 at baseline and so the differences were more normally distributed. The MI was conducted using chained equations and 25 replications, with qualification imputed using a logit model and ASQ:SE-2 imputed using a multivariate normal model by treating each time point as a separate variable in the chain. Complete data were available for all other variables in the primary analysis model.

Table 5 summarises the outputs from all models (primary and sensitivity) and Figure 3 shows a forest plot for all analyses, with results transformed into original ASQ:SE-2 units. The sensitivity analyses suggest that the results were not particularly sensitive to the imbalance between arms and the skewed distribution of the primary outcome. However, the strength of the signal diminishes when methods that allow for missing and incomplete data are employed. To understand why the additional observations had this effect, we explored some features of the missing data by comparing baseline ASQ:SE-2 scores and treatment allocation between the missing and non-missing observations (see Appendix 5, Table 38). It is difficult to draw any conclusion from this comparison because of the complex relationship between missing data, time point, allocation and effect.

FIGURE 3.

Forrest plot for primary analysis models on original scale. MCID, minimum clinically important difference.

Exploratory analysis of the primary outcome

It is interesting to note that comparing the unadjusted differences between arms at each follow-up point suggested a stronger early effect in favour of SAU, which tended to diminish over time (see Table 4). Most of the weight of the signal in the overall effect was due to the larger difference seen at FU1, which then diminished over time. This might suggest that the book alone increased the primary outcome in the treatment group compared with the control group. This pattern was more pronounced after subtracting the baseline score from the score at each follow-up point and looking at unadjusted differences between arms. In this adjusted analysis, the size of the effect decreased from 4.6 at FU1 to 2.9 at FU2 and –0.9 at FU3.

Two planned and one unplanned exploratory analysis were conducted. First, the subgroup analysis was conducted for social and economic background (using ‘primary parent educated to degree level’ as a proxy), whether or not the index child was the first child, sex of the child and site by adding an interaction between treatment group and subgroup into the primary analysis model. For ‘first child’, the main effect was additionally included because it is not a covariate. There was weakly significant evidence of an interaction with treatment for site (p = 0.06), but no evidence of a significant interaction for the other three subgroups (Table 6). The site-level control groups were too small for additional statistical analysis (Figures 17–20 in Appendix 6 do not suggest better scores in the treatment groups compared with control for any of the sites).

Second, planned indicative analysis of whether or not the baby and toddler groups had an impact were undertaken. This analysis is indicative only because allocation to group programme occurs after randomisation. Eligibility for group programme is determined by ASQ:SE-2 and PHQ-9 scores at FU1 for IY-I and at FU2 for IY-T. The indicative analysis compared the eligible participants with the subgroups of control participants that had ASQ:SE-2 and PHQ-9 scores in the eligible range for the groups (referred to a pseudo-controls), using the same model as used for the primary outcome. The analysis found no evidence of a difference between group programme participants and the pseudo-controls (see Appendix 7, Table 39).

Finally, we conducted an unplanned analysis by converting ASQ:SE-2 score into a categorical variable indicating different risk categories in line with how it is commonly used in practice. This analysis was undertaken to explore the possibility that, even though the intervention was not effective at the aggregate level, dyads in certain risk categories may have responded in a clinically important way. We compared the proportion of children in each risk category at each follow-up as a proportion of the number in that risk category at baseline and found no difference between arms in how participants transitioned between risk categories following the intervention (Table 7). Similar analyses are provided for the PHQ-9 (see Table 10) and the Infant CARE-Index (see Table 11) and these analyses, again, showed no difference.

Key secondary outcome: depression (PHQ-9) score of primary caregiver

Table 8 and Figure 4 show that the key secondary outcome (i.e. parent depression score) was lower (i.e. better) in the treatment group at FU1 after the IY-B universal dose than at baseline.

FIGURE 4.

Summary of PHQ-9 by treatment arm and follow-up point. (a) Baseline; (b) FU1; (c) FU2; and (d) FU3.

The PHQ-9 items scores also had a skewed distribution (see Appendix 8, Figures 21 and 22) and so an unplanned sensitivity analysis was undertaken following the same argument as applied to the sensitivity analysis of the primary outcome. We applied three of the unplanned sensitivity analyses used for the primary outcome. It is of interest that, conversely to the primary outcome, the direction of the effect for PHQ-9 consistently favours the intervention, although the effect is not statistically significant (Table 9). The magnitude of the signal shifts in favour of the treatment as we move from the planned model to the two sensitivity analyses (see Appendix 8, Figure 23). It is difficult to interpret the meaning of an effect of this size (i.e. < 1 unit) for the PHQ-9 in the context of this trial. In depression studies, a minimum clinically important difference (MCID) of 5 units is typical. However, in the public health context, a more modest effect would be meaningful because a trial would typically not recruit severely depressed parents.

Similarly, as for the primary outcome, we undertook an additional unplanned analysis of how parents transitioned between PHQ-9 categories during the intervention. PHQ-9 can allocate parents to five levels of depression. Very few parents were moderately or severely depressed and so the transition analysis was based on three categories: (1) no depression, (2) mild depression and (3) moderate to severe depression. Table 10 shows the proportion of people in each depression category at follow-up as a proportion of the people in that category at baseline. Consistent with the weakly significant finding in favour of the treatment for PHQ-9, Table 10 suggests that people in the treatment group were slightly more likely to be categorised as having less extreme depression at follow-up than people in the control group.

Other secondary outcomes

No secondary outcome measures provided any signal to suggest a difference between arms. First, we measured dyadic synchrony for the primary parent–child dyad at all follow-up points using the Infant CARE-Index (Table 11) (for which a higher score indicates higher dyadic synchrony). The number of people available with complete data for the Infant CARE-Index across all four time points was a relatively small subset of the study (i.e. 119 of the 269 participants who consented to video observation). Although the results are not statistically significant, with a modest signal in favour of the control overall (–0.25), a similar slight trend can be seen, that is the magnitude of the signal at FU1 (–1.69) is larger than at FU2 (0.11) and FU3 (0.44). Similarly, as for the ASQ:SE-2 and PHQ-9, we undertook an additional unplanned analysis of how parents transitioned between the Infant CARE-Index dyadic synchrony categories during the intervention. The Infant CARE-Index has four levels of dyadic synchrony: (1) sensitive, (2) adequate, (3) inept and (4) risk. Appendix 9, Table 40, shows the proportion of people in each category at follow-up as a proportion of the people in that category at baseline. Appendix 9, Table 40, suggests that there is no difference between arms with regard to transition through the Infant CARE-Index categories.

Third, two measures were collected at all follow-up points for both parents: (1) parenting skills using the PSOC survey (Table 12) and HRQoL using the EQ-5D-5L (Table 13).

Fourth, three outcomes were collected from primary and co-parents at FU3 only: (1) child behaviour using the SDQ, (2) cognitive development using the PedsQL and (3) parent–child interaction using the MPAS and PPAS, as appropriate (Table 14). Finally, data on patterns in how children were fed were collected to see if the intervention had an impact on the rates and length of breastfeeding, but no obvious differences between arms over time were apparent (see Report Supplementary Material 3).

Summary of intervention take-up

With some exceptions, the IY parenting groups had relatively low take-up, particularly for the IY-T groups, but once a participant attended one session then they typically attended at least half of the remaining sessions, suggesting a very high level of satisfaction once engaged (Figures 5 and 6 and see Report Supplementary Material 1). This is explored more formally in the process evaluation (see Chapter 4). There was a relatively large difference between sites in levels of take-up for IY-I, with sites 2 and 3 both having > 50% of eligible parents participating and sites 1 and 4 both having < 50% of eligible parents participating (see Figure 6 and Report Supplementary Material 1). There was less variation between sites in the uptake of toddler groups, with sites 2 and 4 having 30% of eligible parents participating and sites 1 and 3 having 10% of eligible parents participating. This echoes the finding of a significant interaction between treatment and site allocation. Therefore, it may be appropriate to undertake further analysis of outcomes for the sites with higher uptake using causal methods applicable to smaller sample sizes.

FIGURE 5.

Eligibility, compliance and session attendance: IY-I. (a) Site 1; (b) site 2; (c) site 3; and (d) site 4.

FIGURE 6.

Eligibility, compliance and session attendance: IY-T. (a) Site 1; (b) site 2; (c) site 3; and (d) site 4.

Characteristics of compliers and non-compliers

We expected 151 participants to be eligible for IY-I, with 50 attending, and 147 participants to be eligible for IY-T, with 48 attending. We achieved 51 participants for IY-I and 21 participants for IY-T. To explore the relatively low take-up further, we compared the characteristics of compliers with non-compliers, defining compliance as ‘attended at least one session’ and non-compliance as being ‘eligible but attending no sessions’. This was a relatively low threshold; however, in the absence of any empirical or theoretically based threshold for complete service receipt, compliance was treated only as an indicator of initial intervention engagement and, as mentioned above (see Summary of intervention take-up), participants who attended one session were likely to attend at least half of the subsequent sessions. This analysis is descriptive to avoid inflating the type I error by multiple testing. Table 15 compares compliers with non-compliers in terms of baseline characteristics, including ASQ:SE-2 and PHQ-9 scores (using FU1 for IY-I and FU2 for IY-T). There appears to be little difference between compliers and non-compliers overall, but there are a few exceptions. Parents educated to higher level and in the higher-income bands were marginally more likely to take up the intervention. Dyads with ASQ:SE-2 child development scores below the monitoring zone were more likely to take up IY-T sessions. This result should be viewed with caution because of the small numbers of participants who took up IY-T overall. Dyads with lower PHQ-9 parental depression scores and ASQ:SE-2 child development scores below the monitoring zone were more likely to take up IY-I sessions. The lower take-up by people with more severe baseline scores is of particular interest, as it suggests that the intervention may be less appealing to people who are most likely to benefit.

Overall summary of outcomes

The primary analysis of the primary outcome (i.e. the social and emotional well-being of the child) was borderline statistically significant in favour of the control group. Sensitivity analyses of the primary outcome analysis diminished the strength of the signal. The MCID of the primary outcome stated in the sample size calculation was 5 units (i.e. –5 units, as we expected those receiving the E-SEE Steps model to have lower scores). Therefore, we have strong evidence that the proportionate intervention E-SEE Steps is not clinically effective, as –5 was well outside of the 95% CIs for both the primary and sensitivity analyses. The primary analysis was a repeated-measure analysis over the three follow-up time points, adjusted for baseline. When the three time points were analysed separately (with each adjusted for baseline), the strongest signal was seen at FU1 in favour of SAU, which was the measurement following the universal dose of the IY-B.

By contrast, the analysis of the key secondary outcome (i.e. depression levels among primary parent) provided a moderate signal in favour of the treatment. Considering each time point separately for depression of the primary parent, the strongest signal was, again, seen at FU1.

All of the 95% CIs for the primary analyses and associated sensitivity analyses excluded –5. Therefore, although we did not achieve the target sample size, the high retention rate to the trial resulted in robust evidence that the E-SEE Steps model is not clinically effective in enhancing social and emotional well-being. However, it is worth noting that take-up of IY-T was substantially lower than expected and this will have had an impact on its ability to have an effect overall.

Introduction

The overarching aim of the process evaluation was to contextualise the main trial’s effectiveness and cost-effectiveness findings in terms of why and how the intervention (i.e. the E-SEE Steps model) may or may not have worked. The study was focused on understanding the context and processes supporting the model as a whole, rather than the individual programme components/levels (i.e. IY-I and IY-T). For the pilot trial findings, see Blower et al. 36

The process evaluation had three main aims:

1. assess the uptake and fidelity of the intervention when delivered by public sector services in the UK

2. identify the factors associated with successful implementation of the intervention in the UK

3. assess the acceptability and feasibility of the intervention for delivery in public services in the UK.

These aims were addressed via the following research questions:

• Can a multiagency service deliver the E-SEE Steps model (i.e. IY) in a proportionate universal model?

• What level of fidelity is achieved at each level of the intervention?

• What are the organisational- or systems-level barriers to and facilitators of delivering in this way?

• How acceptable and feasible is delivery of the E-SEE Steps model for key intervention stakeholders?

Method

We used a multimethod approach to explore these process evaluation questions, embedded within the main processes of the trial. In addition, a qualitative substudy (see Appendix 10) explored the views of fathers as co-parents, which supplemented findings from the process evaluation.

Findings

Aim 1

Research question 1: designing and delivering a multiagency intervention

Service design meetings

The E-SEE research team provided service design support to sites in setting up the targeted group-based programmes in their areas. Senior managers and key delivery stakeholders were invited to attend service design meetings. Sites differed in their ability to provide the support known to increase engagement, access and retention to parenting interventions, including the provision of child care, transport support and preparation to ensure leaders were proactive and confident in the materials (Table 17). Suggested leader preparation included undertaking dry runs of delivery to increase familiarity with the materials and processes. Only one site managed to achieve this under the direction of a colleague who was experienced in IY.

TABLE 17 - Service design decisions

Site	Programme	Number of groups	Number of practitioners trained	Co-delivery?	Lead where not co-delivery	Transport and/or crèche provided	Home visits before groups started?
1	IY-I	2	13	Yes		Crèche and transport	Yes (n = 4)
2	IY-I	2	16	No	Health	Crèche	Yes (n = 6)
3	IY-I	2	17	Yes		Crèche	Yes (n = 1)
4	IY-I	2	10	Yes		Crèche and transport	No
1	IY-T	2	9	Yes		Crèche and transport	Yes (n = 7)
2	IY-T	2	10	No	Health	Crèche	No
3	IY-T	1	11	No	Children’s services	Crèche	No
4	IY-T	1	8	Yes		Crèche and transport	No

Five groups across two sites were delivered by a single agency in the main trial because of the need for flexibility on delivery in context, informed by the pilot findings. Health professionals’ time for training and delivery was paid out of NHS excess treatment costs, whereas no budget support was provided to children’s services staff. The number of groups delivered in each site was smaller than the number intended.

Selecting group leaders

Demographics and experience of site practitioners

Ninety-four practitioners were identified by sites to be trained as prospective group leaders, based on their role/remit for supporting parents of children under 2 years of age, experience in parenting programmes and group facilitation, interest in being involved in the E-SEE trial, and capacity to prepare and deliver groups over a full term. Twenty-nine (31%) of these practitioners went on to deliver the IY-I and IY-T groups. Of the practitioners who consented to completing a pre-training research questionnaire (n = 50), all were female, 86% (n = 43) were white British and 14% (n = 7) identified as British ‘other’. Forty per cent of practitioners were aged ≥ 46 years (36% were aged between 31 and 45 years and 24% were aged between 18 and 30 years). Fifty-six per cent (n = 28) of practitioners worked full time and 44% worked part-time, with hours ranging between 15 and 32 per week. All practitioners who completed a questionnaire held a qualification [i.e. an undergraduate degree (n = 21, 42%), a level 3 qualification (e.g. Qualifications and Credit Framework, National Vocational Qualification, National Nursery Examination Board or Advanced Level) (n = 13, 26%), a Business and Technology Education Council qualification (n = 6, 12%), a pre-graduate diploma (n = 5, 10%), a post-graduate diploma (n = 4, 8%) or a master’s qualification (n = 1, 2%)].

Practitioners (n = 26) selected by sites to deliver the E-SEE Steps groups were more likely to work full time and to have previously delivered parenting interventions, including the IY programme. Perhaps unsurprisingly, 42% of those practitioners who went on to deliver the trial groups also reported feeling ‘confident’ prior to being trained in setting up and running a parenting group compared with the same proportion (42%) of practitioners who reported feeling only ‘somewhat confident’ and also went on to deliver the trial groups. Report Supplementary Material 1 presents differences between practitioners who were trained and those who went on to deliver the trial groups.

Training and supervision

Training for IY-I and IY-T was delivered to each site, on average, 2–3 (range 1–11) months before delivering the groups. Ninety-four staff were trained in 10 separate training sessions. Of this total, 29 went on to deliver at least one session of the trial groups. Supervision sessions were carried out remotely via teleconference, with the aim to review goals, questions and video clips. Organisation firewalls prevented the sharing of video footage of delivery for review by the mentor for most of the supervisory groups, thereby relying on leaders’ self-perceptions of their strengths and weaknesses. Report Supplementary Material 1 provides site-level details of the training and supervision provided. Feasibility of the supervision model for future implementation is explored in the themes below.

Site-level services as usual overview

Services as usual were collated for each site for parenting support programmes spanning behaviour management, healthy weight and nutrition, early learning and development, postnatal support and support for children with additional needs. This is important, given the outcome findings of no difference between the E-SEE Step model and SAU.

Across sites, there were a larger number of services and support targeted at children in the early years (i.e. 0–5 years), although there was a proportion of programmes available to all ages. In terms of more targeted provision, this was for ages 0–18 months and 0–5 years across all sites. Sites 2 and 4 had the lowest proportion of programmes focusing on the 0- to 18-month period, and site 3 had the highest proportion of services on offer for this age group, with fewer general/any-age programmes. Table 18 shows the distribution of programmes across age groups.

TABLE 18 - Distribution of parenting support programmes by age group across sites

Site	Age group (%)
	0–18 months	0–5 years	General/all ages
1	26.9	30.8	42.3
2	15	45	40
3	37.7	50.8	11.5
4	13.3	26.7	60

In site 1, the majority of parenting support was focused on early learning and development. Evidence-based programmes focusing on behaviour management (e.g. the Strengthening Families programme) and healthy weight and nutrition [e.g. the Health Exercise and Nutrition for the Really Young (HENRY) programme] were available to parents. Early learning and development services included messy play and story sessions, but no evidence-based programmes were offered.

In site 2, SAU involved evidence-based programmes, including the Solihull Approach, Triple P, Stepping Stones and IY-I, with IY-I as the only universally offered programme. Alongside these services, social toddler workshops and play and sensory sessions were delivered.

Services as usual in site 3 were the widest ranging of the four sites, with a total of 61 programmes on offer. A large proportion of these programmes were focused on the 0- to 18-month stage, including Amazing Babies, Mums in Mind, Baby and Me and IY-I. Both IY-I and IY-T were part of targeted SAU. However, IY was not delivered widely and an agreement was made between the research team and site that the research would be conducted in areas of their site where IY-I and IY-T were not currently offered to reduce any risk of contamination.

Site 4 had the smallest number of SAU focusing on 0- to 18-month stage, with a larger number of programmes available for a broader age range. SAU comprised several evidence-based programmes, including Triple P, Stepping Stones, Family Transitions and Solihull Approach, and these were offered as part of targeted services. Other services included baby massage, sensory groups and play groups.

Chapter 5 explores SAU further as part of an economic evaluation. According to parent reports on the CSRI, only four control families and three intervention families participated in additional parenting programmes listed on the CSRI. The attended programmes were Triple P, Solihull Approach and Home Start (see Report Supplementary Material 4 for a detailed breakdown of SAU accessed by study participants).

Research question 2: fidelity of delivery

Universal level: the Incredible Babies book

A minimum of 82 per cent (82.1%) of books were received by families according to postal tracking numbers. A mailroom error led to missing tracking numbers for the remaining participants across three sites. This error was rectified by asking parents during a scheduled call if they had received a book.

Targeted level: group programmes

Eligibility and attendance

The percentage of parents at each site accepting an offer of a place on the group intervention varied across programmes and between sites (range 29–88%). Table 19 illustrates the comparison between eligibility and take-up across sites. Those parents marked as ‘declined’ were sometimes uncontactable. On average, three attempts were made by group leaders to contact parents via telephone, with an average of two answerphone messages left by group leaders. If answerphone messages were not responded to, then one follow-up text was sent. Letters and home visits were occasionally used when group leaders could not contact the parent by other means.

TABLE 19 - Eligibility and take-up for IY-I and IY-T groups

Site	Programme	Number eligible (% of intervention arm)	Number accepted (% of those eligible)	Number ‘declined’	Reasons for decline, n
					Uncontactable	Other
1	IY-I	35/105 (33)	20 (57)	15	11	4
2	IY-I	24/71 (34)	21 (87.5)	3	1	2
3	IY-I	17/50 (34)	13 (76.5)	4	2	2
4	IY-I	25/59 (42)	11 (44)	14	10	4
1	IY-T	32/105 (30)	12 (37.5)	20	5	15
2	IY-T	26/71 (37)	15 (57.7)	11	2	9
3	IY-T	17/50 (34)	5 (29.4)	12	4	8
4	IY-T	26/59 (44)	16 (61.5)	10	3	7
Total		202	113 (55.9)	89	38	51

Reasons provided by parents for not taking up a place in the group differed slightly for IY-I and IY-T groups. For IY-I, reasons included not wanting to interrupt the baby’s sleep pattern and not wanting to attend a group programme. For IY-T, key reasons were work commitments, perceiving no need of the course and having other groups to attend. For both IY-I and IY-T, length and duration of sessions and child care concerns were raised.

Group attendance

The IY-I groups are designed as eight manualised sessions to be delivered over 10 weeks (the extra 2 weeks allow more time for higher-need vulnerable parents); however, none of the trial IY-I groups utilised the tenth week. Table 20 provides IY-I attendance data. Site 1b finished in seven sessions, as only two parents were available for the majority of sessions. Average individual attendance ranged from 3.3 sessions in site 1b to 8.1 sessions in site 4a. Overall, 15 parents who had accepted a place on an IY-I group did not attend the first session. This drop out is attributable to two sites (sites 1 and 2) in particular. Attendance declined by one-fifth after the first week. However, after this initial drop, rates were largely stable.

TABLE 20 - Attendance at IY-I groups

Only one parent could attend and so the session was cancelled.

Includes research co-parents: two in site 2a and in site 4a.

Notes

Non-research parents included in brackets.

Number of parents who attended at least one session: site 1a, n = 9; site 1b, n = 6; site 2a, n = 12 (includes two co-parents); site 2b, n = 6; site 3a, n = 7; site 3b, n = 8; site 4a, n = 10 (includes two co-parents); site 4b, n = 6. Site 1b is a second group in site 1, but was part of cohort 2 (whereas site 1a was the same site but part of cohort 1). In all other instances where a/b are used, it refers to the second group for that site (i.e. all other site groups ran within the same cohorts).

Site	Number accepted	Week number (n)										Average individual attendance (n)
		1	2	3	4	5	6	7	8	9	10
1a	13 (1)	6 (1)	5 (1)	4	5 (1)	5 (1)	3 (1)	5	4	6		5.3
1b	7	5	4	3	2	a	2	2	2			3.3
2a	21	10b	4	7b	10b	9b	11b	10b	9b	10b		5.8
2b		5	4	3	5	5	5	4	5	5		6.8
3a	13 (2)	6 (1)	6 (1)	5 (1)	5 (1)	4 (1)	5 (1)	5 (1)	6 (1)	3 (1)		7.7
3b		7 (1)	6	5 (1)	6	5 (1)	7 (1)	5 (1)	7	7 (1)		7.6
4a	11 (5)	9b (1)	7b (1)	9b (1)	9b (1)	6b (1)	9b (1)	7b (1)	7b (1)	9b (1)		8.1
4b		2 (4)	2 (4)	0 (3)	2 (3)	2 (4)	1 (3)	0 (2)	2 (4)	2 (4)		7.3
Total	65 (8)	50 (8)	38 (7)	36 (6)	44 (6)	36 (8)	43 (7)	38 (5)	42 (6)	42 (7)	0

Table 21 provides IY-T attendance data. Less than half of the parents who accepted a place attended the first session, and this was particularly noticeable for site 1b where none of the research parents attended any of the sessions. Of those parents who did attend week 1, there was relatively stable retention for the remaining weeks of the intervention, with a small drop appearing in week 3 and very few parents attending the final week of delivery. The average individual attendance of research parents and co-parents ranged from just two sessions in site 1b to more than nine sessions in site 3.

TABLE 21 - Attendance at IY-T groups

Includes research co-parents: one in site 1a, one in site 2a and two in site 4.

Note

Non-research parents included in brackets.

Only a single non-research parent attended the site 1b group on weeks 1 and 3, with no research parents in attendance. The group was stopped after week 4 because of limited research parent attendance. Number of parents who attended at least one session (includes non-research): site 1a, n = 4 (includes one co-parent); site 1b, n = 2; site 2a, n = 6 (includes one co-parent); site 2b, n = 5; site 3, n = 5; site 4, n = 10 (includes two co-parents). Site 1b is a second group in that site but was part of cohort 2 (whereas site 1a was in the same site but part of cohort 1) and this is why this one site is dealt with differently. In all other instances where a/b are used, it refers to the second group for that site (i.e. all other site groups ran within the same cohorts).

Site	Number accepted	Week number (n)												Average individual attendance (n)
		1	2	3	4	5	6	7	8	9	10	11	12
1a	6	4a	3a	3a	1a	2	1	2	0	1				4.3
1b	6 (1)	0 (1)	1 (1)	0 (1)	0									2
2a	15 (3)	4	6a	4	4a	2	4a	2	5a	3	3	4	4	7.5
2b		2 (1)	1 (3)	1 (1)	0	2 (2)	0 (1)	2 (2)	2 (2)	2 (2)	2 (1)	0		6.6
3	5 (3)	2 (2)	2 (2)	2 (2)	2 (2)	2 (2)	2 (2)	2 (2)	2 (1)	2 (2)	2 (2)	2 (2)	2 (2)	9.4
4	16 (1)	9a (1)	7a (1)	6a (1)	7 (1)	8a (1)	5 (1)	7a (1)	6	7a	5 (1)	7 (1)		8.3
Total	48 (8)	21 (5)	20 (7)	16 (5)	14 (3)	16 (5)	12 (4)	15 (5)	15 (3)	15 (4)	12 (4)	13 (3)	6 (2)

Adherence to process and content

Most groups delivered between 75% and 100% of the intended sessions and six out of eight IY-I groups reported group sizes in line with study requirements. By comparison, only one out of six IY-T groups was a viable size. Weekly parent contact was adhered to across both programmes and the IY-I groups reported strong adherence to the checklists, where six of the eight groups delivered met the 80% threshold. Many IY group leaders (n = 18, 46.2%) reported regularly utilising and reviewing weekly fidelity data, six IY group leaders reported using these data occasionally and the rest (n = 15, 38%) said that they seldom or never reviewed the weekly fidelity data.

Group leader contact with parents

IY delivery guidance states that group leaders should contact parents weekly, between each group session, to check in on homework tasks and to increase motivation, by telephone call, home visit, individual session, text, letter or a combination of these methods. The E-SEE Step group leaders recorded their weekly contacts and mainly adhered to IY contact guidance (see Report Supplementary Material 1). For IY-I retention, contacts ranged from zero to nine contacts per parent, with a mean of 4.5 contacts per parent across all groups. IY-T retention contact ranged from 3 to 12 contacts per parent, with a mean of 7.2 contacts per parent across all groups.

Group leader checklists

One measure of programme fidelity was the group leaders’ adherence to the recommended vignettes or the number of vignettes in each session and the actions specified in the ‘Did I’ section of the weekly programme self-complete checklists (Table 22). For IY-I, the checklists indicate which of the programme’s vignettes are recommended as core by the developer. The IY-T checklists do not specify core vignettes, but, instead, give leaders flexibility to choose the most appropriate ones for parents, with a recommended minimum of six vignettes per session. The checklists ‘Did I’ section requests a ‘yes’ or ‘no’ response for core actions, for example ‘Did I . . . write the agenda on the board’.

TABLE 22 - Incredible Years infant programme and IY-T group adherence based on IY checklists

Site 1b completed only three sessions and so the average presented is across three sessions only.

Note

For IY-I, the total for ‘Did I’ statements differs for sites 2 and 1, as the sheet for session 7 has eight items for site 2 and 10 for site 1. For IY-T, site 1b group delivery was stopped after session 3 and adherence presented from sessions delivered is as follows: recommended vignettes, 15/28 (53.6%); and ‘Did I’ statements, 29/32 (90.6%).

Group	Recommended vignettes (IY-I, N = 43; IY-T, N = 117), n (%)	Mean number of recommended vignettes shown per session	Total number of additional vignettes shown	‘Did I’ statements (IY-I, N = 73; IY-T, N = 120), n (%)
IY-I
1a	27 (62.8)	3.4	14	72 (98.6)
1b	30 (69.8)	4.2	3	61 (83.6)
2a	42 (97.7)	5.3	24	65 (89)
2b	36 (83.7)	4.5	19	66 (90.4)
3a	38 (88.4)	4.6	4	64 (87.7)
3b	37 (86)	4.6	19	62 (84.9)
4a	37 (86)	4.6	1	65 (89)
4b	28 (65.1)	3.9	10	64 (87.7)
Average	34.4 (79.9)	4.4	11.8	65.1 (88.9)
IY-T
1a	52 (44.4)	4.3	0	88 (73.3)
1ba	15 (53.6)	5.0	0	29 (90.1)
2a	98 (83.8)	8.2	0	113 (94.2)
2b	94 (80.3)	7.8	3	110 (91.7)
3	71 (60.7)	6.5	1	86 (71.7)
4	58 (49.6)	4.9	1	92 (76.7)
Average	64.7 (55.3)	6.1	0.83	86.3 (71.9)

The majority (six out of eight) of IY-I groups met the criteria of 80% on fidelity of adherence based on self-completed checklists, with the exception of sites 1b and 4b. For IY-T, only the site 2 groups met the criteria on fidelity of adherence based on the checklists overall, the ‘Did I’ statements and vignette use. IY-I groups, on average, showed 4.4 vignettes per session, whereas IY-T groups showed an average of 6.1 vignettes per session, in line with the recommended minimum.

Quality of delivery

Parent programme implementation checklist

Average scores for both IY-I and IY-T show over 50% fidelity (63.7% and 73.6%, respectively) using the PPIC assessment (Table 23). Scores indicate that adherence was the weakest subscale for both IY-I and IY-T. Eight groups achieved a fidelity score of over 80% for certain components (e.g. site 2 quality of delivery and site 3a participant responsiveness), but only three groups achieved this level overall.

TABLE 23 - Agreed PPIC scores for IY-I and IY-T

Item 4 (‘Does the group leader encourage all parents to participate?’) was coded as not applicable for site 1a, as the item is about attending to all members of the group and there was only one parent in attendance.

Notes

The minimum total score is 18 and the maximum total score is 90.

For individual subscales: adherence – eight items, scores range from 8 to 40; quality of programme delivery – seven items, scores range from 7 to 35; and participant responsiveness – three items, scores from 3 to 15.

Group	Adherence (% for subscale)	Quality (% for subscale)	Participant responsiveness (% for subscale)	Total score (% total)
IY-I
1a	29 (72.5)	26 (74.3)	11 (73.3)	66 (73.0)
1b	17 (42.5)	27 (77.1)	9 (60.0)	53 (58.0)
2a	31 (77.5)	31 (88.6)	10 (66.6)	72 (80.0)
2b	23 (57.5)	29 (82.9)	8 (53.3)	60 (66.0)
3a	21 (52.5)	24 (68.6)	13 (86.7)	58 (64.0)
3b	13 (32.5)	23 (65.7)	7 (46.7)	43 (47.0)
4a	28 (70.0)	31 (88.6)	11 (73.3)	70 (77.0)
4b	12 (30.0)	17 (58.6)	7 (46.7)	36 (40.0)
Mean scores	21.8 (54.5)	26 (74.3)	9.5 (63.3)	57.3 (63.7)
IY-T
1a	25 (62.5)	25 (71.4)	9a (60)	59 (65.0)
1b	29 (72.4)	28 (80.0)	13 (86.7)	70 (77.0)
2a	18 (45.0)	20 (57.1)	7 (46.7)	45 (50.0)
2b	36 (90.0)	31 (88.6)	13 (86.7)	80 (88.0)
3	30 (75.0)	27 (77.1)	14 (93.3)	71 (78.0)
4	29 (72.4)	29 (82.9)	14 (93.3)	72 (80.0)
Mean scores	27.8 (69.5)	26.7 (76.3)	11.7 (78.0)	66.2 (73.6)

Session length

Session length was assessed to understand intervention dose. Sessions should run for 120 minutes when including breaks and 90 minutes without breaks. IY-I sessions ranged from 60 to 112 minutes, with a mean time of 69 minutes and, therefore, the majority of sessions were not within 10% of the expected time. IY-T sessions ranged from 92 to 116 minutes, with a mean time of 102 minutes.

Figure 7 compares adherence scores derived from independent observation (i.e. PPIC) with adherence scores from self-completed checklists. In almost all cases, self-rated adherence was higher than observed adherence, although concurrence was closer for IY-T than for IY-I.

FIGURE 7.

Comparison of PPIC and checklist adherence for IY-I and IY-T. Note that there was only one IY-T group for sites 3 and 4 (labelled accordingly). Sites 3 and 4 had two IY-I groups split as 3a and 3b, and 4a and 4b, respectively.

Participant responsiveness

Parent satisfaction: weekly forms

Brief parent satisfaction questionnaires were given to parents at the end of each IY session. Out of 374 weekly attendees, 303 evaluation forms were completed for IY-I (i.e. a 81% response rate). Out of 179 weekly attendees, 158 responses were completed for IY-T (i.e. a 88.3% response rate). A high level of parent satisfaction with the programme content and delivery was found (see Report Supplementary Material 1).

Mean satisfaction scores for IY-T are slightly higher than for IY-I. Parents indicated that they were likely to use the parenting strategies discussed in each session and felt able to participate in the group sessions (IY-I, 98.2%; IY-T, 96.7%). Of those parents who did not feel able to participate fully, reasons included unsettled children, lack of sleep, tending to their child and not contributing to the group discussion. Across both IY-I and IY-T groups, parents commonly found the following most useful: group discussion, advice from other mums, ‘knowing you’re not alone’, ‘confidence that I’m doing things right’ and group-sharing experiences. High parent satisfaction was also reported in the final end-of-programme questionnaire (see Report Supplementary Material 1).

Conclusions

The E-SEE Steps intervention is a complex intervention that combines versions of an evidence-based intervention (i.e. IY) into a tiered, proportionate universal model. The process evaluation has explored the feasibility and potential of the components as they relate to the model as a whole and, as such, does not extrapolate its findings to any stand-alone delivery of either of the group-based programmes. Despite expressed commitment by services to providing good-quality and proportionate parenting support, acceptable levels of fidelity and high levels of satisfaction for those parents who attended sessions, the process evaluation data indicate a number of barriers and challenges in the potential scale-up delivery of the E-SEE Steps intervention. Some of these barriers and challenges are specific to the intervention, which, with adaptation, could be addressed. Other barriers and challenges are inherent to the public services system within which the intervention is situated and are arguably more difficult to resolve or, at least, require more complex system accommodation. Indications of barriers to delivery existed within the pilot, and some were addressed. However other challenges, such as IY programme adaptation, may also be required, which would need buy-in from the developer. The developer was, however, amenable and flexible in revising and republishing the IY-B in the light of BFI concerns.

Introduction

The aim of this economic analysis was to assess the cost–consequences and the cost-effectiveness of the E-SEE Steps programme using evidence from the E-SEE trial.

Health outcomes were considered from parental and child perspectives and were expressed as quality-adjusted life-years (QALYs) and as other available measures of parental and child social and emotional well-being. Costs were measured in Great British pounds (GBPs) (2018–19) from a public sector, family and broader (i.e. both public sector and family) perspective. Costs and QALYs were discounted at 3.5% per annum in line with UK guidelines. 56 Cost-effectiveness results are presented for primary caregivers, children and jointly using incremental cost-effectiveness ratios (ICERs) and incremental net health benefits (INHBs). 57 Econometric modelling was used to control for patient co-variables and uncertainty was explored using alternative assumptions through scenario and sensitivity analyses.

Methodology

The health economic analysis plan is available at URL: www.york.ac.uk/healthsciences/research/public-health/projects/e-see-trial/#tab-3 (accessed 13 January 2022).

Results

Income data had an elevated degree of missingness (22%). The characteristics of the participants who informed the economic analysis can be found in Table 3.

Health outcomes

Table 29 presents mean values for each health outcome for adults and children at each available follow-up period. PedsQL-adjusted life-years and QALY values reported at each follow-up period represent those accrued since the previous time point. Figure 8 provides a detailed breakdown of adult responses to each of the PHQ-9 and EQ-5D-5L questionnaire domains.

TABLE 29 - Adult- and child-related outcomes by treatment arm

Accrued from previous time point and calculated from EQ-5D-3L scores mapped from EQ-5D-5L responses.

Accrued from previous time point, assuming that child baseline HRQoL in both arms is equal to that reported in the SAU arm at the 18-month follow-up.

Outcome	Treatment arm								Difference
	E-SEE Steps programme				SAU
	Baseline	FU1	FU2	FU3	Baseline	FU1	FU2	FU3	Baseline	FU1	FU2	FU3
Adult HRQoL
PHQ-9	3.06316	2.39439	2.48351	2.96632	2.83929	3.34286	2.78571	3.0500	0.22387	–0.94847	–0.30221	–0.08368
EQ-5D-5L	0.93585	0.94729	0.94758	0.93457	0.91457	0.91970	0.91534	0.92878	0.02127	0.02759	0.03224	0.00579
(Mapped) EQ-5D-3L	0.89386	0.91285	0.91859	0.89645	0.86446	0.87181	0.87721	0.89115	0.02940	0.04105	0.04137	0.00530
QALYsa		0.15047	0.53392	0.68113		0.14475	0.51001	0.66229		0.00572	0.02390	0.01884
Child HRQoL
PedsQL: overall				86.54722				87.16596				–0.61874
PedsQL: psychosocial				83.86168				84.37054				–0.50886
PedsQL: physical				90.22215				90.95386				–0.73171
SDQ score				9.67293				9.15094				0.52199
(Mapped) CHU-9D				0.85324				0.85933				–0.00609
PedsQL-adjusted life-yearsb		0.14482	0.50596	0.65026		0.14528	0.50847	0.65374		–0.00046	–0.00251	–0.00348
QALYsb		0.14265	0.49928	0.64194		0.14311	0.50090	0.64402		–0.00046	–0.00162	–0.00208

FIGURE 8.

Participant responses to the EQ-5D-5L questionnaire by follow-up period and treatment arm.

Primary caregivers randomised to the E-SEE Steps arm reported higher baseline PHQ-9 scores (3.06) than the SAU arm (2.84), with an average difference of 0.22 points between arms. Baseline PHQ-9 responses identified tiredness (E-SEE Steps programme 71.2% vs. SAU 69.6% reporting any problems), eating issues (E-SEE Steps programme 33.3% vs. SAU 35.7% reporting any problems) and sleeping difficulties (E-SEE Steps programme 27.7% vs. SAU 19.6% reporting any problems) as the three most common problems experienced by primary caregivers (see Report Supplementary Material 5). All subsequent follow-up periods found PHQ-9 scores to be higher for SAU participants than for E-SEE Step participants, with average differences of 0.95, 0.30 and 0.08 points at FU1 (E-SEE Steps programme 2.39 vs. SAU 3.34), FU2 (E-SEE Steps programme 2.48 vs. SAU 2.79) and FU3 (E-SEE Steps programme 2.97 vs. SAU 3.05), respectively. E-SEE Steps programme-associated reductions in depression and severity scores were concentrated at FU1 and FU2, with modest reductions in the proportion of participants reporting problems with apathy, depression and eating-related issues (see Figure 8). Average PHQ-9 scores were similar between arms by FU3, with E-SEE Steps participants seeing modest reductions in the proportions reporting problems with apathy (E-SEE Steps programme 18.8% vs. SAU 24.5%) and eating (E-SEE Steps programme 28.6% vs. SAU 35.8%), balanced with modest elevations in the proportions reporting issues with tiredness (E-SEE Steps programme 66.4% vs. SAU 58.5%) and concentration (E-SEE Steps programme 20.1% vs. SAU 13.2%).

Baseline EQ-5D-5L scores for primary caregivers were higher in the E-SEE Steps arm than in the SAU arm, with an average HRQoL score of 0.936 in the E-SEE Steps arm compared with 0.915 in the SAU arm. Increased EQ-5D-5L scores were observed at FU1 (0.947) and FU2 (0.948) compared with baseline levels (0.936) for those randomised to the E-SEE Steps programme. Scores did, however, return below baseline values at FU3 (0.935). For those participants randomised to SAU, EQ-5D-5L scores fluctuated between time points, rising at FU1 (0.920), falling at FU2 (0.915) and finally rising again at FU3 (0.929). The differential in EQ-5D-5L scores between the E-SEE Steps programme and SAU rose from a 0.021 (95% CI –0.004 to 0.046) increment at baseline to 0.027 (95% CI 0.002 to 0.052) and 0.032 (95% CI 0.001 to 0.063) at FU1 and FU2, respectively. EQ-5D-5L scores converged at FU3, with a differential score of 0.006 (95% CI –0.026 to 0.038). Results using mapped EQ-5D-3L were aligned with those observed using the EQ-5D-5L, albeit with notably lower values and increased differentials between arms, reflecting differences between the EQ-5D-3L and EQ-5D-5L value sets. Mapped EQ-5D-3L scores were higher for the E-SEE Steps programme than for SAU, with differential scores of 0.029 (95% CI –0.006 to 0.064) (E-SEE Steps programme 0.894 vs. SAU 0.864), 0.041 (95% CI 0.006 to 0.076) (E-SEE Steps programme 0.913 vs. SAU: 0.872), 0.041 (95% CI 0.001 to 0.081) (E-SEE Steps programme 0.919 to SAU 0.877) and 0.005 (95% CI –0.040 to 0.050) (E-SEE Steps programme 0.896 vs. SAU 0.891) at baseline, FU1, FU2 and FU3, respectively. E-SEE Steps programme-associated improvements in EQ-5D-5L- and mapped EQ-5D-3L-defined HRQoL were concentrated at FU1 and FU2. Improvements stemmed from reductions in the proportion of primary caregivers reporting problems with mobility and usual activities, and for reductions in the rates and severity of pain/discomfort and anxiety/depression problems reported. Given that no mortality was observed during trial follow-up, differences in mapped EQ-5D-3L HRQoL scores explain the QALY differentials observed. Those participants randomised to the E-SEE Steps programme accrued 0.006 (E-SEE Steps programme 0.150 vs. SAU 0.145), 0.024 (E-SEE Steps programme 0.534 vs. SAU 0.510) and 0.019 (E-SEE Steps programme 0.681 vs. SAU 0.662) more QALYs than SAU between baseline and FU1, FU1 and FU2 and FU2 and FU3, respectively. E-SEE Steps programme-associated QALY gain was highest between FU1 and FU2 (i.e. the periods with the highest increments in mapped EQ-5D-3L HRQoL scores compared with SAU). We found that the E-SEE Steps programme was associated with moderate gains in HRQoL, primarily at FU1 and FU2. Differences between arms were multidimensional, happening in and across a variety of health domains defined by the EQ-5D-5L and PHQ-9.

Children of primary caregivers randomised to the E-SEE Steps arm had lower PedsQL scores overall and across psychosocial and physical domains than children of primary caregivers randomised to SAU. Compared with SAU, PedsQL responses for E-SEE Steps participants had 0.619 (E-SEE Steps programme 86.5 vs. SAU 87.2), 0.509 (E-SEE Steps programme 83.9 vs. SAU 84.4) and 0.732 (E-SEE Steps programme 90.2 vs. SAU 91.0) decrements in overall, psychosocial and physical summary scores, respectively. This represented a modest reduction in parent-reported paediatric psychosocial and physical HRQoL. SDQ scores were also found to show modest decrements in child outcomes, with E-SEE Steps primary caregivers reporting a 0.52 increase (E-SEE Steps programme 9.67 vs. SAU 9.15) in child SDQ distress and impairment scores compared with SAU. At FU3, mapped CHU-9D scores were lower in the E-SEE Steps arm than in the SAU arm, with an associated 0.006 (95% CI –0.005 to 0.018) (E-SEE Steps programme 0.853 vs. SAU 0.859) decrement in HRQoL. Resultantly, those randomised to E-SEE Steps arm accrued 0.0005 (E-SEE Steps programme 0.1427 vs. SAU 0.1431), 0.002 (E-SEE Steps programme 0.499 vs. SAU 0.510) and 0.002 (E-SEE Steps programme 0.642 vs. SAU 0.644) fewer QALYs than SAU between baseline and FU1, FU1 and FU2 and FU2 and FU3, respectively. PedsQL-adjusted life-years were higher than mapped CHU-9D QALY estimates and had larger differences between arms. Children of participants randomised to the E-SEE Steps programme accrued 0.0005 (E-SEE Steps programme 0.1448 vs. SAU 0.1453), 0.003 (E-SEE Steps programme 0.506 vs. SAU 0.508) and 0.004 (E-SEE Steps programme 0.650 vs. SAU 0.654) fewer PedsQL-adjusted life-years than children of participants randomised to SAU between baseline and FU1, FU1 and FU2 and FU2 and FU3, respectively. We found no evidence of the E-SEE Steps programme enhancing child HRQoL compared with SAU.

Sensitivity analyses

Figures 9 and 10 examine the average programme costs and measures of cost-effectiveness with respect to the number of programmes trained practitioners may be expected to deliver (between programme), and the number of attendees group sessions may be expected to reach (within programme), respectively. As the trial ran only a single E-SEE Steps programme, between-programme fixed costs (training costs, materials costs, etc.) were distributed over a single programme only. In addition, group session within-programme fixed costs and between-programme fixed costs were distributed over a relatively low and highly variable IY-I and IY-I course attendance. Average within-trial programme costs may, therefore, not accurately represent those borne if the E-SEE Steps programme was to be introduced nationwide, where more programmes may be run by trained practitioners for alternative group sizes.

FIGURE 9.

Sensitivity of IY programme costs and measures of cost-effectiveness with respect to the number of future programmes delivered.

FIGURE 10.

Sensitivity of IY programme costs and measures of cost-effectiveness with respect to the number of IY group session attendees.

Figure 9 shows that average programme costs, and corresponding ICERs, fall at a decreasing rate with respect to the number of programmes delivered. In this costing, average programme costs were sensitive to initial expansions in programme delivery, as within-trial values (£486 per family) reduced to £342, £304 and £284 when two, three and four programmes were delivered, respectively. Cost-effectiveness was also particularly sensitive to initial expansions in the number of programmes delivered, with corresponding ICERs of approximately £17,680, £16,424 and £15,797, respectively. Average intervention costs plateaued within £235–250 and measures of cost-effectiveness within £14,300–14,700 for between 10 and 20 deliverable programmes.

Figure 10 shows positive but diminishing reductions in average programme costs and measures of cost-effectiveness with respect to the number of IY-I and IY-T group session attendees. Results were particularly sensitive to changes in attendances, with average programme-specific costs reducing from £1423 per family when provided on an individual basis to £711, £475 and £356 per family when two, three and four caregivers attend group sessions. Corresponding cost-effectiveness estimates were an ICER of £52,110 for individual course attendance and ICERs of £29,137, £21,479, £17,650 for two, three and four attendees, respectively. Average intervention costs plateaued around £118–178 and ICERs around £9993–11,906 for attendances of between 8 and 12 participants.

Discussion

Was the E-SEE Steps programme effective?

The E-SEE Steps programme did not enhance child social and emotional well-being at 20 months of age when compared with the control arm that received SAU. Although the finding was borderline statistically significant, the analysis of the primary outcome (i.e. social and emotional well-being of the child assessed on the ASQ:SE-2) favoured the control group. Sensitivity analyses of the primary outcome diminished the strength of the signal, although results remained in favour of the control group (see Table 5). What is concerning, however, is that both arms showed declining child well-being by our final time point (FU3) on the ASQ:SE-2, with the intervention arm declining more, on average, than the control arm.

The results did not differ depending on how educated parents were, whether or not their child was their firstborn or whether their child was a boy or girl. However, subgroup analysis showed weakly significant evidence of an interaction with treatment for site (p = 0.06). The site-level control groups were too small for additional statistical analysis, but do not suggest better scores in the treatment groups than in the control groups for any of the sites.

The analysis of the key secondary outcome (i.e. parent depression levels as assessed on the PHQ-9) demonstrated the opposite (non-significant) trend (i.e. a moderate non-statistically significant signal in favour of the intervention, with sensitivity analyses slightly strengthening the signal).

Follow-up 1 (2 months post baseline) is when the main difference in arms arises for the ASQ:SE-2 and PHQ-9. The children in the intervention arm demonstrated poorer well-being on the ASQ:SE-2 than children in the control arm at this point. FU1 occurs after the universal dose of the IY-B. We considered whether or not the IY-B led to intervention parents questioning their perceptions and how they interpreted their child’s social and emotional well-being when completing the ASQ:SE-2. Some parents, when interviewed, said that they found the book too ‘difficult’ to understand, which may have left them feeling ‘inadequate’ in their abilities to parent, whereas others may have been more sensitised to their child’s well-being and, therefore, more critical of their developmental progress than they may have been if they had not received the book. We do not have the evidence to support these hypotheses, and we cannot be sure that all intervention families read the book.

Follow-up 1 (2 months post baseline) is also when the largest difference between arms occurred for the PHQ-9, with parents in the intervention arm demonstrating improved depression levels compared with parents in the control arm. This runs counter to the pattern expected, as we would expect the ASQ:SE-2 and PHQ-9 to demonstrate similar change patterns based on existing literature. 83,84 We cannot explain this inverse patterning; however, one possibility lies in the psychometric properties of the ASQ:SE-2, which we intend to explore further (the version used was only just developed at the time the trial started and has not been validated in the UK). A previous systematic review85 of the worldwide evidence on ASQ:SE-2 highlighted that more research is needed to determine its measurement properties in an English sample. There is a lack of research on the reliability and validity of the specific version and associated age intervals administered in the E-SEE trial. The cut-off scores and norms are derived from data on infants and toddlers in the USA.

There was no evidence of a significant difference between trial arms on any of the other secondary parent-reported measures, that is the PSCO survey, HRQoL (using the EQ-5D-5L) and co-parent levels of depression (using the PHQ-9), all of which were collected at all four time points. The same pattern was true for measures of child behaviour (using the SDQ), child cognitive development (using the PedsQL) and parent–child interaction (using the MPAS and PPAS), which were collected at final time point only. We also did not find evidence of differences across arms with regard to how children were fed (e.g. breastfed, bottle fed or mixed feeding, and duration of feeding).

Similar to parent-reported data, the independent observation of dyadic synchrony for the primary parent–child dyad, using the Infant CARE-Index at each time point, demonstrated no difference between study arms. Infant CARE-Index data are a subset of these data from 119 families who consented to be video-recorded and had complete data across all four time points.

Considerations of why the E-SEE programme did not demonstrate effectiveness

Can Incredible Years be delivered as a proportionate universal model, and what are the organisational, or systems-level, barriers to and facilitators of delivering in this way, with fidelity?

We found a number of barriers and challenges to the delivery of the E-SEE Steps programme through our discussions with key service stakeholders and parents, which directly relate to poor take-up, and retention, of parents to the group programmes, and also inform potential future delivery of the E-SEE Steps programme. Two overarching areas will be discussed, comprising challenges for the actual intervention and challenges for the system that delivers it.

Is Incredible Years, and the proposed delivery model, cost-effective in enhancing child social and emotional well-being at 20 months when compared with services as usual?

The aim of the economic analysis was to assess the cost–consequences and the cost-effectiveness of the E-SEE Steps model (see Chapter 5).

Limitations

Several limitations to the trial and data have been acknowledged above and are, in summary, listed below:

• Our internal pilot became external because of design/intervention changes. We had to rework our budget and timeline while maintaining feasible research questions. We recalculated the sample size and had to forgo our planned analysis of effectiveness of each level of dose within the main trial in favour of exploring whether or not the whole E-SEE Steps model was effective.

• We failed to meet the trial recruitment target of 606 participants; however, owing to the high participant retention rate to the trial (94%), low levels of missing data, range of 95% CIs and sensitivity analyses, we are confident in our conclusions that E-SEE model is not effective in enhancing child social and emotional well-being, but there was low take-up of the group-based programmes.

• Implementation issues, specifically service engagement processes, may have resulted in too few participants receiving the intervention. This could be improved in future delivery, but would require additional, or potentially different, service resource/training. Owing to the low take-up, per-protocol and complier-average causal effect analysis were rejected, as the pseudo-control groups were too small. The 5 : 1 random allocation ratio also had an impact on this decision.

• We do not know if our sample is representative at local or national levels; however, as we only had 11 (3%) participants who reported not being in a relationship at baseline this suggests not. Our previous work with E-SEE Step pilot data, and the similar ENRICH (EvaluatioN of wRaparound in Ireland for CHildren and families) study in Ireland,104 demonstrates the many difficulties in assessing representativeness of community samples (see Appendix 10 for substudy information).

• IY has been delivered in many countries and is, typically, delivered with interpreters where needed (as in E-SEE Steps trial and our other IY trials26,105–107). However, within the E-SEE Steps trial, we do not have a large enough, or diverse enough, sample to explore cultural variation or relevance of the intervention in detail.

• The trial timeline had an impact on timings of some group session deliveries and thereby services had reduced time to engage with families.

• We do not know the extent of families’ engagement with the IY-B after receipt. Some families may not have read any of the IY-B, whereas others may have digested every page. Qualitative evidence suggests that families found it difficult to access the content without support/facilitation.

• Service/contextual shifts across the duration of the trial, including commissioning changes for health, significant reductions in budgets and high staff turnover, meant that the recommended ‘dry run’ of group delivery to iron out any teething issues and to give facilitators an opportunity to practise new skills prior to research group delivery did not go ahead in three out of the four sites. In the one site that did have a dry run this was reduced to four summary sessions, as opposed to the whole programme. Therefore, leaders were not as experienced in engaging families in IY and leading IY-I and IY-T groups as they could have been.

• Supervision during delivery enhances performance, skills and confidence; however, supervision was conducted remotely and many access issues were experienced because of an inability of organisational IT systems to communicate with each other (e.g. firewalls prevented communication), or a lack of appropriate IT equipment.

• Co-parent engagement was limited because primary carers were the initial gatekeepers (i.e. co-parents could only be invited via the primary parent). In addition, services were not able to deliver flexible groups to mainly working co-parents (e.g. during the evening or weekend). For these reasons we were unable to look at the relationship between co-parent and child outcomes, such as depression and well-being, respectively.

• There were over the robustness of some measures, including the primary outcome measure (due to lack of UK validation), and the limited age range on other measures, with reliance on older age versions for psychometric information.

• Only a subset of data for the Infant CARE-Index analyses was used, as not all families consented to be video-recorded and, owing to research staff capacity and resource issues, we could include only data from families with complete data at four time points.

• Parent and child service use is reliant only on parent report because of several issues, over a prolonged period of time, when requesting access to Health Episodes Statistics data. In December 2015, an initial application to use health service data for the cohort was submitted to the Health and Social Care Information Centre (now NHS Digital). During this time, the Health and Social Care Information Centre migrated its data access request online and despite contact at least twice a month there was no response to this application. Eventually, it emerged that the application had been lost in the changeover to an online system and to NHS Digital and a new application had to be submitted in 2017. We were allocated a case officer and a number of changes were requested for the consent forms and website. Further changes were then required because of the introduction of General Data Protection Regulation and further requested in autumn 2019 prior to submission to the Independent Group Advising on the Release of Data. By this time, costs had increased substantially to well beyond the budget allocated and there was limited staff support and time left for the ongoing work required. It was decided, therefore, to use self-reported data at this point and explore further funding at a later date to explore the routinely collected data.

• A limitation of all trials is potential control arm bias. The very fact that control families were randomised and told that they had been allocated to the SAU arm could have led to bias, for example through control families taking up more/less services or answering/responding to study assessments differently. Alternative designs, such as the trials within cohorts,108 using random selection to the intervention rather than random allocation may be useful in the future.

• The economic analysis of child outcomes relied on data from FU3 only. By assuming that baseline child HRQoL in both arms were aligned to the SAU control arm at FU3, the analysis failed to account for potential baseline differences in HRQoL or the underlying dynamics of the intervention’s impact on infants over time (as conducted with adult HRQoL). Estimated differentials in child QALYs were, therefore, susceptible to bias.

Strengths

This trial has several strengths as alluded to throughout the discussion. These are summarised below:

• The proportionate universal trial design arguably reflects real-world intervention delivery where services may adopt different approaches or levels of intervention relative to the stages of child development and/or for differing levels of need [e.g. the Health Visiting model – Universal, Universal Plus and Universal Partnership Plus (URL: https://www.gov.uk/government/publications/healthy-child-programme-0-to-19-health-visitor-and-school-nurse-commissioning; accessed 22 March 2022)]. A RCT of one standalone intervention does not consider doses of one or more intervention for families with differing needs at different times.

• We included an in-depth process and economic evaluation.

• The trial was flexible with regard to organisational/site-specific delivery models (i.e. delivery by health or council staff, or both, was acceptable). The areas were also new to IY and so there was no risk of contamination.

• High retention to trial (94%) mitigated the low recruitment figures and enabled us to retain power to answer the main research question with confidence. We also had very few missing data.

• The trial included both parent report and independent observational measures.

• Although there may be some issues with the robustness of some parent-report measures, we are confident, because of our systematic review outcomes and previous research,39,40,109 that those administered were the most appropriate, given the poor choices available in some outcome domains. Furthermore, we included an independent observation measure in addition to parent report. Our reviews have highlighted weaknesses in certain measures, which we will seek to explore or address.

• Our PAC helped in measure selection and were very helpful in various aspects of the trial (see Chapter 2, Patient and public involvement).

• There was no major imbalance between arms at baseline, in terms of either covariates and baseline outcome scores, and we conducted sensitivity analyses to explore these data where there were uncertainties.

• We included substudies that broadened the utility of the research with regard to process, representativeness of sample, predictors of ED and hospital admissions, and father inclusion.

• We have very rich data, some of which will be accessible to other researchers.

• We have consent to follow up parents in the future and will be able to compare arms because control SAU families did not receive the intervention.

Future research

We have identified the following areas for future research:

• The E-SEE Step trial was funded in response to a National Institute for Health Research Public Health Research call to explore the effectiveness of parenting programmes to enhance child social and emotional well-being in children aged < 2 years. This evidence gap still exists and needs exploring further. Other potential alternatives to parenting programmes in early prevention are now gathering an evidence base, but definitive RCTs are needed. For example, video interactive guidance110 demonstrated promise in improving parental sensitivity to their preterm infants in a pilot RCT. 111 Video interactive guidance is used in the UK’s Children and Young People’s Improving Access to Psychological Therapies112 for behavioural and emotional problems in the 0- to 5-year age range. Video interactive guidance is delivered in a targeted way on a 1-to-1 basis, thereby it is potentially more costly than a group-based parenting programme. Continuity of care, whereby women have a named midwife throughout pregnancy, birth and beyond, is building an evidence base and The NHS Long Term Plan113 sets specific targets to ensure that 75% of women from black and minority ethnic groups and those living in social deprivation receive continuity of care from a known midwife by 2024. However, although a RCT in Australia114 demonstrated that, for women at low risk of medical complications, caseload midwifery increased women’s satisfaction with antenatal, intrapartum and postpartum care, there remains relatively little information about the effects of midwife-led continuity models of care on mothers’ and babies’ health and well-being in the longer postpartum period. 115

• Future studies should aim to develop and test a ‘pre-intervention’ component of implementation strategies devoted to identification and engagement of parents and co-parents, particularly those who may have low mood, as this influences (in this case, low) take-up to the group-based programme, which is in opposition to the high retention rate of parents to the trial. This component should pay particular attention to engaging co-parents too.

• It is clear from this study and our own and others’ reviews of measurement tools that there is a paucity of appropriate and robust measures to explore domains of interest across infancy and toddlerhood. This is particularly true for ‘attachment’ in 0- to 12-month-olds, as highlighted by NICE. 116 We aim to explore some measures, and possibly their psychometric properties or relationships with other measures further, as follows:

  • Although the primary outcome ASQ:SE-2 is widely used in the UK, it has not yet been validated in the UK context. It did not perform as expected with the PHQ-9 (which has been validated in the UK). The Infant CARE-Index has also not yet been validated in the UK. The SDQ and PSOC survey have been used, but both have been validated on older samples. We will explore psychometric properties of the above measures.

  • We will complete the coding all videos with the Infant CARE-Index, for example all pilot data and remaining main trial data (current analyses included parents who consented and with complete data at all four timepoints only). This would also enable more in-depth exploration of the relationship between ASQ:SE2 and Infant CARE-Index scores, which we would expect to correlate, plus subgroup analyses.

  • During the trial we identified a gap for a UK short parent–child ‘bonding’ measure to be used as a screener owing to the shortage of appropriate measures. A measure has been co-produced with service managers and health visitors, with parent input, and piloted. Results are promising, with some additional validation work planned.

• We will explore the E-SEE Steps sample with regard to ‘representativeness’, which will build on our pilot work117 that looks at our UK sample and an Irish sample, and consider how representativeness can be assessed. Preliminary findings suggest that it is extremely difficult to assess at both a local and national level, which casts doubt on trial results and generalisability of findings.

• We will compare any differences in process, particularly around pre-intervention engagement strategies by services, in the E-SEE Steps programme and the similar Irish ENRICH study. 104

• We will perform a meta-analysis combining the results from the pilot study (pre-design changes) with the results from the full trial, limiting the outcome to FU1. This will enable us to explore the universal IY-B outcome with increased power.

• Our service use data are based on parent report only because of issues with accessing routine data (see Limitations). It is important to check that parent report is accurate and we will re-apply to access Health Episode Statistics data to assess this.

• Longer-term follow-up is needed when index children enter school to assess any differences between arms on the primary outcome (i.e. social and emotional well-being), including child behaviour, possibly through school records. In addition, parental mental health could be assessed via a postal survey. We have consent for future contact, and participants in our SAU arm did not receive the trial intervention and would not be eligible in any future delivery in the site (because of age appropriateness), which gives an opportunity for longer follow-up comparisons. Parenting programmes typically have an impact on parenting behaviour (proximal) before child outcomes (distal). This makes sense, as it is the parent behaviour that needs to change to influence child outcomes. The IY BASIC programme for parents of children aged ≥ 3 years has successfully demonstrated parent, and child, outcome change post intervention in ‘stand-alone’ intervention RCTs. 105–107 As the E-SEE Steps intervention is a long-term intervention, that is, longer than a single/stand-alone parent programme (with one to four possible doses over an 18-month period dependent on the level of family need), and with more time for behaviour change to become embedded, we would have expected some changes, or trends, in parenting and subsequent child well-being by the end of the trial.

Conclusions

Contributions of authors

Tracey Bywater (https://orcid.org/0000-0001-7207-8753) (chief investigator) was involved in the following: conceptualisation, methodology, validation, investigation, writing (original draft, review and editing), supervision, project administration and funding acquisition.

Vashti Berry (https://orcid.org/0000-0001-6438-3731) (process evaluation lead and co-applicant) was involved in the following: conceptualisation, methodology, validation, formal analysis, investigation, writing (original draft, review and editing), supervision and funding acquisition.

Sarah Blower (https://orcid.org/0000-0002-9168-9995) (trial manager) was involved in the following: conceptualisation, methodology, validation, investigation, data curation, resources, writing (original draft, review and editing), supervision and project administration.

Matthew Bursnall (https://orcid.org/0000-0002-6519-3558) (trial statistician) was involved in the following: methodology, validation, formal analysis, data curation, writing (original draft and review) and editing.

Edward Cox (https://orcid.org/0000-0001-8981-0699) (health economist) was involved in the following: validation, formal analysis, data curation and writing (original draft, review and editing).

Amanda Mason-Jones (https://orcid.org/0000-0002-4292-3183) (substudy 1 lead and co-applicant) was involved in the following: conceptualisation, methodology, validation, investigation, writing (original draft, review and editing) and funding acquisition.

Sinéad McGilloway (https://orcid.org/0000-0003-0988-3201) (substudy 3 lead and co-applicant) was involved in the following: conceptualisation, methodology, validation, writing (original draft, review and editing) and funding acquisition.

Kirsty McKendrick (https://orcid.org/0000-0003-0497-3582) (trial manager) was involved in the following: methodology, validation, investigation, data curation, writing (original draft, review and editing), supervision and project administration.

Siobhan Mitchell (https://orcid.org/0000-0002-4085-3898) (research associate, process evaluation) was involved in the following: methodology, validation, investigation, formal analysis, data curation, writing (original draft, review and editing) and project administration.

Kate Pickett (https://orcid.org/0000-0002-8066-8507) (co-applicant) was involved in the following: conceptualisation, methodology, writing (review and editing) and funding acquisition.

Gerry Richardson (https://orcid.org/0000-0002-2360-4566) (health economic lead and co-applicant) was involved in the following: conceptualisation, methodology, validation, writing (review and editing), supervision and funding acquisition.

Kiera Solaiman (https://orcid.org/0000-0003-2244-5345) (research assistant) was involved in the following: investigation, data curation, writing (review and editing) and project administration.

M Dawn Teare (https://orcid.org/0000-0003-3994-0051) (lead statistician and co-applicant) was involved in the following: conceptualisation, methodology, validation, formal analysis, writing (original draft, review and editing) and funding acquisition.

Simon Walker (https://orcid.org/0000-0002-5750-3691) (health economist and co-applicant) was involved in the following: conceptualisation, methodology, validation, writing (review and editing), supervision and funding acquisition.

Karen Whittaker (https://orcid.org/0000-0002-3493-9396) (substudy 2 lead and co-applicant) was involved in the following: conceptualisation, methodology, formal analysis, investigation, writing (original draft, review and editing), supervision, project administration and funding acquisition.

All authors participated in the interpretation of the findings, contributed ideas and were involved in critically revising this report for important intellectual content. All authors read and agreed the final report. Author contribution statements are provided in line with the CRediT statement guidelines.

Publications

Bywater T, Berry V, Blower SL, Cohen J, Gridley N, Kiernan K, et al. Enhancing Social–Emotional Health and Wellbeing in the Early Years (E-SEE): a study protocol of a community-based randomised controlled trial with process and economic evaluations of the incredible years infant and toddler parenting programmes, delivered in a proportionate universal model. BMJ Open 2018;8:e026906.

Blower SL, Gridley N, Dunn A, Bywater T, Hindson Z, Bryant M. Psychometric properties of parent outcome measures used in RCTs of antenatal and early years parent programs: a systematic review. Clin Child Fam Psychol Rev 2019;22:367–87.

Bywater T, Gridley N, Berry V, Blower S, Tobin K. The Parent Programme Implementation Checklist (PPIC): the development and testing of an objective measure of skills and fidelity for the delivery of parent programmes. Child Care Pract 2019;25:281–309.

Candlish J, Teare MD, Cohen J, Bywater T. Statistical design and analysis in trials of proportionate interventions: a systematic review. Trials 2019;20:151.

Gridley N, Blower S, Dunn A, Bywater T, Bryant M. Psychometric properties of child (0-5 Years) outcome measures as used in randomized controlled trials of parent programs: a systematic review. Clin Child Fam Psychol Rev 2019;22:388–405.

Gridley N, Blower S, Dunn A, Bywater T, Whittaker K, Bryant M. Psychometric properties of parent-child (0-5 years) interaction outcome measures as used in randomized controlled trials of parent programs: a systematic review. Clin Child Fam Psychol Rev 2019;22:253–71.

Whittaker K. Public Health and the Life Course: Supporting Parents and Families. In Cowley S, Whittaker K. editors. Community Public Health in Policy and Practice. London: Elsevier; 2020. pp. 159–80.

Blower SL, Berry VL, Bursnall MC, Cohen J, Gridley N, Loban A, et al. Enhancing Social–Emotional Outcomes in Early Years (E-SEE): randomised pilot study of Incredible Years infant and toddler programs. J Child Fam Stud 2021;12:1–7.

Gridley N, Hickey G, Bywater T, McGilloway S, Blower S, Whittaker K, Berry V. The challenges of assessing sample representativeness within community-based evaluations of parenting programmes delivered in England and Ireland [published online ahead of print August 24 2021]. Health Soc Care Community 2021.

Bywater T, Barry V, Blower S, Bursnall, Cox E, et al. A randomized controlled trial of a proportionate universal parenting program delivery model (E-SEE Steps) to enhance child social-emotional wellbeing. PLoS ONE 2022;17:e0265200.

Data-sharing statement

All data requests should be submitted to the corresponding author (TB) and/or third author (SB) for consideration from September 2021. Access to available anonymised data may be granted following completion of a proforma for review by the data access team (comprising members of the TMG).

Disclaimers

This report presents independent research funded by the National Institute for Health and Care Research (NIHR). The views and opinions expressed by authors in this publication are those of the authors and do not necessarily reflect those of the NHS, the NIHR, the PHR programme or the Department of Health and Social Care. If there are verbatim quotations included in this publication the views and opinions expressed by the interviewees are those of the interviewees and do not necessarily reflect those of the authors, those of the NHS, the NIHR, the PHR programme or the Department of Health and Social Care.

Copy of the Patient Health Questionnaire-9 items

Residual plots for primary outcome

FIGURE 12.

Residuals from the primary analysis model.

FIGURE 13.

ASQ:SE-2 scores at each follow-up point. (a) FU1; (b) FU2; and (c) FU3.

FIGURE 14.

Residuals from the primary analysis model of ASQ:SE-2 after square root transformation.

FIGURE 15.

Residuals from the primary analysis model of ASQ:SE-2 after calculating difference from baseline.

FIGURE 16.

Residuals against fitted values for the ASQ:SE-2 primary analysis model.

Residual plots for primary parent Patient Health Questionnaire-9 items

FIGURE 24.

Residuals from planned PHQ-9 analysis.

FIGURE 25.

Residuals from unplanned analysis of PHQ-9 after calculating difference from baseline.