IMPRoving Outcomes for children exposed to domestic ViolencE (IMPROVE): an evidence synthesis

Defining exposure to domestic violence and abuse

Early research characterised children as ‘witnessing’ DVA, which implied the direct observation of violence between adults. 7 The term ‘exposure’, which is more common in recent studies, is broader in scope and includes instances in which children see, hear, are directly involved in (e.g. attempt to intervene), experience the aftermath of assaults that occur between their caregivers, or are otherwise aware of abuse or threatening behaviour between adults. 2,8 It also pertains to those children who may ostensibly be unaware of the occurrence of abuse but may be affected indirectly by disrupted family processes and poor parental mental health. 8 This broader concept of exposure is, to some extent, reflected in the UK’s definition of ‘significant harm’, which sets out the legal basis or threshold criteria on which a family court can make a care or supervision order that justifies compulsory intervention in family life in the best interests of children,9 and which was updated in 200210 to include the ‘impairment from seeing or hearing the ill treatment of another person’.

We do not yet know whether or not different forms of exposure have differential effects on children’s adjustment and other outcomes. Research and service delivery tend to treat exposure to DVA as a ‘homogenous unitary’ phenomenon (see Gonzalez et al. 11 for an exception), despite differentiation in the adult-focused literature between different types of partner violence that are marked by a greater or lesser degree of coercive control or bidirectionality of abusive behaviour between adults. 12

Prevalence of children’s exposure to domestic violence and abuse

A UK national study of children’s mental health carried out in 2004 found that, based on parent reports, 4% of children had witnessed severe abuse putting it as the most frequently reported type of childhood trauma for children aged < 16 years. 13 A more recent large-scale study carried out by the National Society for the Prevention of Cruelty to Children (NSPCC)14 estimated that 3.3% of children aged < 11 years had witnessed at least one incident of domestic violence or threatening behaviour in the preceding 12 months, as had 2.9% of young people aged 11–17 years. A total of 12% of children aged < 11 years and 18.4% of young people aged 11–17 years had witnessed at least one incident of domestic violence or threatening behaviour in their lifetime.

The most frequent type of reported DVA documented by the survey entailed one parent throwing or breaking things in the context of an argument, although consistent with the earlier survey, 3.8% of children surveyed had witnessed severe abuse (kicking, choking and beating up). 13,14

UK prevalence figures are similar to those in other high-income countries that have measured children’s exposure to DVA. For example, the US National Survey of Children’s Exposure to Violence15 reported that among the target sample of > 4500 children, approximately one-sixth had witnessed an assault between parental partners in their lifespan, and that 6% had witnessed an assault between parental partners in the past year.

Risk and protective factors for exposure to domestic violence and abuse

Numerous risk factors are associated with DVA perpetration and victimisation at individual, family, community and societal levels (see MacMillan and Wathen16 for an overview). These include parental history of childhood maltreatment, drug and alcohol use, unemployment, young age, presence of mental health issues, social isolation, poverty and associated household factors (e.g. overcrowding). 16–19 The presence of these risk factors in families with interparental violence (IPV) creates an environment of pervasive adversity for the child, in which multiple stressors can accumulate and impinge upon the child’s development. 20

Exposure to multiple forms of abuse

Children exposed to one incident of violent victimisation are likely to be repeatedly exposed to the same type of violence21 and are at greater risk of experiencing multiple different types of victimisation, known as poly-victimisation. 22 For example, Hamby et al. 23 demonstrated a close association between witnessing partner violence and experiencing other types of family violence. Of the children and young people who had witnessed DVA in the past year, 33.9% had experienced other types of maltreatment; the proportion for lifetime exposure to DVA was more than half (56.8%).

Serious case file reviews both in the UK and in the USA further highlight the co-occurrence between DVA and child maltreatment, with DVA noted in one-third to half of cases in which children were killed or seriously harmed,24,25 and, although the risk of maltreatment increases with the severity of abuse, less serious forms of violence (e.g. pushing and shoving) may also increase the risk of maltreatment. 26

Impact of exposure to domestic violence and abuse on children’s health and well-being

In addition to the risk of physical harm, exposure to DVA during childhood and adolescence increases the risk of negative mental health outcomes, with a moderate to strong association between children’s exposure to DVA and internalising symptoms (e.g. anxiety, depression), externalising behaviours (e.g. aggression) and trauma symptoms. 2,3,5 Children exposed to DVA are two to four times more likely to exhibit clinically significant problems than children from non-violent homes. 3 There are also associations between children’s exposure to DVA and disrupted social development, poor academic attainment, engagement in risky health behaviours and other physical health consequences,3,27,28 although uncertainty over the magnitude and consistency of detrimental effects on these domains of children’s functioning remains. Evidence relating to children’s emotional and behavioural development is less equivocal. 29 Several studies suggest that boys and girls may be affected differently by exposure,2,4 but the moderating role of sex is uncertain. 30,31

Exposure to DVA in childhood is associated with negative outcomes in adulthood, such as mental health problems, conduct disorder (CD) and criminal behaviour, as well as DVA victimisation and perpetration. 31–33 There is considerable evidence to suggest that the risk of negative outcomes in adulthood is mediated by adjustment difficulties, particularly behaviour problems that develop during childhood and adolescence. 34

Although exposure to DVA is undoubtedly a significant stressor in children’s lives, there is considerable variation in their reactions and adaptation following exposure,35,36 with several studies indicating that a significant minority of children exposed to DVA are not adversely effected in terms of behavioural or health outcomes, reflecting resilience to this adverse family context, at least in the short term. Factors such as maternal mental health, the quality of parenting and children’s perceptions of abuse potentiate or mitigate the risk of poor outcomes for children exposed to DVA. 35,36 There is also heterogeneity in adult outcomes of exposed children; most will neither suffer long-term harm, nor perpetrate or experience DVA in adulthood. 37

Evidence of effectiveness

The most direct way of preventing the negative consequences of DVA for children is to prevent or end the violence itself. 38 Systematic reviews highlight the lack of evidence for effective interventions to prevent the initiation of DVA and, therefore, to prevent children’s exposure to it. 39,40 There is some evidence that community-based advocacy is a promising strategy for reducing the frequency and severity of DVA (especially physical violence) once it has taken place, but replication of these results is needed. 40 Furthermore, few studies examining the effectiveness of advocacy quantify the effects of reduced DVA on children’s health and well-being (see Chapter 3 for exceptions).

The most common approach to preventing or reducing the harm associated with DVA is to provide services directly to children and their parents once abuse has taken place. 41 Based on frequently used definitions set out by the Institute of Medicine,42 interventions offered after exposure to DVA can be categorised as selective or indicated prevention (preventing full-blown disorder) or as treatment. Selective preventative interventions are offered to all exposed children or to parents of exposed children based on the increased risk of maladjustment, irrespective of children’s clinical profiles. Indicated preventative interventions are targeted at children showing signs or symptoms of mental, emotional or behavioural disorders who do not meet diagnostic criteria for these disorders. Interventions for children with diagnosed conditions are known as treatments. In this report we refer to all interventions aimed at exposed children as targeted; where appropriate, we distinguish between the populations of the children who were specifically targeted.

Two systematic reviews have identified a range of (often) complex targeted interventions delivered solely to children, or to children and their non-abusive parents following exposure to abuse. Rizo et al. 41 categorised interventions evaluated in 31 studies into four categories: (1) counselling/therapy; (2) crisis/outreach; (3) parenting; and (4) multicomponent. They noted that, despite the different approaches of the programmes, they shared common aims for children (e.g. learning about and dealing with IPV, enhancing communication, reducing psychological distress and enhancing well-being) and non-abusing parents (developing knowledge of the impact of DVA on children, developing parenting skills, increasing self-esteem and parenting efficacy). Owing to the methodological shortcomings of the studies reviewed, the authors were not able to draw conclusions about the effectiveness of specific interventions, and recommended that studies seek to recruit larger samples, account for missing data, follow up participants beyond the end of the intervention and use randomised experimental designs with statistical techniques to account for the correlated nature of the data.

A second review, conducted by the British Columbia Centre of Excellence for Women’s Health (BCCEWH),43 built on the work by Rizo et al. ,41 although it also included qualitative evaluative studies and grey literature. The authors categorised interventions into seven categories to reflect the therapeutic technique, the target population and whether the intervention was single- or multicomponent. This review concluded that there was moderate to strong evidence that psychotherapeutic interventions delivered to mothers and children improved child outcomes, moderate evidence for psychoeducational interventions delivered to children, moderate evidence for parenting-focused interventions, and mixed evidence for psycho-educational interventions delivered to mothers and children. However, these conclusions were founded mostly on non-experimental research. It is also notable that of the studies identified, only three were conducted in the UK (two peer-reviewed studies; one study in the grey literature). Nevertheless, the findings of this review informed the recent National Institute for Health and Care Excellence (NICE) guidance on DVA. 44

Stakeholder views on targeted interventions

Research conducted in the UK has suggested that children most frequently report that they want to feel safe and that they want someone to talk to about their experiences,45–48 identifying talking as a strategy to reassure them that they are not alone. 45 In these earlier reports, children often identified their mothers as the person to whom they most want to be able to talk and the person from whom they most want support,47 although they also felt reticent about sharing their experiences owing to concerns that they may upset and burden their parent. 46,47 Likewise, it was noted that mothers may be reluctant to talk to children, thereby creating a ‘conspiracy of silence’ in families affected by DVA. 49,50 In an evaluation of materials designed to assist mothers and their children to rebuild their relationships in refuges (shelters) and community-based settings in the aftermath of DVA (discussed in detail in Chapter 4), mothers and children described the positive benefits of engaging in shared activities, including spending time together, and of enhanced communication, both in general and with regard to experiences of abuse. 49,51 Children also identified teachers, family members and peers as people to whom they would talk, and several evaluations highlighted the significant role that specialist domestic abuse workers have in helping children to understand their experiences. 49

A UK Delphi consultation involving just under 300 participants52 sought consensus among survivors of child sexual abuse, DVA, rape and sexual assault, as well as among experts in the field, on an effective mental health service response for people affected by these issues. With regard to interventions for children exposed to DVA, a broad range of approaches was viewed as helpful in reducing the harms associated with DVA, including play therapies, attachment-based approaches, cognitive–behavioural therapy (CBT) for older children, child psychotherapy, eye movement desensitisation and reprocessing, and, in some instances in which the abusive party remains in the family setting, family therapy. Approaches that were deemed to be inappropriate for this group were mediation/reconciliation, joint therapy with an abuser and cognitive approaches that are not suited to the cognitive maturation of the child. The inclusion of the abusive parent or party in therapeutic interventions represented a key area of disagreement among respondents. With regard to the process of delivering services, professionals and survivors agreed that the choice of therapeutic intervention for children should be needs-led and guided by the age and maturity of the child, their individual experiences and the degree of victimisation, and that children should be able to access interventions independently of their parents and their parents’ therapeutic requirements.

UK policy context

Government policy in England and the devolved nations recognises the impact of DVA on children and emphasises the need for support for those who are at risk of, or who have experienced, DVA. 53–55 English policy places a particular focus on prevention and early intervention, whereas other nations explicitly recognise the need for targeted services for children once they have been exposed to DVA. Overall, national policy has been largely focused on the response to children by the health and social care sectors. In 2011, a government-commissioned review of child protection in the UK56 emphasised the high prevalence of children experiencing DVA in the home and the links between DVA and child protection, and argued that service development should reflect this high prevalence. In 2014, guidelines developed by NICE44 to underpin the response of health- and social-care professionals and commissioners to DVA made the specific recommendation that interventions should be commissioned that aim to strengthen the relationship between the child and their non-abusive parent or carer through the delivery of joint or parallel sessions that include advocacy, therapy and other support that addresses the impact of DVA on parenting. This is consistent with the more general recommendation in the World Health Organization (WHO) guidelines for children to be offered psychotherapeutic interventions. In 2010, following an independent review of child and adolescent mental health services (CAMHS), the UK Government identified children experiencing DVA as a vulnerable group requiring mental health services engagement. 57 This was reiterated in a second review of CAMHS, which recommended the development of a better service for vulnerable groups of children and young people, including those who have been exposed to DVA. 58

UK service delivery landscape

Despite the importance of specialist provision for children exposed to DVA being recognised in UK national policy, mapping studies indicate that provision for these children is patchy. Humphreys et al. 59 mapped the availability of support for children and families affected by DVA. A telephone questionnaire completed by nearly all of the 326 specialist DVA service providers approached indicated that 69% of services in England and 89% of services in Wales, Scotland and Northern Ireland employed specialist children’s workers, with the most frequent forms of support offered being group-based support for pre-school-aged (72% in England, 89% in Wales, Scotland and Northern Ireland) and school-aged children (69%, 89%), one-to-one support (this did not include counselling, 61%, 89%) and advocacy for children (72%, 86%). However, services reported that these efforts were being hampered by underfunding. A total of 12% of English refuge groups and 14% (n = 10) of refuge groups in the rest of the UK reported that they were unable to provide any children’s services owing to a lack of funding; and 42% of refuge groups in England and 60% of refuge groups in the rest of the UK were dependent on volunteers to keep their children’s programmes going.

As part of the same study, a survey of children’s charities found that, of the 449 (60%) charities that responded, < 2% reported offering dedicated projects dealing specifically with women, children and/or men from families affected by DVA, although a large number of non-specialist projects (74%) considered DVA to be relevant to service users and dealt with it during the course of their work. In a third component to the study, 20% of social services departments reported that there was no provision for work with children, women or men in DVA situations in their local area, and specific service provision for children living in situations of DVA featured in only 20% of children’s services plans.

A second mapping study of services in London, undertaken a decade later, described a similar picture of service provision, marked by significant gaps and unsustainable services. 50 In line with the earlier study, a survey of key individuals and services with a role in responding to DVA indicated that the majority of specialist services (56%) were provided by the voluntary sector. The majority (76%) of respondents also perceived gaps in DVA services for children, with the most frequently mentioned being a lack of counselling, group work and school-based prevention activities. However, the low response rate to the survey (19%, 193/1020) means that its findings are prone to participation bias. Radford et al. 50 noted that some of the gaps in service provision had been created or exacerbated by funding difficulties, with interviewees reporting innovative services being run for a pilot period and subsequently closed owing to a lack of sustainable funding.

Qualitative interviews (n = 79) with professionals identified a particular gap in support that fell between universal services and the acute specialist mental health services provided by CAMHS. In addition, documentary analysis identified limited evidence of support being provided for children living with domestic violence where the risk (of serious harm) to the mother was not perceived to be high, suggesting that children’s need for support was not perceived as independent from that of their non-abusing parent. Radford et al. 50 also noted that proactive responses to DVA by frontline services, such as the police, often had limited impact owing to the lack of support available to children in the community. There were also gaps noted in services for young people who had grown up living with DVA and were now abusive in their adult relationships. Violence from older boys towards their mothers was also a particular concern.

Mothers reported particular difficulties in securing timely access to children’s services, in particular to CAMHS as well as, in some cases, negative experiences of engaging with social workers who had made them feel responsible for abuse. However, other mothers reported a more helpful response.

In England and Wales, CAMHS have a remit to deliver services to children experiencing high levels of distress and mental health disorders. However, despite exposure to DVA being the most frequent type of trauma experienced by children and young people,13 a CAMHS mapping exercise carried out in 2008–957 found that only 7% (n = 320) of participating services described themselves as providing targeted services for children experiencing DVA. There is little published material describing what these specialist services are. 60 The coercive and often chronic nature of DVA, which is specifically directed at a child’s parent by a perpetrator with whom the child almost always has an ongoing emotional relationship, coupled with the distinct possibility of re-exposure, means that standardised treatments, even those that are trauma-informed, may not be appropriate for this group of children and parents. 61,62 Research with adult survivors of DVA suggest that psychological interventions that do not directly address DVA may be perceived as unhelpful, and that DVA-informed interventions are preferred. 63

Aim

The aim of this evidence synthesis was to formulate recommendations for further research, undertaken in the UK and internationally, that seeks to evaluate interventions that aim to improve outcomes for children exposed to DVA.

Objectives

To generate these recommendations, we applied evidence synthesis methods to provide a comprehensive overview of all available research evidence relating to the clinical effectiveness, cost-effectiveness and acceptability of targeted interventions for children exposed to DVA. We then contextualised this evidence in the current UK service delivery landscape, taking account of the views and priorities of stakeholders.

We specified five objectives at the outset of this study, namely to:

1. conduct systematic reviews of existing studies evaluating the acceptability and short-term benefits of interventions targeted to children exposed to domestic violence

2. estimate the potential medium-to-long-term clinical benefits of interventions tested in controlled experimental studies

3. estimate the cost-effectiveness of selected interventions

4. calculate the expected value of partial perfect information to help identify research priorities for future trials of these interventions

5. formulate recommendations for further research in consultation with survivors of DVA, young people and service providers.

The synthesis answered seven research questions to address these objectives.

Research questions

1. What is the nature of the evidence base evaluating targeted interventions to improve outcomes for children exposed to DVA?

2. What is the nature of existing interventions to improve outcomes for children exposed to DVA?

3. What is the evidence that existing interventions are clinically effective?

4. What is the evidence that existing interventions are cost-effective?

5. How are outcomes defined and measured in evaluations of defined interventions?

6. What is the evidence that existing interventions are acceptable to stakeholders and feasible to deliver?

7. What is the nature of the UK evidence base and service delivery landscape?

Deviations from the initial protocol

With regard to the review of qualitative studies, we initially intended to include studies that reported stakeholders’ preferences for particular interventions and the acceptability of different models, even if respondents had no direct experience of receiving an intervention. Owing to the number of papers identified and our interest in the different perspectives of three groups of informants (children, parents and professionals), we had to narrow the scope of our review to those studies reporting the views of people with first-hand experience of child-focused interventions. This meant that we were unable to produce findings on what would encourage or deter children and parents from taking up the offer of such an intervention in the first place. This remains an important piece of work to be undertaken to inform service providers about interventions that may be more or less acceptable to different groups, or at different points in the abuse trajectory.

We planned to map the outcomes measured in trials to the baseline measures of child and adolescent mental health and behaviour assessed in longitudinal cohort studies, and to use the results from the longitudinal studies to extrapolate the effects of the various interventions over the long term (using estimates from the NMA). However, given the uncertainty in the findings derived from the NMA, and further assumptions that would have been required to project outcomes into adulthood, it would have been misleading to model the longer term impact of interventions offered in childhood and, therefore, we did not model the medium- and long-term effectiveness of interventions (objective 2). The fact that we were unable to do this is a finding of this synthesis. The absence of robust estimates of long-term outcomes of participation in a specialist intervention following exposure to DVA remains a significant evidence gap. Finally, we were unable to fulfil objective 4 of the study, given that there was too much uncertainty in our cost-effectiveness analysis to support the expected value of partial perfect information analysis that we specified in our original protocol.

Public and patient involvement

We held two sets of meetings with three stakeholder groups: (1) young people with experience of using DVA services; (2) mothers with experience of using DVA services; and (3) professionals involved in commissioning and delivering services to families affected by DVA.

One set of meetings was held at the outset of the study, at which point groups commented on the need for research, as well as the scope and design of the study. Professionals, in particular, drew our attention to the need to focus on the role of context at multiple levels, which resulted in context becoming a much stronger theme in the synthesis than it might otherwise have been. The second set of meetings was held in the later stages of the study when preliminary results were available for discussion. At this stage, the groups worked with us to identify key messages and knowledge gaps.

The stakeholder groups that contributed to the study had a dual role: first, they acted as advisors to the study and, second, they contributed directly to the research (see Chapter 8). We recognise that the contribution of stakeholder views and experiences to the study falls outside the conventional remit of patient and public involvement (PPI) in research (which relates to research that is carried out with or by members of the public rather than about or for them). However, the challenge of identifying individuals with experience of DVA and use of DVA services to contribute to this study meant that we worked with these groups in both capacities.

Criteria for including papers in the review

Search methods

We identified search terms from the literature by checking indexing in the relevant databases, and, by using thesauri of the electronic databases, we identified appropriate subheadings. For example, for MEDLINE we used medical subject headings (MeSH) and text word terms for <Children and adolescents> combined with MeSH and text word terms for <domestic violence>. We then combined these with text word terms for <exposure of children to domestic violence or witnessing or growing up with domestic violence>. We searched for literature in MEDLINE (1946 to April 2013), PsycINFO (1806 to April 2013) and EMBASE (1974 to April 2013) on the OVIDSP platform; in the Cumulative Index to Nursing and Allied Health Literature (CINAHL) on EBSCOhost (1937 to April 2013); in the Cochrane Central Register of Controlled Trials (CENTRAL) on The Cochrane Library (1890 to April 2015); in the Science Citation Index, Social Science Citation Index on Web of Science (1900 to April 2013); in the Applied Social Science and Abstracts Index (ASSIA) (1987 to April 2013), International Bibliography of the Social Sciences (IBSS) (1951 to April 2013), Social Services Abstracts (1980 to April 2013) and Sociological Abstracts (1963 to April 2013) on ProQuest; in Social Care Online (socialcareonline.org.uk) (1980 to April 2013); in the WHO trials portal (1999 to April 2013); and in Clinicaltrials.gov (2000 to April 2013). All databases were searched from inception to April 2013. We used neither a filter to limit the search by study methodology nor date or language limits. We excluded letters and editorials, and records for which there was no abstract. For details of search terms used for MEDLINE, see Appendix 1. We reran the searches in September 2015 in MEDLINE, PsycINFO, EMBASE, CINAHL and CENTRAL, and as this update was for this quantitative review, we used search filters for controlled studies taken from the InterTASC website and the Cochrane Handbook of reviews of interventions. 80–82 For details of search terms for the update, see Appendix 2.

Data collection and analysis

For the selection of studies, two reviewers (EH and TM) independently screened the titles and abstracts of all potentially eligible references. Full-text reports of all potentially eligible trials were retrieved and assessed independently by the same reviewers. Cases of disagreement were resolved by consensus and recourse to a third reviewer if necessary (GF).

For data extraction and management, data were extracted independently on a bespoke form by two reviewers (TM, and RB or EH). Any disagreements were resolved by discussion and recourse to a third reviewer if necessary (GF). The following types of data were extracted from included studies: country, setting, intervention (type, frequency and intensity), practitioners, population, outcome types reported, number of participants and length of follow-up. We also extracted parameters to assess the risk of bias.

Assessment of risk of bias in the primary studies

The risk of bias in the primary studies was assessed independently by two reviewers (TM and AC or RB), with recourse to a third reviewer (GF) who resolved disagreements. We assessed risk of bias on the following domains using the Cochrane risk-of-bias tool:83 random sequence generation (selection bias), allocation concealment (selection bias), blinding of participants and personnel (performance bias), blinding of outcome assessment (detection bias) and incomplete outcome data (attrition bias).

Data synthesis

We categorised studies as a basis for a narrative synthesis to answer the following questions:

1. What is the direction of treatment effect?

2. What is the size of the effect?

3. Is the effect common across all studies?

4. What is the strength of evidence for the effect?

Had there been sufficient data (e.g. data on the same outcome from at least three studies of the same design, intervention and population), we planned to pool data in a meta-analysis to allow quantification of the direction of treatment effect and consistency of treatment effect. We did not proceed to classical meta-analysis, as the studies were too variable in both intervention type and participants; therefore, the estimated overall effect would have little context-specific meaning,84 although we did use the study findings in a NMA (see Chapter 5). We examined treatment effect direction and consistency with a structured (tables and descriptive text) narrative summary of the evidence from the studies. When data were available, we calculated effect sizes as standardised mean differences (SMDs) (Hedges’s adjusted g). 85

Intervention types

Interventions were categorised according to both the types of component intervention offered within the programme and the relative emphasis on each component both in terms of time and therapeutic focus. We defined components as psychotherapy, play-based therapeutic interventions, parenting skills training interventions, advocacy and psychoeducation (see Table 9). This categorisation is similar to that used in the BCCEWH review,43 although we did not use their single- or multicomponent categorisation. In addition, we noted whether the studies were aimed at children exposed to DVA or at children exposed to DVA who also met criteria for behavioural or emotional problems. The constituent parts of each intervention were categorised independently by two team members. A decision on the final categorisation for synthesis was made by the full team. Interventions were then further distinguished by the target population for the intervention. We provide definitions of constituent parts of interventions in Table 1.

From the descriptions of the intervention programmes we identified ‘components’ that constituted each intervention. These components were then used in the NMA (see Table 1).

Narrative synthesis

We grouped outcomes under broad headings corresponding to each intervention type, as described above. We then grouped studies by age of children, setting, duration and frequency of intervention.

Search

A total of 34 relevant papers referring to 13 primary research studies with 1345 participants were identified (Figure 2). 61,90–119,122–124 See Appendix 3 for a list of papers that report each study, with one paper identified as the main paper for the study. When we refer to a study in this report we reference only this main paper, unless we need to refer to data that are not reported in the main paper. We identified one protocol to a study that was ongoing. 62

FIGURE 2.

Flow chart showing selection of papers.

The search identified 9489 records for assessment from electronic databases and six additional records from reference lists of included papers. We excluded 9402 records as not relevant from reading the title and abstract (see Figure 2). A total of 87 papers were identified as potentially relevant and the full-text articles were assessed for eligibility. We excluded 52 full-text articles and recorded reasons for exclusion (see Figure 2). For a list of references to excluded studies, see Appendix 4.

Description of primary studies

None of the studies were set in the UK. Ten of the 13 completed studies were from the USA: four from Texas;90,91,93,100 three from Illinois;96,98,99 and one each from Pennsylvania,61 California92 and Oklahoma. 94 One study was from Canada,97 one was from Israel101 and one was from the Netherlands. 95 Nine studies were RCTs61,90–97 and four were CCTs, where allocation was quasi-random; participants were allocated sequentially98,99 according to the timing of their stay in a refuge,100 or no specific allocation method was reported. 101 Studies varied in size, but most were relatively small, ranging from 66 to 258 participants. There were 1345 participants in total. Studies were published between 1995 and 2015 (Table 2 and see Appendix 5).

The study in progress62 is an assessment of the efficacy of a 6-week, pre-intervention preparatory programme for parents and children undertaking a longer trauma-focused CBT intervention similar to that described by Cohen et al. 61 (see Appendix 6).

Assessment of the risk of bias

Overall, four studies were assessed as being at high risk of bias,98–101 two were assessed as being at low/unclear risk,94,95 and for the remaining seven studies it was unclear whether they were at high or low risk61,90–93,96,97 (Table 3 and see Appendix 7). Four studies were not randomised and were rated as being at high risk of bias for random sequence generation and allocation concealment. 98–101 In studies of psychological interventions, it is not possible to blind participants to allocation83,120 and, therefore, all studies were rated at high risk of bias for blinding for subjective outcome. Attrition bias was high, as many studies had a high dropout rate, had not used any methods to deal with missing data, or provided too few data for us to assess whether or not attrition was treatment-related. We rated four studies as being at low risk of bias for incomplete outcome data. 94–97 Often, the studies did not report the information needed to assess the risk of bias, leading to the high proportion of domains assessed as being at unclear risk of bias.

Outcome measures

Studies included reports of child behaviour, self-esteem and psychological outcomes such as anxiety, depression and PTSD. None of the studies measured self-harm, school attainment, school attendance, abuse in children’s or adolescents’ own relationships, child quality of life, reporting of concerns about children to social services, or intervention by social services (see Appendix 8).

Participants

Four studies included children aged ≤ 6 years, one of which included infants. 92,99–101 Six studies included children aged between 4 years and 12 years. 90,91,94–96,98 Two studies recruited children from 7 to 14 years of age. 61,97 One study tested an intervention specifically for women, but, to be eligible, women were required to have at least one child aged between 18 months and 18 years; no direct intervention was offered to these children. 93 We found no studies of interventions targeting older adolescents (15–18 years). Of the interventions involving parents, all eight were aimed at the non-abusive parent; in all but one study these parents were mothers. In Overbeek et al. ’s study,95 95% of non-abusive parents were mothers and 5% (148/155) were fathers (see Table 2 and Appendix 5). 95

The studies undertaken in the USA included participants from a range of ethnic backgrounds (white, African American and Hispanic), except for the study by Graham-Bermann et al. ,98 which reported no Hispanic children in its sample. Eight of the nine US studies reported that 30–49% of participants were African American,61,90,91,93,96,98–100 with one study including just 14.7% African American participants. 92 The study by Waldman-Levi,101 conducted in Israel, included children from Asian, African, European and American backgrounds. The study by Overbeek et al. 95 included 19% of children from Turkey/Morocco, 20% from the Antilles or Surinam and 18% from other countries.

Characterisation of abuse

Eight studies used a variety of methods to report exposure of children to DVA (see Appendix 9). Three studies did not describe their participants’ current exposure to DVA. 93,94,100 Cohen et al. 61 reported that 14% of mothers (non-abusive parents) were experiencing ongoing trauma related to DVA during the study. Graham-Bermann et al. ,98 in their 2007 study, found that women experienced between 1 and 252 events of DVA, and in their 2015 study that women reported 173 events [standard deviation (SD) 131.9 events] in the preceding year. 108 McWhirter94 reported that the level of abuse prior to the study was 15 or higher on the Hurt-Insult-Threaten-Scream (HITS) measure for IPV, where HITS scores range from 4 to 20 and a cut-off of 10.5 reliably identifies victims of domestic violence, with 91% sensitivity and 96% specificity. 94,121 Wagar and Rodway97 reported that children witnessed a mean of 1.5 violent incidents per week (SD 0.8 violent incidents per week) during the study, and Sullivan et al. ,96 using a Likert scale to measure three aspects of abuse (ridicule/control, threat of harm, actual harm), recorded a mean score of 1.5 violent incidents per week (SD 0.9 violent incidents per week) for the sample at the start of the intervention.

The duration of children’s exposure to DVA was reported in only three studies (see Table 2 and Appendix 9). Cohen et al. 61 reported that six children had been exposed for 2 years, 23 for 2.5 years and 95 for > 5 years. 61 Graham-Bermann et al. 98 and Overbeek et al. 95 reported that the mean length of abusive relationships was 10 years. 95,98 Given that the mean age of children was 9.6 years (SD 2.6 years) in the study by Cohen et al. ,61 8.3 years (SD 2.1 years) in the Graham-Bermann98 study and 9.2 years (SD 1.5 years) in the Overbeek et al. 95 study, we can see that the majority of children had been exposed for most, if not all, of their lives.

Three studies93,94,100 did not report whether or not there was ongoing contact with the perpetrator of the abuse. Jouriles et al. 90,91 and Waldman-Levi and Weintraub101 reported no contact with the perpetrator. Graham-Bermann et al. 99 reported that some families were intact, and thus, were living with the perpetrator, but no numbers were provided. 99 Wagar and Rodway97 reported ‘no contact in most cases’; four studies reported that 14–17% of non-abusive parents were living with the perpetrator, with 51–68% still having contact with the perpetrator (see Table 2 and Appendix 9). 61,92,95,96

Co-occurring abuse to children

Five studies reported whether or not children had experienced physical or sexual abuse (see Table 2 and Appendix 9). Lieberman et al. 92 reported that 19% of children had experienced physical abuse and 15% had experienced sexual abuse. Graham-Bermann et al. 98 reported that 30% of children were physically harmed during the year prior to the study, with 33% of those being harmed weekly and 12% having received a significant physical injury. 105 Sullivan et al. 96 used a composite score based on combining three scales of emotional abuse, physical abuse and physical injury to describe abuse to the child. The higher the score, the greater the abuse, with 0 corresponding to no abuse. They found a mean score of 1.4 (SD 0.9) on the composite scale of abuse prior to the intervention, which decreased to 0.7 (SD 0.6; range 0–5.0) at 4 months’ follow-up. 96 Waldman-Levi and Weintraub101 reported that 65% of children in the experimental group and 30% of children in the control group experienced physical punishment. In the study carried out by Overbeek et al. ,95 the non-abusive parent reported a mean of six episodes of psychological maltreatment of the child by themselves and 13.4 episodes by their partner in the past year (range 0–104 episodes). Mean incidents of physical child maltreatment by the non-abusive parent and the abusive parent over the past year were reported as 0.5 (range 0–15) and 13.9 (range 0–104), respectively. 95

Setting, recruitment and targeting of interventions

Studies recruited from a range of settings, although mostly from women’s shelters (refuges) or a combination of women’s shelters and community DVA services. One study recruited from a primary care clinic97 and one from child family services and the police95 (see Table 2 and Appendix 5).

Eligibility criteria for all studies specified that children be exposed to DVA. Some studies, however, set additional clinical criteria. Cohen et al. 61 required children to be experiencing PTSD symptoms; both Jouriles et al. 90,91 studies recruited children who met DSM-IV diagnostic criteria for ODD and CD. 90,91 Lieberman et al. 92 recruited mother–child dyads for whom there were ‘clinical concerns about the child’s behaviour or mother’s parenting after the child witnessed or overheard marital violence’. 92 Although one study recruited through primary care and women and child clinics,115 the inclusion criteria were simply that women experienced DVA and had a child (see Table 2 and Appendix 5). None of the interventions was provided to abusive parents (Table 4).

Categorisation of intervention programme

Ten different intervention programmes were assessed in the 13 studies. Two studies by Jouriles et al. 90,91 assessed the effects of a parenting plus advocacy programme in a feasibility study published in 200191 and in a larger study in 2009. 90 Overbeek et al. 95 assessed the effects of the psychoeducational intervention devised and assessed by Graham-Bermann et al. in 200798 and in 2015,99 with small adaptations for cultural applicability. The 10 programmes were generally complex. To assess the effects of these components in a NMA (see Chapter 5), we identified and labelled the individual components that constituted each programme. Across these 10 programmes, we identified seven component parts: (1) advocacy; (2) emotional support; (3) group activities; (4) parenting skills training; (5) play therapy; (6) psychoeducation and psychotherapy; and (7) various control conditions (Table 5). To synthesise the results of studies for this review, we categorised the interventions using their main therapeutic emphasis based on the components above. From this emerged five distinct intervention categories: (1) psychotherapeutic (including play therapy); (2) psychoeducational; (3) parenting skills training plus advocacy; (4) psychoeducation plus advocacy; and (5) advocacy (Table 6). We used the main therapeutic aspects of the intervention as the basis for categorisation and did not include the mode of delivery (group or individual), or the intervention target (mother, child or mother and child) as this would have created too many classifications into which to divide the relatively few studies.

The format and target of delivery differed between interventions (see Table 5). Seven interventions delivered individual components to either mothers alone, children alone or non-abusive parent–child dyads (see Table 5). Some were delivered in a group format and others used a mixture of individual and group formats. The time frame over which the interventions were delivered differed, ranging from 2 weeks to 18 months. Most psychotherapeutic and psychoeducation programmes were 5–10 weeks in duration, but Lieberman et al. 92 delivered a long-term psychotherapy programme of 50 weeks and McFarlane et al. 115 provided advocacy for 18 months (see Table 4 and Appendix 5). A complete list of studies by categories is provided (see Table 6).

Manualisation of interventions, details of personnel, training of staff, supervision and assessment of treatment fidelity

People in a variety of different professions delivered interventions in these studies, and this variation was observed across all studies; different professions were not seen to be delivering specific intervention types. Psychotherapeutic interventions were delivered by a range of people, including social workers,61 clinical psychologists,92 occupational therapists and unspecified counsellors or therapists. 94,100 Parenting skills training and advocacy were delivered by clinical psychology students90,91 psychoeducation was delivered by social workers, mental health professionals,95 social work and clinical psychology graduates, and people described as group leaders. 97 Advocacy and psychoeducation was delivered by female undergraduates plus group leaders (not specified)96 and advocacy alone was delivered by nurses. 115 Most of the interventions were manualised, included specific training for personnel, monitored treatment fidelity and included supervision. Although most reported that fidelity to treatment was measured, few reported the results of these assessments (see Table 4 and Appendix 5).

Comparator interventions

Five studies used a comparison that was either a waitlist control (WLC) or delayed control, treatment as usual or no intervention. 96–100 Four studies used a minimum active intervention, such as allowing mothers to access a play room with their children,101 telephoning monthly to offer advocacy support90,91 or providing a referral card with useful local information on services plus a telephone support number. 93 One study provided a group activity of similar duration and intensity as the intervention, but without the focus on DVA. 95 Four studies used an active control of psychotherapy,61,94 psychoeducation98 or advocacy92 (see Table 4 and Appendix 5).

Psychotherapeutic interventions

Five studies examined the effectiveness of psychotherapeutic interventions. Cohen et al. 61 compared the effects of CBT with child-centred therapy for children with PSTD symptoms. The intervention was delivered in eight group sessions over 8 weeks to non-abusive parents and their children, in which two sessions brought the mothers and children together. Lieberman et al. 92,113,114,123 compared child–parent psychotherapy (in which the mother–child relationship is focused on as the means of therapeutic change) with telephone advocacy, where Doctor of Philosophy (PhD)-level practitioners telephoned the mothers on a monthly basis and advised them about mental health services for psychotherapy. The children were aged 3–5 years. The intervention was delivered to non-abusive parent and child dyads weekly for 50 weeks. Lieberman et al. 92 reported that 55% (n = 17) of children and 45% (n = 14) of mothers and children from the control group received psychotherapy as a result of the advocacy intervention. McWhirter94 compared two types of psychotherapy, namely goal-oriented CBT and emotion-focused CBT. There was no further targeting of this intervention beyond children who had been exposed to DVA. The psychotherapy was delivered weekly to the children and to the non-abusive parents in separate groups for 5 weeks. Kot100,112 examined the effectiveness of a 12-session play therapy intervention delivered daily over 2–3 weeks to individual children, aged 4–10 years in combination with eight psychoeducational sessions. Therapy was delivered by three trained counsellors including a registered play therapist. Waldman-Levi101,118,119 investigated the effects of a play therapy intervention for children aged 1–5 years, designed to improve mother–child communication and play behaviour. It was delivered to the non-abusive parent and child dyads by occupational therapists, who were trained in delivering the Family Interaction for Improving Occupational Performance (Fi-Op) intervention. Fi-Op was delivered once or twice per week for 8 weeks while the women and children were residing in a women’s refuge. The play therapy differed from that in the Kot et al. study100 in that it was provided daily to children only, whereas Waldman-Levi et al. 101 provided weekly play sessions to mother and child and modelled good play behaviour to mothers to improve parent–child communication. Therapists also differed; Kot et al. 100 included one PhD-level play therapist and two master’s-level play therapists, and Waldman-Levi et al. 101 had occupational therapists with 10 years’ experience of paediatric occupational therapy who had been trained to deliver Fi-Op. For both studies the therapists were trained specifically in play therapy.

Psychoeducational interventions

Four studies examined the effectiveness of psychoeducational interventions. Graham-Bermann et al. 98,104,105 conducted a three-arm trial: (1) parallel group-based interventions for the children (Kids’ Club) and their non-abusive parents [Moms’ Empowerment Program (MEP)]; (2) a group-based intervention delivered to children alone (Kids’ Club); and (3) WLC. The intervention was delivered weekly to non-abusive parents and children aged 6–12 years over 10 weeks. The children’s groups were age-specific (6–8 years, 9–12 years) with five to seven children per group. The interventions were delivered by trained graduate clinical psychology students and local mental health services therapists. Graham-Bermann et al. replicated this study for pre-school children and compared parallel group-based interventions for younger, pre-school children aged 4–6 years (Pre-Kids’ Club) and their non-abusive parents (MEP) with a control group which received no intervention. The intervention was delivered twice weekly over 5 weeks by trained social workers and graduate psychology students. 99 Overbeek et al. 95 evaluated the clinical effectiveness of an intervention based on that developed by Graham-Bermann and adapted to take account of cultural differences in the Netherlands (e.g. attention to the context of child welfare, as both parents are afforded parenting rights in the Netherlands; more non-verbal activities, as there is a diversity of immigrant backgrounds and languages in the Netherlands; and use of the term ‘caregiver parent’, as some attending parents were fathers). The intervention was delivered to children aged 6–12 years and their non-abusive parent. The psychoeducational intervention focusing on DVA was delivered by mental health-care professionals, with social workers delivering group sessions without DVA content to the control arm. 95 Wagar and Rodway97 compared the effectiveness of a group psychoeducational programme delivered to children aged 8–13 years with a WLC. The intervention was delivered weekly for 10 weeks. The content of the psychoeducation delivered to children in all four studies included identification of emotions, safe methods of expressing emotions including anger, and correct attribution of responsibility for violence. Psychoeducation methods were tailored for a younger age group (4–6 years) in the study by Graham-Bermann et al. 99 The psychoeducation content delivered to parents was similar for the three studies as they followed the same programme, that is, the MEP, with the aims of educating women about the effects of DVA on themselves and their children, building parenting competence and managing child behaviour, and helping parents to understand and correctly attribute children’s emotions. 95,98,99

Parenting skills training plus advocacy interventions

Two studies looked at the effects of an intervention combining parenting skills training with advocacy support for parents and mentoring for children compared with the provision of monthly telephone-based advocacy for parents. 90,91 The intervention was delivered on an individual basis to non-abusive parents and children aged 4–9 years in their homes, approximately weekly for 8 months. The intervention was targeted at children about whom concerns had been raised with regard to the child’s behaviour. Mothers were provided with advocacy (emotional and instrumental support), for example listening and supporting, help to access housing or legal services and help with developing problem-solving skills. Mothers were also given parent skills training that included instruction, practice and feedback, and helped them to develop warm regard for, and positive relationships with, their children. The skills training was tailored around experiences of DVA.

While the non-abusive parents were receiving parenting skills training and advocacy, the children were looked after by students who ‘served as mentors’ (e.g. providing positive support and ‘prosocial’ models). The intervention was delivered by clinical psychology graduates who were trained and provided with weekly supervision.

Advocacy plus psychoeducation

Sullivan et al. 96 evaluated the effectiveness of an advocacy intervention provided to non-abusive parents combined with a psychoeducational intervention (The Learning Club) delivered to children aged 7–11 years. The advocacy was delivered to women on an individual basis over 16 weeks in their homes; The Learning Club was delivered weekly to children in groups over 10 weeks. Advocacy was delivered by highly trained female ‘paraprofessionals’. Psychoeducation was delivered to children by group leaders with extensive experience of working with children. Advocacy delivered to the parents comprised emotional and instrumental support with the aim of helping mothers to achieve their goals. Advocacy for the child was in the form of emotional support for children and an assessment of their needs and goals. The Learning Club was psychoeducation for children which helped children to understand safety, emotions and respect for themselves and others.

Advocacy

McFarlane et al. 93,115 assessed the effects of an advocacy intervention delivered by nurses plus a referral card detailing the availability of local resources compared with the provision of a referral card alone. The intervention was delivered to the non-abusive parent only. No intervention was offered to children. After the initial visit, nurses visited women in the intervention arm of the study at 6, 12 and 18 months, provided the women with a 15-item safety plan that aimed to improve safety behaviour and provided supportive care as ‘empathetic listeners’ as well as advocacy. The content of the advocacy differed from that delivered in both the Jouriles et al. studies90,91 (see Parenting skills training plus advocacy) in that it did not include the same level of emotional support, befriending or social support. Rather, it focused on what women could expect from abuse intervention services and referrals for job training needs.

Theory underpinning the interventions

We looked at the publications associated with each study to assess whether or not the intervention was described within theoretical frameworks (i.e. whether or not reference was made to relevant theory when talking directly about the intervention) and whether or not the causal mechanism by which the intervention was expected to have an effect on the stated outcomes was described. Four studies provided a theoretical grounding for the study and set out the theory for how each part of their intervention would lead to change in outcomes. 90,95,100,101 Graham-Bermann et al. 99 did not use a theoretical framework and Jouriles et al. 91 had a grounding in theory but did not describe a stepwise, causal mechanism. However, both sets of authors have described the theory underpinning their interventions well in their earlier publications. 90,98 Two studies grounded their intervention in theory but did not provide details about the pathway to change93,96 and three studies provided neither background theory nor a causal pathway. 61,94,97 We recognise that there may be additional papers to those we identified for each study or training manuals providing additional details that we were unable to locate, and so we cannot be definitive. Nonetheless, there is scope for interventionists to provide more detail as to how interventions are designed to meet their aims through theories of child development, and parent–child interactions.

Sample size and retention of study participants

Studies were mostly small, with eight recruiting fewer than 80 participants90–92,94,96,97,100,101 and only two with more than 200 participants. 93,98 Attrition ranged from 5% to 52% across the 13 studies. Five studies had good participant retention with under 10% attrition,91,93,95,96 and four of these studies paid their participants for participating: three paid for both initial participation and for follow-up visits93,95,96 and one paid for participation with few details. 91 Five studies were nearly as successful, with between 13% and 16% dropping out;90,92,94,98,122 of these, four studies paid their participants and one did not. 94 Three studies lost between 43% and 52% of their participants. 61,100,101 This high attrition rate may in part have been attributable to participants moving from living in shelters to going back to the community after enrolment in the study100,101 or because of very open inclusion criteria whereby mothers with substance abuse and mental health problems were included in the study, thus representing a more pragmatic study design. 61 However, none of these studies paid their participants to take part in the study or to complete follow-up assessments. Overall, it was hard to determine the precise reasons for variation in participant retention, but providing financial incentives to parents for their time looks like it might be a helpful strategy. Establishing safe protocols to follow up participants who move home or move from a refuge to their own home, which were given as the reasons for high attrition reported in two studies,100,101 might also prevent attrition.

Psychotherapeutic interventions

The size of the study and the risk of bias indicate that caution in interpreting findings from some of the studies in this section must be exercised. Kot et al. 100 reported just 22 completing treatment; this was a controlled trial without randomisation, and thus was scored as being at high risk of bias. These factors, coupled with a very short follow-up time, mean that we have less confidence in the strength of this evidence. Cohen et al. 61 recruited 124 participants and met criteria for low risk of bias for two important domains concerning selection bias; therefore, we are more confident in the data reported from this study. McWhirter et al. 94 recruited 50 participants and had a low risk of both selection and attrition bias. Lieberman et al. ’s study92 was judged as being at ‘unclear’ risk of bias, as a lack of reporting prevented assessment. In addition, some triallists used post hoc analysis of subgroups; we have indicated where this is the case.

Parenting with advocacy interventions

The size and risk of bias of the two studies included in this section indicate caution in interpreting findings included here. Overall, both studies were found to be at unclear risk of bias because there was a lack of reporting; Jouriles et al. ,91 a feasibility study, recruited only 33 participants. However, Jouriles et al. 90 recruited 66 participants, and this study was assessed as being at low risk of bias for sequence generation, although it was at high risk of bias for allocation concealment.

Psychoeducational interventions

The size of the studies and the risk of bias indicate that caution in interpreting findings from some of the studies in this section must be exercised. The two studies by Graham-Bermann et al. 98,99 both recruited > 100 participants; however, they were both scored as being at high risk of selection bias and attrition bias and were rated at ‘high’ risk of bias overall. Wager and Rodway97 reported insufficient details for us to assess risk of bias, and thus we had to rate it as ‘unclear’. They did, however, recruit > 40 participants. However, greater confidence can be placed in the data reported in the study by Overbeek et al. ,95 as this was rated as being at low risk of bias, and recruited 164 participants.

Psychoeducation plus advocacy intervention

Advocacy

Psychoeducation plus advocacy

Summary of findings

We identified 10 RCTs and three CCTs, with a total of 1345 participants, reporting results on the effectiveness of a range of interventions delivered to different combinations of children and their non-abusive parents. In 12 studies 100% of parents were mothers, and in one study 95% of parents were mothers. None of the studies was UK-based. We found no interventions delivered to families that included the perpetrator of DVA. The majority of studies were at high or unclear risk of bias, and four studies had fewer than 50 participants, which means that we must interpret their findings with caution. Studies targeted programmes to different age groups of children, ranging from 18 months to 14 years, but none evaluated programmes for older adolescents. Some studies targeted interventions to children with trauma symptoms or behavioural problems, while others targeted children exposed to DVA without any clinical inclusion criteria. The comparator interventions varied across studies to the extent that, even when two studies assessed the effects of the same intervention,95,106 we could not directly compare their results because the comparator arms differed. Interventions were delivered to children and parents in a variety of formats, including delivery to individuals, to dyads of mother and child together, and to groups of children and groups of parents with some joint sessions.

The heterogeneity of the studies, the virtual absence of replication and the high and unclear risk of bias makes any overall conclusions about the effectiveness of these interventions in general or for specific populations of children uncertain.

Summary of the effects of interventions

Overall completeness and applicability of evidence

Interventions were mostly delivered to pre-teens and teenagers up to the age of 14 years, and only one study included children under the age of 4 years. 101 Although the majority of studies included samples representing a range of ethnicities, it is not clear if these studies are relevant to the UK, as none of the studies was UK-based. Interventions focused on children and their mothers as non-abusive parents, except one in which some of the abused parents were men, although they formed only a small proportion (7/155) of recruited participants. 95 We are aware that there are studies of family-based interventions that include children and both abusive and non-abusive parents;129,130 however, these programmes have not yet been assessed in controlled trials. Studies reported a range of situations in which the child has contact or lives with the abusive parent, although some interventions required families to be out of contact with the abusive partner. Children in families in which the perpetrator is still in the home, or is in contact, may have needs that differ from those of children who are experiencing the upheaval of living in shelters (as well as the safety they offer). Some studies and interventions targeted children presenting with or assessed for behavioural problems91,92 or trauma symptoms (PTSD),61 whereas others were open, including any children exposed to DVA.

A range of outcomes were reported across studies, but just three measured PTSD and depression, and we were especially surprised to find that only one study measured anxiety, when feelings of anxiety may be a daily experience for many children who are exposed to DVA. Seven studies used the CBCL to measure externalising behaviour90,91,93,95,98–100 and internalising symptoms and, of these, two did not publish outcome data for both domains. 90,98 None of the studies measured self-harm, school attendance, school attainment or children’s quality of life. Few of the studies set out to examine adverse events or harms that might ensue from the intervention, for example an increase in child maltreatment, without which studies might provide an overly positive picture of the effects of their interventions. 131 We recommend that a core set of outcomes be developed for children exposed to DVA including harm to the child and adverse events;132,133 the set should be applicable to all forms of DVA and could be used in assessing effectiveness of interventions. We were also disappointed by the absence of cost-effectiveness analyses or even reporting of costs, although this is consistent with its absence more generally in the DVA research field. 134

Some replication of intervention was attempted. The Kids’ Club and MEP were replicated in the two studies by Graham-Bermann et al. 98 in children aged 6–12 years and later in children aged 4–6 years, modifying the Kids’ Club for pre-school children. 99 The Kids’ Club and MEP were also tested in the Netherlands in the study by Overbeek et al. 95 Difficulty in consolidating the data from these replications was encountered because of the age difference of children in the case of the studies of Graham-Bermann et al. 98,99 and in the comparator arm in the case of the study in the Netherlands, which had an active control rather than no intervention or a waitlist. Jouriles et al. 90,91 replicated their intervention in two studies, namely a feasibility study and a control, without changing any facet of the intervention content or study design.

Description of the mechanisms by which interventions could be hypothesised to lead to change in the stated outcomes of each study was reported well in eight of the 13 studies. 90–92,95,98–101 However, two studies provided background theory without setting out a clear pathway of how outcomes would be achieved,93,96 and three studies appeared to present no theory at all. 61,94,97 We note that multiple publications or the generous word count of the PhD thesis may be necessary for this level of exposition. Although we have been able to systematically characterise the components of the interventions, we were unable to define the role of these components in the causal pathway leading to benefit, no effect or harm to those children receiving them. This would better inform interpretation and extrapolation of findings, as well as the design of future interventions and trials. 135

Quality of the evidence

Only two studies had a low/unclear risk of bias overall,94,95 and four were rated as being at ‘high’ risk of bias, mainly because they did not use randomisation to allocate participants, did not conceal allocation and had a high rate of attrition. 98–101,122 Seven studies were rated as ‘unclear’ overall and this was mainly because our assessment was hampered by a lack of detail in reporting the study design and conduct. 61,90–93,96,97 We are aware that authors often want to publish more detail, but find this difficult owing to the word limits of most journals. This could partly be mitigated by the publication of protocol papers, more widespread requirements to report trials in concordance with Consolidated Standards of Reporting Trials (CONSORT) guidance,136–138 and the use of web-based extra appendices. The lack of reporting has undermined the utility of the research findings, which, had we had the relevant information, may have promoted the research to categorisation as low risk of bias. Only one completed study95 and one ongoing study62 had a published protocol; these can be enormously helpful in allowing more transparent reporting of study methods and guarding against outcome reporting bias.

Eight studies recruited fewer than 80 participants90–92,94,96,97,100,101 and four studies recruited 50 or fewer participants. 91,94,97,100 This casts doubt on the ability of a study to detect a meaningful difference in outcome data between the intervention and control. Participant retention was poor across most studies, with attrition of between 5% and 52%, and three studies losing > 40% of participants. 61,100,101 Many participants dropped out before the intervention started. Paying participants for their time appeared to help with study retention. High attrition may also reflect the fact that participants are experiencing a chaotic period in their lives, moving from a shelter to a safe home or returning to an abusive relationship, for example. Incorporating safe protocols to follow up participants who move home or return to their abuser might also help to mitigate attrition, and using qualitative research embedded in the trial can help to identify reasons for withdrawal.

Most of the studies delivering a psychoeducational or psychotherapeutic intervention had used manuals, assessed treatment fidelity, reported training of staff and provided supervision (see Table 3), but none assessed therapeutic alliance. Assessing these technical aspects of psychological interventions helps to identify why a treatment works and the mechanism by which it leads to specific effects. 139–142

Consideration must be given to comparator interventions. For example, Overbeek et al. 95 conducted a trial at low risk of bias and reported that their intervention performed no differently to their comparator across all outcomes measured. They used an attention control. It may have been beneficial to have included a WLC arm as a true comparison, but the study authors were precluded from doing so by their ethics committee. 95

No studies provided stratification at randomisation to account for sex, exposure to DVA or co-occurring child maltreatment. One study, by McFarlane et al. ,93 did publish data for different sexes, and one study, by Lieberman et al. ,92 published data from post hoc subgroup analyses of children at high risk versus children at relatively low risk.

Follow-up was generally short, with only four studies reporting beyond the end of treatment up to 2 years. 90,91,95,99 We applaud those studies for taking the difficult but important step of making assessments beyond the end of the intervention. Studies with WLC were largely unable to report beyond the end of treatment. 96–98,100

Agreement with other published studies or reviews

There are two systematic reviews of interventions for children exposed to DVA with a broader scope than ours; these include maltreated children or exposure to community violence and studies other than controlled trials. Rizo et al. included 31 studies and BCCEWH included 37 studies. 41,43 We identified an additional three studies published since these reviews. 95,101,122 The BCCEWH review identified seven categories of interventions based on the number of components [single (n = 4) or multiple (n = 3)], delivery of the intervention to the mother and child or to the child alone. Categories included psychoeducation, therapy, advocacy and parenting and provide a listing of the strength of evidence for each before going on to highlight the methodological problems including the lack of RCTs, the lack of follow-up beyond the end of treatment and the lack of gender-based analyses in spite of the gendered nature of DVA and the potential gendered effects of exposure to DVA. 43 Rizo et al. 41 defined four types of intervention: (1) counselling/therapy; (2) crisis/outreach; (3) parenting; and (4) multicomponent intervention programmes. They recognised the rigour of the content of studies by Graham-Bermann et al. ,98 Jouriles et al.,91 Lieberman et al. ,92 McFarlane et al. ,93 Sullivan et al. 96 and Wagar and Rodway,97 while making constructive criticism of the methodological robustness. However, they were unable to state which of the four approaches held the most promise for the future. They suggest that studies be set up with the content rigour of those listed above and with additional methodological rigour of experimental design with random allocation, that appropriate statistical analyses be used, and that larger population samples be recruited.

Limitations (potential biases in the review process)

Our first search of 10 electronic bibliographic databases was from inception to April 2013 (see Appendix 1). We reran the searches for this review in September 2015. We expedited the search by limiting it to the five main medical electronic bibliographic databases and employing study filters (see Appendix 2). By omitting sociological electronic bibliographic databases it could be argued we may have missed studies; however, we believe that this is unlikely, as this had not been the origin of any of our included studies to that point. In our risk-of-bias assessment we did not assess reporting outcome bias. This was because only one included study had a protocol and we would had to have scored ‘unclear’ for all studies. However, it may have been useful to assess this, as several studies appeared to have measured some outcomes and not reported them; for example, two studies purported to have measured two broad-band indices of adjustment using the CBCL (internalising and externalising), but published the data on only one of the domains. 90,98 It can be difficult to publish null or negative data in some journals, but all trial data should be available for the compilation of meaningful metasyntheses.

Recommendations

We would exhort future triallists to adopt robust randomisation procedures and provide clear and thorough reporting including the registration or publication of a study protocol. Future studies need to recruit samples of sufficient size to have the power to detect meaningful differences in outcome between intervention and control, and should work towards high retention of study participants, possibly by using qualitative research nested within the trial design. Scope for follow-up assessments at meaningful time points post intervention (e.g. 6 months, 1 year and 2 years) should also be considered. Outcomes measured should cover all facets of child development and health, including quality of life, mental health, behaviour, school attendance and school performance, and costs should be measured so that commissioners can assess the viability of introducing programmes. Triallists might consider using stratification of randomisation to assess the effects of important subgroups, such as sex, presenting symptoms, level of DVA exposure and current or past exposure.

Types of studies

We included only qualitative research studies and excluded surveys or quantitative studies that contained descriptive free-text data. Our eligibility criteria were:

• Empirical qualitative studies (standalone or discrete components of mixed-method studies) employing qualitative methods for data collection and analysis. For example, an interview study using detailed thematic or narrative analysis would be included, but a survey report with free-text responses analysed quantitatively would be excluded.

• Studies focusing on children’s, parents’ or stakeholders’ views and experiences of receiving or delivering child-focused interventions following children’s exposure to domestic abuse.

• Published articles or reports that have undergone some level of peer review.

Participants

• Children who have been exposed to DVA and have received a child-focused intervention.

• Parents of children exposed to DVA who have received a child-focused or child- and parent-focused intervention.

• Stakeholders involved in developing and/or delivering interventions to children exposed to DVA.

Phenomena of interest

Child, parent or stakeholder views and experiences of:

• receiving or delivering an intervention for children exposed to DVA

• benefits of specific intervention programmes, including specific aspects of interventions that are perceived to be helpful or otherwise

• barriers to and facilitators of uptake of and engagement with interventions.

Material content of papers

We included papers if they contained qualitative data from children, parents or stakeholders and/or author interpretations derived from qualitative research that directly related to children’s, parents’ or stakeholders’ experiences of interventions for children exposed to DVA.

Selection of papers

Two reviewers (TM, EH) independently screened titles and abstracts of references identified through the search strategy. Those considered to be relevant by both reviewers were obtained as full-text papers or reports. Eligibility of full-text papers for inclusion into the study was assessed independently by two reviewers (TM, EH) using pre-specified inclusion and exclusion criteria. Any disagreements were resolved by discussion or recourse to a third reviewer (GF or AS). Following previously published qualitative syntheses,155,156 we also applied two initial screening criteria for each paper to answer the questions ‘Is this qualitative research?’ and ‘Is this paper relevant to the qualitative synthesis?’ All included papers were categorised into groups based on whether or not they reported the views of children, parents or stakeholders. Where a paper reported the views of more than one group (e.g. children and parents), we assigned it to both groups (child, parent) to ensure that we included the views of all relevant participants within each group. A flow chart delineating the process of exclusion of reports was prepared (Figure 3).

FIGURE 3.

Flow of studies through the review process.

Data extraction and management

For each paper, two reviewers (TM, EH, AS, GF) independently extracted study data from the abstract, methods, results and discussion sections of papers using a standardised data extraction form. We extracted two types of data:

• views and experiences of children, parents or stakeholders of interventions for children exposed to DVA, reported as quotations in the papers (first-order constructs)

• authors’ interpretations representing their analysis and identification of implications based on the qualitative data from all participants in their study (second-order constructs).

We also extracted the following study details: author name(s); publication date; country; methods of data collection and analysis; phenomena of interest; characteristics of the study population; intervention characteristics, including setting, model of intervention delivery (group, individual), core components of the intervention, intervention duration and intensity; and items relating to study quality (Tables 9 and 10). Reviewers met to discuss and agree the constructs chosen. Disagreements were resolved by recourse to a third reviewer. A single ‘agreed’ copy of the constructs was saved.

Appraisal of study quality

Following the conventions of many published qualitative syntheses,29,153,163,164 we assessed each included paper using the Critical Appraisal Skills Program (CASP) tool163 to evaluate the quality of qualitative papers. Two reviewers independently appraised each of the papers meeting the inclusion criteria according to items in the CASP and the total number of questions for which ‘yes’ (or a positive answer) was obtained to give an indication of the reporting quality of the studies. Each paper was scored out of 10. We took an inclusive approach; thus, rather than excluding relevant papers based on reporting quality, we used the quality appraisal as a form of ‘sensitivity analysis’ (see Discussion) to explore how removing studies judged to be of poorer quality impacted on the synthesis findings.

Data analysis and synthesis

We synthesised studies using the principles and methods of meta-ethnography,149 which have been used in several systematic reviews and qualitative syntheses, with the aim of deriving new interpretations and conceptual insights. 149,153,155,164 We sought to identify first- and second-order constructs in the papers and to translate these across the papers in order to generate third-order constructs. The levels of construct can be defined as follows:

• First-order constructs: children’s, parents’ or stakeholders’ accounts of receiving or delivering child-focused interventions, reported as quotes from interviews, observations or field notes.

• Second-order constructs: interpretations of the authors of the paper regarding children’s, parents’ and stakeholders’ experiences of interventions, derived from the qualitative data from all participants in their study.

• Third-order constructs: views and interpretations of the synthesis team, derived from the identified first- and second-order constructs, and translation of these across all the papers included in the review.

Stage 2

Working as a team and taking one respondent group at a time (child, parent, stakeholder), two or three reviewers met to read through all the first- and second-order constructs for all the papers within that group. Focusing on the second-order constructs and using the relevant first-order constructs to illustrate these, and ensuring that all first-order data were represented by second-order constructs, the team compared the second-order constructs across papers within each respondent group. This enabled us to see if different authors used different language to describe and label the same experience or phenomenon. We thus sought to translate the constructs across papers in order to develop new overarching third-order constructs that included the interpretations of the synthesis team. This was done for each respondent group. Our generation of third-order constructs was largely inductive but was guided by issues of interest to the synthesis team. Therefore, the broad overarching third-order constructs that we generated were shaped by the team’s interest in factors that may aid or inhibit engagement in interventions for children exposed to DVA. However, the subconstructs within each broad third-order construct were inductively derived from the translation process. As part of this process, we developed a table of third-order constructs and a conceptual diagram for each respondent group to represent the key third-order constructs and how they might relate to each other (Figure 4; see Appendices 11–13).

FIGURE 4.

Conceptual diagram depicting relationships of third-order constructs between parents, children and stakeholders engaged in interventions for children exposed to DVA.

Results of the search

We found 8764 references (see Figure 3). From reading the titles and abstracts, we identified 105 references that were potentially relevant to this review of qualitative evidence, and obtained full-text copies. Fifty-seven papers were excluded as a result of not meeting our eligibility criteria. Of the 48 papers remaining, 39 were excluded because they did not report respondents’ direct experiences of an intervention but, rather, discussed the effects of DVA on children in broad terms. We identified nine papers reporting on five intervention programmes (see Appendix 14 and Table 9). 49,51,130,157–162

Readiness to engage with interventions

Benefits derived from interventions, with facilitating factors

The second overarching construct in this qualitative synthesis is dimensions of benefit derived from interventions, along with factors facilitating those benefits. Owing to overlap in the benefits articulated by each respondent group, we present two integrated sections: (1) benefits and facilitators for children; and (2) benefits and facilitators for parents, as expressed by all respondent groups. Benefits to stakeholders were not reported in the papers and, therefore, do not appear in the synthesis. When dimensions of benefit were identified only by particular respondents, or agreed on by various respondents, this will be noted. As highlighted previously, some benefits overlap with readiness.

Barriers, challenges and tensions within interventions

The third and final overarching construct relates to barriers, challenges and tensions within interventions for children exposed to DVA, as experienced by children, parents and stakeholders. We present these from the perspective of each of the three respondent groups (see Figure 4).

Readiness

The personal readiness of children and parents is crucial for engagement with interventions. For children, readiness has three facets: (1) change and adaptation as the context for readiness; (2) willingness to break the secret of violence; (3) and understanding and acknowledging DVA. For children, readiness is a process. Getting to the point of feeling ready to talk about and acknowledge their experiences of living in abusive situations may be the most profound change associated with taking part in an intervention.

To be ‘ready’, non-abusive parents (mothers) need to have a degree of stability (beyond the initial trauma and practical difficulties associated with separation from the abusive partner) to be ready to accept that DVA may have had a detrimental effect on their children, to be able to look beyond their own needs to those of their children, and to recognise that their children are in need of help. A lack of parental readiness may preclude take-up and, as parents are gatekeepers for their children, may prohibit children’s access. For whole-family interventions, perpetrators (fathers) need to acknowledge that the relationship has been abusive, admit and take responsibility for their actions and accept the impact of their abuse on their children before they are able to accept or support a child attending an intervention. Engagement in an intervention may further enhance readiness for parents, as their understanding of abuse and how their children are affected increases. This may increase their willingness to engage in further interventions in the future (see Figure 4).

Parents and children may need to be ‘primed’ to become ready to engage in interventions, for example through the provision of appropriate information about the nature of the intervention, why and how it might be helpful, and reassurances regarding confidentiality. Practitioners also need to be ready to deliver an intervention, in terms of their attitudes, skills and workload management. Practitioner readiness is nested in a context of broader organisational readiness, which includes adequate resourcing, a commitment to upskilling the workforce, workforce stability and acceptance of the validity of the intervention by managers.

Benefits and facilitators

Children experience a range of benefits through engaging in interventions, from simply enjoying time with their mothers, having fun and making new friends, to the relief of realising that they are not alone and being given a forum and permission to talk about their experiences. In the interventions children learned many new skills and concepts, including denormalising, naming, recognising and labelling abuse, ascribing responsibility for abuse to the abusive parent, developing empathy for their mother and developing skills in expressing emotion, problem-solving, interacting with other children and managing their own behaviour. Through witnessing prosocial behaviour from group leaders, a process of positive feedback, and response to their behaviours and interactions with others, children develop a greater sense of self-efficacy and esteem. Nevertheless, the benefits to children are not uniform: although some children respond visibly and relatively quickly, for others the intervention represents the first step in a much longer journey.

An important gain for non-abusive mothers participating in interventions was simply spending time with their child/children and learning to communicate with and better understand their child/children. Mothers come to realise that they are not alone in their experience of abuse and in their challenging role of parenting and keeping children safe within the context of a coercive and abusive family. They gain more insight into the nature of abuse and the effect of DVA on their children and may acquire an increased ability to manage their emotions. Although none of the interventions mentioned specific parenting-skills training, the opportunity to share parenting practices with each other and with intervention providers, the use of video-feedback on play sessions, and interventions for increasing mother–child communication may enhance the quality of the mother–child relationship and increase mothers’ self-confidence as parents.

Several factors appear to facilitate benefits to children. Both parents and children identified the significance of the group environment, which appears to deliver benefits in and of itself. Group work enables children and mothers to recognise that they are not alone in their experience of DVA, and this realisation contributes to further disclosure and reduces the experience of stigma. Unstructured free play interspersed with more structured therapeutic activities is crucial for providing opportunities for children to relax and enjoy themselves as well as for revealing to practitioners a little more of children’s internal selves. Similarly, enjoyable activities provide respite and balance the difficult emotional work involved in group interventions (see Figure 4).

Mothers may act as facilitators, seeking out interventions and encouraging children to participate. Children’s enhanced emotional expressiveness and empathy can enable them to understand, and offer support to, their mothers. When children and their mothers both participate in programmes, either in parallel or together, the benefits for one can be seen to interact with and augment those for the other. In full-family interventions parents may need more time to work together both in single-sex and mixed-sex groups. However, mixed-sex groups may provide settings within which fathers’ coercive behaviours can be sustained.

Barriers, challenges and tensions

Children in receipt of interventions experience barriers to initial engagement alongside challenges and tensions throughout their participation. Lack of readiness, having to learn to break the secret of DVA, not wanting to talk about the past and anxious predictions of what the intervention might involve can be overcome with input from parents and stakeholders. Children have to manage the difficulties of revisiting painful memories and of conflicting images of their fathers as both caregivers and creators of the abusive situation. Within group interventions, children find the rules of confidentiality useful and reassuring, but these rules can also leave children unsure of how to talk about the group with friends and family. They also have to overcome the need to keep their secret, and learning to trust and share can take several weeks. Children’s expectations about relationships may shape the way in which they engage with an intervention in the first instance. Some children may arrive with clear ideas about gendered control and may use power play to gain control of a group. Others may feel disempowered, and thus to speak up in the presence of other children or adults who they perceive as powerful (see Figure 4).

Practical barriers for parents include finding time for interventions in an already busy family schedule. Mothers struggling with the demands of organising a move from an abusive home may have no spare capacity to engage with a new intervention. Once engaged in an intervention, it is upsetting for parents to hear their children’s views and emotional responses to the DVA they have witnessed, and parents may find it difficult to encounter challenges from their child about their own behaviour. Parents may feel that a barrier between them and their children is created by the confidentiality rules for children’s parallel group sessions, and may want to know more of what takes place within them. Fathers can find it difficult to engage if they disagree with the ethos of the organisation delivering an intervention, particularly when it diverges from a patriarchal perspective.

Stakeholders regard the timing of an intervention to be crucial, highlighting the importance of implementing an intervention when families are beyond the time of immediate crisis and upheaval. However, crisis and upheaval may characterise the lives of families recovering from DVA for long periods of time. It is possible to deliver some forms of intervention in transitional settings with families in crisis. Stakeholders share parental concerns for the safety of children who might, newly empowered by an intervention, stand up to an abusive parent, thereby increasing their risk of harm. Cultural issues also need to be considered for interventions delivered in contexts serving minority ethnic groups, where mothers may be fully mothering their children for the first time.

Summary of main results

We identified nine papers describing the views of children, parents and stakeholders who attended or facilitated one of five programmes aimed at children who have grown up in a home with DVA. 49,51,130,157–162 Three overarching constructs were developed: (1) ‘readiness’ to engage in interventions; (2) dimensions of benefit derived from interventions with facilitators; and (3) barriers, challenges and tensions involved in engagement with interventions for children exposed to DVA (see Figure 4).

Quality of the studies

Six of the nine papers were consistently rated as good quality by the review team, that is, they had ‘yes’ as an answer for all 10 CASP questions (see Table 10). 51,130,157,158,161,162 None of the papers had fewer than six ‘yes’ answers, indicating that all the papers that we identified were of good or acceptable methodological quality according to CASP. Papers scoring < 10 may have done so owing to a lack of space to report methods rather than a lack of research quality. For example, the paper by Humphreys et al. 49 scored 8 out of 10 because the statement of findings was brief. This paper appears to be a short report of the first cycle of the action research ‘Talking to my Mum’ published by Humphreys et al. in 2011,51 in which the authors had more space to describe what they had done, and which earned a score of 10. Therefore, for the Humphreys et al. 49 paper, scores reflect a lack of reporting rather than low research quality. The study by Kearney et al. 160 was given a ‘yes’ answer on seven questions. The authors did not describe how the data were collected and there was no reflection on the role of researcher. This paper reports the results of both quantitative and qualitative research studies and, thus, space to report all results in detail was limited. Both Peled and Edelson130 and Peled158 were allocated 10 ‘yes’ answers, but Peled and Edelson159 had just six ‘yes’ answers. All three Peled papers130,158,159 were based on the same programme. We believe that the lower score for the 1999 paper159 reflects a lack of reporting rather than a lapse in quality.

In a sensitivity analysis, we looked at the effects of removing the three studies that scored lower on CASP, in this case < 9 (see Table 11). 49,159,160 Overall, 22 of 53 subconstructs were affected by the removal of the three ‘lower quality’ papers. Only three subconstructs were dependent solely on data from one of these ‘lower quality’ papers. Two were dependent entirely on data from Peled and Edelson159 and informed constructs related to perpetrators nested within ‘Readiness’ (see Fathers’ readiness to acknowledge that their relationship has been affected by domestic violence) and ‘Barriers, challenges and tensions within interventions’ (see Divergence of ‘ethos’). The third subconstruct related to ‘Dimensions of benefit derived from interventions and facilitators to benefit’ where the construct ‘Realising that you are not alone’ (mothers) relied entirely on evidence from the Kearney et al. paper. 160 Removal of the three lower scoring studies meant that 16 subconstructs were reliant on fewer data but each was maintained by data from an additional one (n = 14), two (n = 3) or three (n = 2) papers (see Table 11). Overall, we conclude that removing the studies of ‘lower’ CASP quality did not have a significant impact on the overarching constructs within our synthesis. If we were to remove the three slightly lower quality papers, three subconstructs would not be present in the synthesis, namely fathers’ readiness to acknowledge the effect of DVA on their children, fathers’ difficulty in accepting interventions with an ethos (feminist) divergent from their own (patriarchal), and the benefit to mothers of realising that they are ‘not alone’ in their parenting situation.

Overall completeness and applicability of evidence

The studies that we identified were carried out in the USA and the UK. There was a relative paucity of studies reporting the views of different stakeholder groups who had received or delivered an intervention; only nine papers were found in which the children had received an intervention, and these reported on just five different programmes that represented three distinct models of intervention. Interventions included psychoeducational programmes, play therapy and guided self-help materials for parents and children, although individual programmes varied in terms of who they targeted (children only, mothers and children, children and both parents). The number of respondents per intervention type was relatively small, particularly for male (perpetrator) respondents and for the play therapy intervention which was based on four children attending one programme, along with one stakeholder who both delivered the intervention and authored the PhD report and the paper from which we extracted primary data. 161,162

The quality of all the studies was relatively high, with only three scoring below the maximum 10 points possible on the CASP tool (and even then scoring between 6 and 7). The removal of these lower scoring papers did not substantively change the range of constructs, but it did reduce the number of studies reporting a particular phenomenon.

Support from other reviews and implications for future interventions

The findings of this review resonate with those of other reviews in this field; for instance, this review has identified the importance of readiness for both children and parents, and those designing and delivering interventions need to consider how readiness for a programme is defined and assessed. Readiness was seen to be an ongoing process and appeared to develop through engagement with the programme we also saw that differential readiness between children and mothers could pose a barrier to engagement. However, another key message is that readiness can be promoted through priming or support and this might involve providing information about the content and form of an intervention, or offering reassurances about confidentiality. This is something that future interventions may be able to incorporate as a formal part of the delivery process to increase rates of initial engagement and ongoing retention.

The format in which an intervention was delivered was significant in that the group setting generated some benefits such as fun and friendships with other children or, for both children and mothers, the realisation that they were not alone in their experience of DVA. Likewise, interventions for mothers and children together offered the added benefit of shared time and activities. The format of an intervention is integral to the acceptability of a programme and is, therefore, an issue that commissioners should consider when making decisions about which model of intervention to fund.

The review identified a rich seam of data on mothers’ and children’s experiences of receiving a targeted intervention following experience of DVA, but this is countered by the limited range of intervention types considered in these studies. The benefits for mothers and children appeared to interact, with children having opportunities to benefit from mothers’ increased awareness of the impact of DVA and parents deriving pleasure and increased confidence in their own parenting from spending time with their children. The review43 undertaken to inform the NICE guidance on DVA44 included both quantitative and qualitative studies, and concluded that, for single-focus interventions, those aimed at mothers and children together appeared to be more beneficial for both than interventions aimed at children only. Similarly, they found that multicomponent interventions resulted in improved outcomes for children, which were also linked to improved outcomes for mothers. Stanley’s60 non-systematic review also highlighted evidence for the effectiveness of interventions that targeted mothers and children together in the aftermath of DVA, noting that such interventions harnessed the parent’s engagement with the child’s perspective on DVA as a mechanism for change.

The data identified in this review summarising fathers’ views and stakeholders’ views of fathers were rather scant and not very robust compared with the data for children and mothers; this reflects the fact that a recognition of perpetrators of DVA as fathers has been slow to emerge,60 and most of the literature reporting interventions that address abusive men’s fathering is available as grey literature (see Chapter 7). The studies that we identified included samples of only non-abusive mothers and abusive fathers. Although DVA is a heterogeneous phenomenon that sometimes includes bidirectional abuse between adult partners, typically, the burden of suffering is higher for women than for men165 and, therefore, the findings may not extend to families in which abuse is enacted by both adult caregivers.

It was interesting to note that the perceived benefits of participation in an intervention were much broader than the relatively narrow health-focused outcomes measured in the trials reviewed in Chapter 3, highlighting not only the importance of qualitative evaluation in order to capture the full range of potential impact, but also perhaps the need to consider whether outcomes measured in trials could be expanded to reflect some of the benefits voiced by those receiving and delivering interventions.

In relation to this point, this review enabled us to identify tensions and challenges associated with participating in DVA interventions that were not indicated by the quantitative research. Understanding the unintended consequences of an intervention is important for several reasons. Identification of some of the negative aspects of interventions may help to optimise existing or future programmes, for example by planning the timing of an intervention, ensuring effective communication channels with parents and providing transport for families to reach venues. It may also help those delivering programmes to give a full and honest account of what to expect from an intervention to parents and children who are considering taking part. Furthermore, O’Doherty et al. 166 underscore the importance of the deliberate and systematic measurement of harm in any type of trial, although they suggest that this is rarely done in the DVA field owing to a lack of approaches and measures. The findings of this review may give some direction as to what to measure.

Although our review highlighted the importance of intrapersonal contexts as a potential moderator of intervention acceptability and impact, relatively little attention was given to the broader context in which interventions were delivered. The nested layers of context (macro, micro and meso) in which an intervention is embedded are likely to have a bearing on the implementation of an intervention and the outcomes that can be achieved, with qualitative methods offering an opportunity to explore these influences. 154,167 The finding relating to organisational readiness resonates with the review of preventative interventions in DVA for children and young people,168 although it is based on the findings of one study. 49,51 Organisational readiness could be achieved in a variety of ways, but relevant training for staff delivering the intervention was identified as valuable.

Although there was some variation in the setting in which interventions were delivered (see Table 9), there were limited data reported on the acceptability of any particular setting; however, as explored in Chapter 8, this may have a significant influence on the way in which an intervention is perceived. Workers’ skill levels emerged as relevant for engaging and retaining children and mothers in interventions, with some suggestion that the skills required to deliver practical support to families in the aftermath of abuse differ from those required to engage them in therapeutic work. This finding again highlights the need for training but perhaps also for the careful selection of staff to deliver child-focused therapeutic work.

Limitations

We used an electronic search of bibliographic databases to seek studies for both the qualitative and quantitative review. The electronic search was broad and included key social science and educational databases, and we anticipated that we would identify sufficient studies this way. For this qualitative review we identified 48 studies, of which 39 reported phenomena about children living with DVA and only nine reported the views of parents or children who had been the recipients of a child-focused intervention. However, the number of titles and abstracts retrieved initially was such that we lacked the time to implement broader search strategies outside the electronic search such as ‘snowballing’ (citation and reference tracking) from included studies as recommended. 169 It is possible that adding additional search methods may have helped us to identify additional studies not found in the electronic searching.

We initially intended to include studies that reported parents’ and children’s views on the types of interventions that they would have found helpful, even if they had not directly received an intervention. Unfortunately, owing to the number of papers identified and our interest in the perspectives of three different groups of reporters, we had to narrow the scope of our review to those studies reporting the views of people with first-hand experience of an intervention for children. The effect of this was that we could no longer investigate how parents and children might conceive of an appropriate intervention prior to having experienced one. This meant that we were unable to produce findings on what would encourage or deter them from taking up the offer of such an intervention in the first place. This is an important piece of work that still needs to be done so as to guide service providers about the interventions that may be more or less acceptable to different groups or at different points in the abuse trajectory.

This review is based on a small number of qualitative papers, which is not unusual for a qualitative synthesis that prioritises the in-depth interpretive analysis of a small number of papers over a more superficial analysis of a larger number. A simple descriptive summary of the papers might have been sufficient, as some of the second-order constructs were derived from only one paper, for example. However, a descriptive summary may have produced a superficial aggregation of the content of the qualitative evidence, and would not have enabled the development of higher level overarching constructs, nor the clear identification of divergence and convergence in perspective across the diverse range of participant groups (children, parents and stakeholders). Instead, we believe that our systematic interpretive approach has allowed us to reinterpret the evidence to produce overarching constructs that ‘go beyond’ the findings of the individual primary studies. 170

Network meta-analysis and the consistency assumption

We used NMA (also known as mixed-treatment comparisons, or multiple treatments meta-analysis) to pool evidence across a connected network of intervention comparisons. NMA is increasingly being used to inform health technology assessments and clinical guidelines182 and has been used for pharmacological and non-pharmacological interventions. More recently, the methods have been applied to complex interventions,179 such as those identified in this review. The main assumption made in NMA is that the intervention effects would be expected to be similar in all trials if they had all included all interventions (the consistency assumption). 172 In other words, the populations included in each study should be similar. Importantly for reviews of complex interventions, there is also an assumption that interventions of the same ‘type’ are similar across studies. So, for example, it is assumed that advocacy interventions are comparable across all studies that include them. In this review, we found that interventions differ in the setting in which they are delivered, the target to whom they are delivered and the format of delivery (individual or in groups) (see Chapter 3). Setting may vary between the clinic and the community and between the group and the individual. Targets for the intervention differed by the child’s age, developmental stage and level of need, as well as by whether the intervention is delivered to the child only, the parent only, the child–parent dyad or child and parent parallel groups.

Definition of interventions

The intervention comparisons have been made in line with the classification of interventions in Chapter 3 (and subsequent chapters), in terms of the type of intervention, the target to whom the intervention is delivered, and the setting in which the intervention is delivered. As setting was identical in all arms within a given study, there was no comparative evidence available to make comparisons between settings. We also did not have sufficient evidence to look at combinations of intervention type, setting and target. However, we were able to examine the relative effectiveness of intervention type and target to whom the intervention is delivered, which we consider separately.

It is possible through NMA to estimate a set of coherent effect estimates that allow us to make comparisons across the entire network, as long as the network is connected (i.e. there is a path from any one intervention category to another). 173 We plot network diagrams for each outcome separately for (1) intervention type and (2) target to whom the intervention is delivered, to check whether or not our networks are connected. We make comparisons only between intervention categories that are connected.

Definition of outcomes

As noted in Chapter 3, there was little commonality in the outcomes reported across studies, with the main common measurement scale being the CBCL,183 which includes internalising and externalising scales. The total score is a sum of the CBCL-Internalising, CBCL-Externalising and other components. Eight studies reported on at least one index of the CBCL outcome (Table 16).

The outcomes reported could, however, be broadly categorised into mental health, behavioural and social competency and esteem outcomes. Table 16 shows which (useable) outcomes were reported by each primary study according to type of outcome (full details given in Table 7). By ‘useable’ we mean that sufficient information was reported to include the outcome in the statistical analysis. It was unclear whether the peer conflict outcome reported by McWhirter94 was a behavioural or social competency and esteem outcome and we therefore consider it in both outcome categories. McFarlane et al. 115 broke down their results by age, and we treat these as two separate substudies, for children aged < 5 years and for children and adolescents aged 6–18 years. Wagar and Rodway97 did not report any outcomes of relevance to our review, and Sullivan et al. 96 did not report sufficient information for the results to be used in the evidence synthesis (no SDs given); thus, these studies could not be included in the analysis and are not considered further.

Implementation and model fit

All analyses were conducted using a Bayesian Monte Carlo Markov chain simulation approach using the software WinBUGS version 1.4.3 (Imperial College London, London, UK and MRC Biostatistics Unit, Cambridge, UK). 184 The WinBUGS model code and data are given in Appendix 19. In all models, a burn-in period of 20,000 or 30,000 iterations gave satisfactory convergence based on inspection of Brooks–Gelman–Rubin plots,185 and all results presented are based on a further sample of 40,000 or 60,000 iterations (respectively).

We use the posterior mean of the residual deviance186,187 to assess goodness of fit. The residual deviance measures how close the observed values are to those predicted from the model, so that small values of the residual deviance indicate a better fit. As a rule of thumb, where a posterior mean residual deviance is approximately equal to the number of data points this indicates a good fit. Model fit was satisfactory for all results reported in this chapter.

Note that because the only ‘loop’ in the networks of comparisons was a three-arm trial (which cannot be inconsistent), it was not possible to assess the consistency of the evidence in these networks. 172 This does not mean that there is no inconsistency, just that we cannot check for it.

Networks of evidence

A key to the abbreviations used for the types and targets of the interventions is given in Table 17, and a description of the intervention comparisons made in each study is given in Table 18. We describe the evidence networks available first for a comparison of different types of intervention and then for a comparison of different targets of the interventions. As can be seen from Table 18, there is no comparative evidence of different formats (i.e. group or individual sessions) for the interventions, and, therefore, we can make no comparisons across formats.

Network diagrams for types of intervention

Figures 5–7 show the network of comparisons across intervention types with usable data for mental health, behavioural, and social competency and esteem outcomes, respectively, and for all studies (RCTs and CCTs) included in the quantitative systematic review (Chapter 3). Each solid line indicates that a study has made that comparison, and the width of the lines is proportional to the average sample size per arm of each study: the larger the sample, the thicker the line. Note that not all studies report all outcomes (see Table 16). As previously noted in Chapter 3, the study populations in this review differed across studies; in particular, there was a distinction between studies on children with known health or behaviour problems61,90–92,114 and those with no stated health or behaviour problems. 94,96–98,112,115,119 It is possible that intervention efficacy may be different across these two subpopulations, which are indicated by dashed and solid lines, respectively, in Figures 5–7.

FIGURE 5.

Intervention type: mental health outcomes. Network plot showing all comparisons made across intervention type in the RCTs and CCTs included in the quantitative review with usable data on mental health outcomes. Each connective line represents interventions that have been compared, and the study in which it has been made is indicated. The joined-up triangle represents a three-arm trial. 98 Dashed lines indicate studies on children with known health or behavioural problems, and solid lines indicate studies that do not state whether or not there are health or behavioural problems. The width of the connecting lines is proportional to the average number of subjects per arm (see Table 17 for key to interventions). a, Non-randomised studies (CCTs). 112,119 PEd, psychoeducation; C, child; P, parent; PlayTh, play therapy; PTh, psychotherapy; Adv, advocacy; Pting, parenting skills training.

FIGURE 6.

Intervention type: behavioural outcomes. Network plot showing all comparisons across intervention type made in the RCTs and CCTs included in the quantitative review with usable data on behavioural outcomes. Each connective line represents interventions that have been compared, and the study in which it has been made is indicated. The joined-up triangle represents a three-arm trial. 98 Dashed lines indicate studies on children with known health or behavioural problems, and solid lines indicate studies that do not state whether or not there are health or behavioural problems. The width of the connecting lines is proportional to the average number of subjects per arm (see Table 17 for key to interventions). a, Non-randomised studies (CCTs). PEd, psychoeducation; C, child; P, parent; PlayTh, play therapy; PTh, psychotherapy; Adv, advocacy; Pting, parenting skills training.

FIGURE 7.

Intervention type: social competency and esteem outcomes. Network plot showing all comparisons across intervention type made in the RCTs and CCTs included in the quantitative review with usable data on social competency and esteem outcomes. Each connective line represents interventions that have been compared, and the study in which it has been made is indicated. The joined-up triangle represents a three-arm trial. 98 Dashed lines indicate studies on children with known health or behavioural problems, and solid lines indicate studies that do not state whether or not there are health or behavioural problems. The width of the connecting lines is proportional to the average number of subjects per arm (see Table 17 for key to interventions). a, Non-randomised studies (CCTs). PEd, psychoeducation; C, child; P, parent; PlayTh, play therapy; PTh, psychotherapy; Adv, advocacy; Pting, parenting skills training.

The trial conducted by Overbeek et al. 116,117 was ongoing during our study, and results became available only after the statistical analysis for this chapter had been completed. We note that this study only adds a ‘spur’ to the network diagram and, as such, does not provide information on the rest of the network. In particular, the results from this study do not influence any of the comparisons between intervention types A–K, and provide information on only the comparison intervention types C versus L [i.e. between psychoeducational interventions to child and parent (in parallel) and ‘non-specific group activities’]. For this reason, this study is excluded from the remaining analyses.

As explained above, NMA is possible only for connected networks (where there is a path from any one intervention type to another). The network for mental health outcomes (see Figure 5) is connected for intervention types A–I, but not for intervention types J (play therapy) and K (advocacy + psychoeducation delivered to the child). It is, therefore, not possible to estimate the relative effect on mental health outcomes of these interventions compared with any of the other interventions. If we restrict the analysis to RCTs only, then Kot112 is excluded with the consequence that intervention type I (play therapy + psychoeducation) is not connected and we can make comparisons only between intervention types A–H.

The network for behavioural outcomes (see Figure 6) is connected for intervention types A–E, but not for intervention types F–K. We can, therefore, estimate relative efficacy of intervention types only within types A–E. Although Cohen et al. ’s61 and Lieberman et al. ’s studies92,114 do not report any behavioural outcomes, they do report the CBCL-Total, which is a sum of mental health, behavioural and other constructs. 183 If we are prepared to assume that the relative intervention effect on the CBCL-Total is approximately the sum of the relative intervention effect on mental health and behavioural outcomes (the additivity assumption), then we can include Cohen et al. ’s61 and Lieberman et al. ’s92,114 studies. This is because Cohen et al. ’s61 and Lieberman et al. ’s studies92,114 reported mental health outcomes alongside the CBCL-Total, which together with the additivity assumption enabled us to estimate a behavioural relative effect (see Modelling details). Including the CBCL-Total outcome for Cohen et al. ’s61 and Lieberman et al. ’s studies92,114 in this way connected the network for intervention types A–I, but not for J and K, giving a network plot identical to that for mental health outcomes (see Figure 5), with the addition of the Jouriles et al. 90 study. This is a significant additivity assumption (CBCL-Total relative effect = mental health relative effect + behavioural relative effect); however, for those studies that report CBCL-Internalising, CBCL-Externalising, and CBCL-Total, we found that the data were consistent with this assumption (see Appendix 17). If we restrict the analysis to RCTs only, then Kot112 is excluded with the consequence that intervention type I (play therapy + psychoeducation) is not connected and we can make comparisons only between intervention types A–H.

The network across intervention types for social competency and esteem outcomes is not connected (see Figure 7), and, thus, no evidence synthesis is possible. (See Chapter 3 for the results from the two studies reporting these outcomes.)

The key assumption of consistency, integral to NMA, is that the populations included in the studies are similar in their intervention effect estimates, allowing comparisons between more studies. If we restrict analysis to studies in which children have health/behaviour problems, we can make comparisons only between intervention types D–F, whereas if we restrict analysis to studies for which it is not stated whether or not children have health/behaviour problems, then we can make comparisons only between intervention types A–D and, if CCTs are included, type I. If possible, the consistency assumption is tested with sensitivity analyses with subpopulations, checking to see if the effect sizes are comparable. Because there is no overlap in the intervention types studied in these different populations, we did not perform sensitivity analysis by population type. When interpreting the results from the analyses, this needs to be with the knowledge that particular intervention types have been tested on only one of the two subpopulations, and, therefore, it is possible that results from a study based on an indicated sample may not extend to an unselected population, and vice versa.

Network diagrams for target of intervention

Figures 8–10 show the network of comparisons across the targets of interventions for all primary studies included in the systematic review of controlled studies (RCTs and CCTs) with usable mental health, behavioural, and social competency and esteem outcomes, respectively. The networks for mental health and behavioural outcomes are connected for intervention targets T1–T4, but not for targets T5–T7. It is, therefore, possible only to estimate the relative effects of targets T1–T4 compared with each other and not with any of the other targets for these outcomes. Restricting the analysis to RCTs only excludes the Kot112 study; however, the networks remain connected for intervention targets T1–T4. Neither Cohen et al. ’s61 nor Lieberman et al. ’s studies92,114 are included in the connected networks, so it is not necessary to include the CBCL-Total outcome.

FIGURE 8.

Intervention target: mental health outcomes. Network plot showing all comparisons made across target of intervention in the RCTs and CCTs included in the quantitative review with usable data on mental health outcomes. Each connective line represents interventions that have been compared, and the study in which it has been made is indicated. The triangle represents a three-arm trial. Asterisk indicates non-randomised studies (CCTs). Dashed lines indicate studies on children with known health or behavioural problems, and solid lines indicate studies that do not state whether or not there are health or behavioural problems. The width of the connecting lines is proportional to the average number of subjects per arm (see Table 17 for key to interventions).

FIGURE 9.

Intervention target: behavioural outcomes. Network plot showing all comparisons across target of intervention made in the RCTs and CCTs included in the quantitative review with usable data on behavioural outcomes. Each connective line represents interventions that have been compared, and the study in which it has been made is indicated. The triangle represents a three-arm trial. Dashed lines indicate studies on children with known health or behavioural problems, and solid lines indicate studies that do not state whether or not there are health or behavioural problems. The width of the connecting lines is proportional to the average number of subjects per arm (see Table 17 for key to interventions). a, Non-randomised studies (CCTs).

FIGURE 10.

Intervention target: social competency and esteem outcomes. Network plot showing all comparisons across target of intervention made in the RCTs and CCTs included in the quantitative review with usable data on Social Competency and Esteem Outcomes. Each connective line represents interventions that have been compared, and the study in which it has been made is indicated. Dashed lines indicate studies on children with known health or behavioural problems, and solid lines indicate studies that do not state whether or not there were health or behavioural problems. The width of the connecting lines is proportional to the average number of subjects per arm (see Table 17 for key to interventions). a, Non-randomised studies (CCTs).

The network across targets of intervention for social competency and esteem outcomes was not connected, (see Figure 10) and so no evidence synthesis was possible. (See Table 7 for the results from the one study reporting a social competency and esteem outcome.)

Network meta-analysis results

Types of intervention

In all analyses the posterior mean of the residual deviance was lower than the number of data points, representing a good fit to the data. In the primary analysis (RCTs only, no CBCL-Total outcomes), the between-outcomes (within-study) SD is estimated at 0.06, with a 95% credible interval (CrI) of 0.004 to 0.200. This was robust in all sensitivity analyses, suggesting relatively low between-outcome heterogeneity on a SMD scale.

Figure 11 shows the posterior mean (95% CrI) of the SMD for mental health and behavioural outcomes for RCTs only, where no CBCL-Total outcomes are included. The SMD can be interpreted as the change in mean outcome (measured in SD units) for each intervention type relative to type A = none, where we defined none as treatment as usual, WLC, minimum control, or the receipt of no intervention (see Table 4). Negative SMDs are consistent with benefit of the intervention.

FIGURE 11.

Randomised controlled trials only. Posterior mean SMD and 95% CrIs for each of intervention types B–I relative to intervention type A = none. Results are shown for (a) mental health outcomes and (b) behavioural outcomes. Only RCTs are included in the analysis. Negative SMDs indicate a better outcome (see Table 17 for key to interventions). Adv, advocacy; PEd, psychoeducation; PlayTh, play therapy; PTh, psychotherapy; Pting, parenting skills training.

For mental health outcomes, all intervention types have CrIs that cross 0 (no effect). However, types B (psychoeducational intervention delivered to the child) and F (psychotherapy) have CrIs that are nearly all negative. Note that many of the CrIs are very wide, reflecting the small numbers included in the primary studies. For behavioural outcomes, all intervention types have CrIs that cross 0 (no effect), although type E (advocacy + parenting skills training) has a CrI that is nearly all negative.

Figure 12 shows results for a sensitivity analysis where we continued to restrict studies to RCTs only, but allowed the CBCL-Total outcome to be used to connect the network for the behavioural outcomes. Comparing Figures 11 and 12, the results for mental health outcomes are seen to be robust to including the CBCL-Total outcome (i.e. they are similar for those trials that were in the main analysis). For behavioural outcomes, however, we now have additional results for intervention types F–H. Intervention type E (advocacy + parent skills training) is still the only type with a CrI that is mostly negative, whereas type G (CBT) is significantly worse than A = none for behavioural outcomes [although we need to treat this conclusion with caution, as there is no direct evidence on behavioural outcomes for intervention G (CBT)].

FIGURE 12.

Randomised controlled trials CBCL-Total included. Posterior mean SMD and 95% CrIs for each of intervention types B–I relative to intervention type A = none. Results are shown for (a) mental health outcomes and (b) behavioural outcomes. Only RCTs are included in the analysis. In addition, the CBCL-Total outcome is included under the additivity assumption. Negative SMDs indicate a better outcome (see Table 17 for key to interventions). Adv, advocacy; PEd, psychoeducation; PlayTh, play therapy; PTh, psychotherapy; Pting, parenting skills training.

Figure 13 shows results for a sensitivity analysis in which we included both RCTs and CCTs (but did not include the CBCL-Total outcome). Comparing Figures 11 and 13, results are seen to be robust to including the CCTs for mental health outcomes; however, we now have additional results for intervention type I = (play therapy + psychoeducation), which has similar efficacy to type B (psychoeducation delivered to the child). Results are also seen to be robust to including the CCTs for behavioural outcomes; however, we now have results for intervention type I (play therapy + psychoeducation) which has similar efficacy to type E (advocacy + parenting skills training).

FIGURE 13.

Randomised controlled trials and CCTs. Posterior mean SMD and 95% CrIs for each of intervention types B–I relative to intervention type A = none. Results are shown for (a) mental health outcomes and (b) behavioural outcomes. Both RCTs and CCTs are included in the analysis. Negative SMDs indicate a better outcome (see Table 17 for key to interventions). Adv, advocacy; PEd, psychoeducation; PlayTh, play therapy; PTh, psychotherapy; Pting, parenting skills training.

Figure 14 shows results for a sensitivity analysis in which we include both RCTs and CCTs and allow the CBCL-Total outcome to be used to connect the network for the behavioural outcomes. For Figures 13 and 14, results are seen to be robust to including the CBCL-Total outcome. For behavioural outcomes, we now have additional results for intervention types F–H. Intervention types E (advocacy + parenting skills training) and I (play therapy + psychoeducation) are still the only types with CrIs that are mostly negative.

FIGURE 14.

Randomised controlled trials and CCTs, CBCL-Total included. Posterior mean SMD and 95% CrIs for each of intervention types B–I relative to intervention type A. Results are shown for (a) mental health outcomes and (b) behavioural outcomes. Both RCTs and CCTs are included in the analysis. In addition, the CBCL-Total outcome is included under the additivity assumption. Negative SMDs indicate a better outcome (see Table 17 for key to interventions). Adv, advocacy; PEd, psychoeducation; PlayTh, play therapy; PTh, psychotherapy; Pting, parenting skills training.

Targets of interventions

In all analyses the posterior mean of the residual deviance was lower than the number of data points, representing a good fit to the data. In the primary analysis (RCTs only), the between-outcomes (within-study) SD is estimated at 0.09 with 95% CrI of 0.003 to 0.277, and this was robust in all sensitivity analyses, suggesting relatively low between-outcome heterogeneity on a SMD scale.

Figure 15 shows the posterior mean (95% CrI) of the SMD for mental health and behavioural outcomes for RCTs only. For mental health outcomes, all intervention targets have CrIs that cross 0 (no effect), with the exception of target T2 = child, for which outcomes are significantly improved compared with T1 = none (at 5% level). For behavioural outcomes, all intervention targets have CrIs that cross 0 (no effect), with the exception of target T3 = child + Parent, for which outcomes are significantly improved compared with T1 = none (at 5% level).

FIGURE 15.

Randomised controlled trials only. Posterior mean SMD and 95% CrIs for each of intervention target T2–T4 relative to intervention target T1 = none. Results are shown for (a) mental health outcomes and (b) behavioural outcomes. Only RCTs are included in the analysis. Negative SMDs indicate a better outcome. C, child; C + P, child plus parent; P, parent (see Table 17 for key to interventions).

Neither Cohen et al. ’s61 nor Lieberman et al. ’s92,114 studies are included in the connected networks (see Figures 8 and 9), and so results are unchanged by including the CBCL-Total outcome.

Figure 16 shows results from a sensitivity analysis where we include both RCTs and CCTs (but do not include the CBCL-Total outcome). Comparing Figures 14 and 15, results are seen to be robust to including the CCTs for both mental health and behavioural outcomes.

FIGURE 16.

Randomised controlled trials and CCTs. Posterior mean SMD and 95% CrIs for each of intervention target T2–T4 relative to intervention target T1 = none. Results are shown for (a) mental health outcomes and (b) behavioural outcomes. Both RCTs and CCTs are included in the analysis. Negative SMDs indicate a better outcome. C, child; C + P, child plus parent; P, parent (see Table 17 for key to interventions).

Neither Cohen et al. ’s61 nor Lieberman et al. ’s92,114 are included in the connected networks (see Figures 11 and 12), and so results are unchanged by including the CBCL-Total outcome.

Key findings

Assumptions and limitations

In order to perform the short-term modelling, we have had to make major assumptions (Box 1), and these must be borne in mind when interpreting the results.

Different outcomes were reported in different studies, with the only common outcome being the CBCL scale/subscales. Furthermore, the absolute values of the CBCL varied greatly across studies. We therefore pooled across outcome measures using the SMD for each trial measure, grouping the outcomes according to whether they measured aspects of behaviour or mental health. For studies that did not report the internalising and externalising subscales of the CBCL, we had only the CBCL-Total outcome. We excluded the CBCL-Total outcome in our primary analyses and included it in a sensitivity analysis by assuming that intervention effects on the CBCL-Total can be considered a sum of the relative effects on mental health and behavioural components of the scale. There are, however, other dimensions that make up the CBCL-Total, which may also add to the relative effectiveness as assessed on the CBCL-Total scale. We found that our results and conclusions did not change with inclusion of the CBCL-Total outcome. However, the relative effectiveness of intervention types F–H on behavioural outcomes must be interpreted with caution, as they depend on the validity of this assumption owing to a lack of direct evidence. It was not possible to obtain relative effect estimates for social competency and esteem outcomes owing to a lack of evidence.

Relative effects for different outcomes within an outcome category (e.g. mental health) are unlikely to be identical, and, therefore, we have used a hierarchical model in which the relative effects for outcomes within a category (mental health or behaviour) are assumed to be ‘similar’, using a common pooled mean effect across outcomes. This approach is less restrictive than pooling the outcomes by assuming that they are all identical and reflects the additional heterogeneity between outcomes in the estimated pooled mean effect. We have summarised this hierarchical model using the pooled mean across outcomes within outcome category in Figures 11–16.

We have assumed that the populations in the included studies are comparable; in fact, they differ by age of children and also by whether or not the study restricted inclusion to children with health or behavioural problems. Owing to insufficient studies, we were unable to perform sensitivity analysis of results for these different subgroups. We are, therefore, assuming that the relative effectiveness of the intervention types and targets do not depend on age or existing health/behavioural problems. Of course, interventions should be targeted to age and level of impairment, and the interventions that were actually delivered in each trial were tailored to the population and setting in which they were delivered. However, there is simply not enough evidence for us to be able to draw conclusions according to age group, level of impairment and setting, and there is a clear need for further research to fill this evidence gap. We were, however, able to draw some general conclusions that appeared to hold across these important subgroups. Because the interventions delivered in each of our included trials were tailored to the trial populations and settings, our relative effectiveness estimates must be interpreted together with the inclusion criteria and setting of the trials grouped by type of intervention (see Chapter 3). Furthermore, because there were only one or two studies making each comparison in the network, the results are driven by these studies, and we may expect that different results will be obtained in other settings/populations.

In order to ‘connect’ the networks of evidence, we assume that interventions of the same type that are tested in different studies are comparable, although owing to the sparse nature of the evidence, this was necessary only for advocacy, psychotherapy and CBT interventions. In reality, the interventions will be different even within type, so, for example, an advocacy intervention in one study may differ considerably from an advocacy intervention in another study. We have assumed that, although these interventions may differ, their true relative effectiveness on the SMD scale can be considered ‘similar’ for mental health and behavioural outcomes.

We have considered intervention type and target in two separate analyses; however, these two aspects of interventions may be expected to interact. For example, CBT may be more effective if delivered to a parent and child together than to a child alone. To explore this fully we need more studies with different combinations of intervention types and targets. The studies included in this review were not sufficiently diverse or numerous to do this; the study by Graham-Bermann et al. 98 was the only one that compared an intervention (psychoeducational) with different targets in two of the trial arms (child only vs. mother and child) as well as a WLC.

We have not distinguished between interventions delivered to individuals and to groups. This is a key question for commissioners and has cost implications. As can be seen in Table 18 there is no evidence comparing individual and group-based interventions, and so it was not possible to explore the relative effectiveness of these two different formats. Our relative effectiveness estimates must, therefore, be interpreted together with the format in which these interventions have been delivered in the trials in which those intervention types were included (see Chapter 3 and Table 4).

There was one non-randomised study112 that reported useable data on outcomes of interest for this review. We included this study in a sensitivity analysis. The main impact of this inclusion was that we could obtain a relative effect for the play therapy plus psychoeducation model, and this was seen to be effective for both mental health and behavioural outcomes. There was substantial imbalance at baseline across arms of this study, especially for the CBCL outcomes, which could have biased the study results. Furthermore, this study was found to be at high risk of bias across all domains (see Chapter 3). We are, therefore, cautious in drawing any conclusions about play therapy plus psychoeducation.

Intervention costs

Appendix 20 describes how the intervention costs are estimated for each arm of the studies included in the systematic review that inform the economic analysis. These are summarised in Table 19 grouped by intervention type, together with the pooled intervention cost (and uncertainty) assumed for each intervention type in our model.

Cost-effectiveness analysis

Table 20 shows the ICERs for intervention types for different relative weights, p_MH, given to mental health outcomes compared with behavioural outcomes, and Figures 17–21 show the corresponding CEACs.

TABLE 20 - Incremental cost-effectiveness ratios for intervention types for different relative weights, pMH, given to mental health outcomes compared with behavioural outcomes

BEH, behavioural; EvB, Adv + Pting vs. PEd (C); EvC, Adv +  Pting vs. PEd (C + P); MH, mental health; PEd, psychoeducation; PTh, psychotherapy; Pting, parenting skills training.

Notes

p_MH = 0 puts all weight on behavioural outcomes, whereas p_MH = 1 puts all weight on mental health outcomes.

ICERs are computed as incremental mean cost divided by incremental SMD on weighted outcome scale. Intervention types are ordered by costs and effects. ICERs are computed for each intervention type relative to the previous (non-dominated) intervention, unless otherwise indicated. Interventions are dominated if they have greater (or equal) mean costs but a more positive SMD on the weighted outcome scale (negative SMDs represent an improvement in outcome).

Intervention	Mean cost (£)	Mean MH outcomes	Mean BEH outcomes	Mean weighted outcomes	ICER (£)
pMH = 0
A: None	0	0	0	0	Reference
B: PEd (C)	335	–0.39	0.27	0.27	Dominated
D: Adv	335	0.07	0.18	0.18	Dominated
C: PEd (C + P)	670	0.05	–0.18	–0.18	3722.22
E: Adv + Pting	2915	–0.31	–0.46	–0.46	8017.86
pMH = 0.25
A: None	0	0	0	0	Reference
B: PEd(C)	335	–0.39	0.27	0.105	Dominated
D: Adv	335	0.07	0.18	0.1525	Dominated
C: PEd(C + P)	670	0.05	–0.18	–0.1225	5469.39
E: Adv + Pting	2915	–0.31	–0.46	–0.4225	9716.67
pMH = 0.5
A: None	0	0	0	0	Reference
B: PEd(C)	335	–0.39	0.27	–0.06	5583.33
D: Adv	335	0.07	0.18	0.125	Dominated
C: PEd(C + P)	670	0.05	–0.18	–0.065	67,000 (extendedly dominated)
E: Adv + Pting	2915	–0.31	–0.46	–0.385	EvC: 7015.63
					EvB: 7938.46
pMH = 0.75
A: None	0	0	0	0	Reference
B: PEd (C)	335	–0.39	0.27	–0.225	1488.89
D: Adv	335	0.07	0.18	0.0975	Dominated
C: PEd (C + P)	670	0.05	–0.18	–0.0075	Dominated
E: Adv + Pting	2915	–0.31	–0.46	–0.3475	21,061.22
pMH = 1
A: None	0	0	0	0	Reference
B: PEd (C)	335	–0.39	0.27	–0.39	858.974359
D: Adv	335	0.07	0.18	0.07	Dominated
C: PEd (C + P)	670	0.05	–0.18	0.05	Dominated
G: CBT	1238	–0.43		–0.43	22575
H: PTh + PEd	1872	–0.15		–0.15	Dominated
E: Adv + Pting	2915	–0.31	–0.46	–0.31	Dominated
F: PTh	4286	–0.51		–0.51	38,100

FIGURE 17.

Cost-effectiveness acceptability curve, p_MH = 0. The probability that each intervention is most cost-effective for a given willingness to pay per weighted outcome relative to A: none, where the proportion weight to SMD mental health. Mental health outcomes is pMH = 0, and the proportion weight to SMD behavioural outcomes is (1 – p_MH) = 1. Relative efficacies are taken from the NMA results using RCTs only, and not including CBCL-Total (RCTs only; see Figure 10), and so only interventions A–E can be compared. Adv, advocacy; PEd, psychoeducation; Pting, parenting skills training.

FIGURE 18.

Cost-effectiveness acceptability curve, p_MH = 0.25. The probability that each intervention is most cost-effective for a given willingness to pay per weighted outcome relative to A: none, where the proportion weight to SMD mental health. Mental health outcomes is pMH = 0.25, and the proportion weight to SMD behavioural outcomes is (1-p_MH) = 0.75. Relative efficacies are taken from the NMA results using RCTs only, and not including CBCL-Total (RCTs only; see Figure 14), and so only interventions A–E can be compared. Adv, advocacy; PEd, psychoeducation; Pting, parenting skills training.

FIGURE 19.

Cost-effectiveness acceptability curve, p_MH = 0.5. The probability that each intervention is most cost-effective for a given willingness to pay per weighted outcome relative to A: none, where the proportion weight to SMD mental health. Mental health outcomes is pMH = 0.5, and the proportion weight to SMD behavioural outcomes is (1-p_MH) = 0.5. Relative efficacies are taken from the NMA results using RCTs only, and not including CBCL-Total (RCTs only; see Figure 10), and so only interventions A–E can be compared. Adv, advocacy; PEd, psychoeducation; Pting, parenting skills training.

FIGURE 20.

Cost-effectiveness acceptability curve, p_MH = 0.75. The probability that each intervention is most cost-effective for a given willingness to pay per weighted outcome relative to A: none, where the proportion weight to SMD mental health. Mental health outcomes is pMH = 0.75, and the proportion weight to SMD behavioural outcomes is (1-p_MH) = 0.25. Relative efficacies are taken from the NMA results using RCTs only, and not including CBCL-Total (RCTs only; see Figure 10), and so only interventions A–E can be compared. Adv, advocacy; PEd, psychoeducation; Pting, parenting skills training.

FIGURE 21.

Cost-effectiveness acceptability curve, p_MH = 1. The probability that each intervention is most cost-effective for a given willingness to pay per weighted outcome relative to A: none, where the proportion weight to SMD mental health. Mental health outcomes is p_MH = 1.0, and the proportion weight to SMD behavioural outcomes is (1-p_MH) = 0. Relative efficacies are taken from the NMA results using RCTs only, and not including CBCL-Total (RCTs only; see Figure 10), and so only interventions A–H can be compared. Adv, advocacy; PEd, psychoeducation; PTh, psychotherapy; Pting, parenting skills training.

If we put all weight on behavioural outcomes (p_MH = 0), then a psychoeducational intervention provided to parents and children in parallel [type C: psychoeducation (PEd) (C + P)] is cost-effective (among the interventions we compared) if willingness to pay per SMD in behavioural outcome is greater than the ICER of £3722 (see Table 19). If willingness to pay is > £8017, then advocacy plus parenting skills training [type E: advocacy (Adv) + parenting skills training (Pting)] is cost-effective (among the interventions we compared) (see Table 20). Uncertainty in the most cost-effective intervention is greatest around a willingness to pay of £5000 (see Figure 17). As willingness to pay per SMD in behavioural outcome increases, the probability that advocacy plus parenting skills training is the most cost-effective intervention (among the interventions that we compared) increases (see Figure 17).

If we put all weight on mental health outcomes (p_MH = 1), then a psychoeducational intervention delivered to the child [type B: PEd (C)] is cost-effective (among the interventions we compared) if willingness to pay per SMD in mental health outcome is greater than the ICER of £858 (see Table 20). Among the interventions that we compared, if willingness to pay is > £22,575 then CBT (type G) is cost-effective, and if we are willing to pay > £38,100 then psychotherapy is cost-effective (see Table 19). When willingness to pay is low, there is a high probability that (among the interventions that we compared) a psychoeducational intervention delivered to the child is most cost-effective, but as willingness to pay increases, then it becomes equally as likely to be cost-effective as CBT (see Figure 20).

Among the interventions that we compared, as the weight given to mental health outcomes, p_MH, increases from 0 to 1 (see Table 19 and Figures 17–21), advocacy plus parenting skills training (type E: Adv + Pting) has a lower probability of being cost-effective, whereas psychoeducational intervention for the child [type B: PEd(C)] increases in the probability of being cost-effective. A psychoeducational intervention delivered to both parent and child separately [type C: PEd (C + P)] is cost-effective (among the interventions we compared) only if we give a high weight to behavioural outcomes. We have evidence only on CBT for mental health outcomes, so we can comment only on the cost-effectiveness of this intervention when we give all weight to mental health outcomes, where it is seen that CBT may be cost-effective for high willingness to pay per SMD in mental health outcomes.

Key findings

For behavioural outcomes, a psychoeducational intervention delivered to parent and child in parallel [type C: PEd(C + P)] is likely to be cost-effective among the interventions that we compared (ICER = 3722 per SMD). Health benefits are translated onto a monetary scale for direct comparison with intervention costs. If willingness to pay per SMD in behavioural outcome is greater than approximately £8000, then advocacy plus parenting skills training (E: Adv + Pting), also delivered to parent and child, is likely to be cost-effective (among the interventions that we compared).

For mental health outcomes, it is very likely that, among the interventions that we compared, a psychoeducational intervention delivered to the child [B: PEd (C)] is cost-effective. If willingness to pay per SMD in mental health outcomes is high, then CBT (delivered to Parent, Child and Dyad) may be cost-effective.