Non-pharmacological interventions for attention-deficit/hyperactivity disorder (ADHD)delivered in school settings: systematic reviews of quantitative and qualitative research

Article history

The research reported in this issue of the journal was funded by the HTA programme as project number 10/140/02. The contractual start date was in May 2012. The draft report began editorial review in November 2013 and was accepted for publication in April 2014. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The HTA editors and publisher have tried to ensure the accuracy of the authors’ report and would like to thank the reviewers for their constructive comments on the draft document. However, they do not accept liability for damages or losses arising from material published in this report.

Declared competing interests of authors

Ken Stein is chair of the NIHR HTA Editorial Board and a member of the NIHR Journals Library Board.

Disclaimer

This report contains quotations from transcripts of interviews conducted in the course of research and contains language that may offend some readers.

Copyright statement

© Queen’s Printer and Controller of HMSO 2015. This work was produced by Richardson et al. under the terms of a commissioning contract issued by the Secretary of State for Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK.

What is attention-deficit/hyperactivity disorder?

Attention-deficit/hyperactivity disorder (ADHD) is a neurodevelopmental disorder characterised by age-inappropriate levels of inattention, impulsivity and hyperactivity. 1 The current Diagnostic and Statistical Manual of Mental Disorders-Fifth Edition (DSM-5) categorises particular constellations of symptoms into three presentations of ADHD. These are (1) predominantly inattentive; (2) predominantly hyperactive/impulsive; and (3) combined, where symptoms of both inattention and hyperactivity/impulsivity criteria are present.

The DSM-5 diagnostic criteria for ADHD, published in May 2013, increased the age by which some symptoms must have been present from 7 to 12 years of age. 1 Another potentially key change in DSM-5 relates to the level of impairment; Diagnostic and Statistical Manual of Mental Disorders-Fourth Edition (DSM-IV)2 required clear evidence of clinically significant impairment for diagnosis, whereas DSM-5 requires only interference or a reduction in functioning. DSM-5 also requires clinicians to specify the severity level of a diagnosis: mild, moderate or severe. Given the recent publication of DSM-5 and the retrospective nature of evidence synthesis, the studies included in this review refer to diagnoses made according to Diagnostic and Statistical Manual of Mental Disorders-Third Edition (DSM-III) or DSM-IV criteria. 2,3

The International Classification of Diseases, Tenth Edition (ICD-10) classifies core ADHD symptoms as hyperkinetic disorder. 4 The diagnosis for hyperkinetic disorder is more restrictive, whereby inattention, hyperactivity and impulsivity must all be present. Thus, the ICD-10 diagnosis represents a subsample of the DSM-5 ‘combined presentation’ of severe ADHD. In Europe, ‘ADHD’ has become the diagnostic phrase most commonly used in practice, even when the narrower ICD-10 criteria are being used.

The aetiology of ADHD is complex and may be most clearly understood as involving the interplay of biological, psychological and social factors. 5 Several causal factors have been suggested to contribute to the onset and maintenance of ADHD. These include genetic factors, psychosocial factors, complications in pregnancy and delivery and environmental factors such as high lead exposure, head injury and diet. 6–9 Heritability is a major factor and appears to contribute approximately 75% of the aetiology of ADHD. 6 Although no large single gene effect has been isolated, the DRD2, DRD4 and DRD5 dopamine receptors appear to be involved. 10 Although the current DSM-5 diagnostic criteria continue to weigh heavily on core symptoms of inattention, hyperactivity and impulsivity, some research suggests that deficits in self-regulation and executive functioning could explain many of the problems of the condition and its impairments. 8 More recently, leading experts have argued for deficits in emotional self-regulation as a core feature of ADHD,9 but this has received relatively little research attention compared with the core symptoms of ADHD,11 and problems in emotional self-regulation are now the basis of a new category in DSM-5, ‘disruptive mood dysregulation disorder’, which may be associated with ADHD. 12

Prevalence

Attention-deficit/hyperactivity disorder is one of the most common disorders to present to child and adolescent mental health services; however, findings from studies that ascertain prevalence vary. The worldwide prevalence of ADHD is estimated at 2–5% for children aged < 18 years,13 with 3–16% of children displaying difficulties that do not reach the diagnostic threshold. Numerous studies have shown increases in the prevalence of clinical ADHD diagnoses and prescriptions for stimulant medication over recent decades, for instance, a 33% increase in prevalence of diagnosis from 5.69% in 1997–9 to 7.57% in 2006–8 according to US survey data. 14 Prevalence in the UK was found to be 3.62% for boys and 0.85% for girls, for a sample of over 10,000 children aged from 5 to 15 years in 1999. 15 Although a systematic review of research between 1978 and 2005 found no differences between European and US rates of ADHD,13 more recent studies sampling parent reports found that in the USA between 2008 and 2010, 6.3% of all children aged 5–9 years were reported by parents as having an ADHD diagnosis,16 whereas in the UK between 2007 and 2009, only 1.4% of children aged 6–8 years held ADHD diagnoses according to parent reports. 17

Although boys are more commonly diagnosed with ADHD than girls, the ratio varies between two and nine boys to one girl, depending on the category of ADHD presentation2 and whether prevalence is based on clinical or epidemiological populations. 18 Girls with ADHD may be less likely to be seen in clinics. 19 Younger childhood and low socioeconomic status have also been shown to be positively associated with prevalence. 20,21 Although prevalence appears to decline with age, a 2013 prospective study reported that ADHD persisted into adulthood for nearly a third of young people. 22 Despite the increasing clinical recognition in the UK, ADHD remains underdiagnosed in certain populations, including adults,23 children with intellectual difficulties24 and those with inattentive symptoms. 25 Differences in findings about prevalence are suggested to result from different study approaches to identification. 17 Differences in clinical practice,26 expression of symptoms and behaviour according to cultural, social and developmental contexts27,28 are argued to be additional contributing factors both within and between countries.

Co-existing issues

Attention-deficit/hyperactivity disorder frequently co-exists with other mental health disorders, particularly antisocial and oppositional behaviour, but also tic disorders, specific learning difficulties, autistic spectrum disorders (ASDs), anxiety and depression. 11,29–31 As many as two-thirds of all children with ADHD in the general population are reported to have at least one other co-existing condition. 32 Many children with ADHD also have difficulties with social interaction and low self-esteem that affect their relationships with their parents, relatives and friends, as well as practitioners at school. 33 Often, these problems are at least as important as ADHD in contributing to the longer-term outcome in the individual child. 34 In addition, ADHD has also been linked with lower than average intellectual potential and academic underachievement across the developmental spectrum, from preschoolers to adults. 14 Inattentive symptoms and their related executive functioning deficits have been particularly associated with learning difficulties. 7 In the longer term, those with ADHD are less likely to be employed full-time, and more likely to have a lower household income than age and gender matched controls. 18

Controversy

Despite the breadth of research relating to prevalence, the diagnosis of ADHD has stimulated considerable debate and sometimes strong and conflicting views. 35 ADHD remains a contested disorder; for instance, in 2009 O’Regan found that 50% of general practitioners and 20% of special educational needs (SEN) co-ordinators in the UK did not believe that ADHD was a ‘real neurological condition’ (p. 4). 36 Bailey discusses the contested aetiology of ADHD and the issues that acceptance of a biological basis for ADHD can raise,37 which include concern over the use of medication for long periods of time in young children38 and the belief that the core symptoms of ADHD, inattention, hyperactivity and impulsivity, are traits seen in many children, especially the very young. 39 It has been shown that higher percentages of diagnosis of ADHD may occur for the youngest children in school year groups, which has led some to suggest that behavioural differences attributed to ADHD may be due at least in part to immaturity. 40 The labelling of hyperactivity, impulsivity and inattention as ADHD when these behaviours interfere with expectations for correct classroom behaviour has been criticised by those who believe ADHD is a constructed response to the demands of modern academic education. 41 Timimi and Taylor argue that ADHD is best understood as a cultural construct, given the variability in prevalence internationally and rise in diagnosis rates in Western culture. 42

By contrast, other studies suggest that solely biological understandings of ADHD can be adopted by educational staff and/or parents and others as a means to transform ‘multifaceted problems into organic dysfunctions’ (p. 1). 43 The recently documented increase in diagnosis of, and medication for, ADHD has been linked to reduction in educational funding and restricted classroom discipline policies. 44 It has been suggested that pressures on health and educational institutions for higher attainment and rapid improvement mean that there is often a focus on short-term solutions provided by medication at the expense of more intensive, long-term educational support, such as that provided by non-pharmacological interventions. 45 Thus, polarised views of either cultural or biological origins continue to be reported in the literature, despite repeated research findings that suggest ADHD is a transaction between biological, environmental and psychosocial factors. 6,8–10

Interventions for children with or at risk of attention-deficit/hyperactivity disorder

The National Institute for Health and Care Excellence (NICE) clinical practice guidelines recommend that for school-age children and young people with severe ADHD, medication should be offered as the first-line treatment. 46 Parents should also be offered a group-based parent training/education programme. If the child or young person with ADHD has moderate levels of impairment, the parents or carers should first be offered referral to a group parent training/education programme, either on its own or together with a group treatment programme [such as cognitive–behavioural therapy (CBT) or social skills training] for the child or young person. Pharmacological treatment may then be added to the management plan if symptoms do not sufficiently respond to this approach. Teachers who have received training about ADHD and its management should provide behavioural interventions in the classroom to help children and young people with ADHD. Medication for children and young people with ADHD should always form part of a comprehensive treatment plan that includes psychological, behavioural and educational advice and interventions. Use of both medication and psychosocial treatment for ADHD varies widely within and across nations. 47 At present, the most common approaches to the treatment of ADHD are medication and/or psychological or behavioural interventions.

Pharmacological interventions

The most frequently used pharmacological treatments, and those with the largest evidence base, are the stimulant medications, methylphenidate and dexamfetamine,48 but their use remains controversial among some people who work with children. 49,50 The Care Quality Commission revealed in 2013 that the prescribing of methylphenidate for ADHD in the UK had risen by 56% in primary care from 420,421 in 2007 to 657,358 in 2012. 51 Meta-analyses of stimulant medications have shown them to be effective at decreasing the symptoms of hyperactivity, impulsivity and inattention, although their effectiveness on cognition and achievement are more modest. 52 However, any positive effects do not appear to persist once stimulants are no longer used53 and as many as 30% of children do not respond to stimulants. 54 The potential adverse effects of treatment with stimulants include decreased appetite, weight loss, insomnia, stomach ache, headache and irritability. 55

Several studies of the cost-effectiveness of pharmacological ADHD interventions have been undertaken. In 2001 in the UK, Gilmore and Milne examined the cost-effectiveness of different medications from the perspective of the NHS, finding methylphenidate to offer the best value for money. 56 NICE estimated the cost per quality-adjusted life-year gained by methylphenidate at £9200 to £14,600. 57 Cost-effectiveness studies have compared medication to behavioural treatment and combined treatments, often finding in favour of medication alone. 58

Non-pharmacological intervention

Non-pharmacological interventions target behaviour directly or indirectly through cognitive and affective processes and typically target children, teachers and parents. Interventions that target teachers and parents usually involve training for delivery of interventions that target the children. An early meta-analysis compared a range of treatments for ADHD including both pharmacological and non-pharmacological interventions and found larger effects from all interventions on behavioural (d = 0.56) rather than educational outcomes (d = 0.28). 59 These overall effects were larger for medical interventions (d = 0.45) than for educational (d = 0.39), psychosocial (d = 0.39), or parent training interventions (d = 0.31). However, the effects on educational outcomes were greater for educational interventions (d = 0.58) than for other types of psychosocial intervention. There was little support for the influence of any reduction in behavioural problems on educational outcomes across studies. Although it is widely accepted that intervention in ADHD should be based on multimodal treatment,55 some research has suggested that adding psychosocial interventions to medication does not improve outcomes significantly. 60,61

Several reviews point to the effectiveness of behavioural interventions, particularly behavioural parent training (BPT) and behavioural classroom management. 62–64 For example, Pelham and Fabiano examined three types of interventions, including behavioural modification in classroom settings (BMC) that consisted of ‘contingency management’ based on ‘social learning theory’, BPT and behavioural peer intervention (BPI). 64 Results were synthesised across a range of constructs related to ADHD (including behavioural observations, family functioning, academic productivity, peer relationships and cognitive functioning), informants (e.g. parents, teachers, children and clinicians), and intervention type. Average effects by study design indicated that methodologically weaker single- and within-subject (treatment group received multiple treatments over time in a crossover fashion) designs generally reported larger effects than controlled trials. For the comparison of treatment with waitlist/no treatment Cohen’s d ranged from –0.03 to 0.44 for BMC; –0.02 to 0.70 for BPT; and 0.29 to 0.63 for BPI. Cohen’s classification for interpreting effect sizes distinguished between ‘small’ (d = 0.20), ‘medium’ (d = 0.50) and ‘large’ (d = 0.80) sizes, with very small findings < 0.20 having the least clinical impact. 65 Although these findings include evidence for the effectiveness of these interventions, the wide range of reported effects does not clarify their size and consistency. Pelham and Fabiano suggested a range of potential moderating variables that could have influenced the treatment outcomes, which included recipient gender, age, comorbidity, socioeconomic status, therapist characteristics and treatment characteristics such as intensity and adherence. 64

Fabiano et al. conducted meta-analyses to evaluate the effectiveness of behavioural interventions that included parent training, child training and classroom-based behavioural interventions. 63 Effects were synthesised across intervention type; constructs related to ADHD (including two observational behavioural measures; ADHD symptoms, externalising symptoms, impairment productivity and achievement); intervention context (school and clinic); and informant types. As was found by Pelham et al. ,66 average effects by study design indicated that methodologically weaker single-subject, within-subject (treatment group received multiple treatments over time in a crossover fashion) and pre–post (treatment groups assessed pre and post intervention) designs reported larger effects than controlled trials. For the 20 controlled trials, the average weighted effect size was 0.67 with 95% confidence intervals (CIs) from 0.54 to 0.80. Significant reported heterogeneity across studies was perhaps due in part to the collapsing of data across different settings, outcomes and informants. 63

There is less evidence for the effectiveness of neurofeedback on ADHD. For example, Willis et al. conducted a systematic review of 14 empirical reports of neurofeedback treatment for children with ADHD and reported that neurofeedback is not well supported as an intervention for this disorder. 67 Furthermore, the cost-effectiveness of neurofeedback has been called into question. 68 Social skills training, summer treatment programmes and academic modifications have some support in the treatment of a range of ADHD symptoms and related impairments. 69 However, Barkley reports that social skills training shows less benefit for teenagers with ADHD compared with younger children, which suggests that age may be an important moderator of the effectiveness of social skills interventions. 70

In 2013 Sonuga-Barke et al. published a systematic review of peer-reviewed randomised controlled trials (RCTs) investigating the efficacy of non-pharmacological interventions delivered in various contexts (including home, school and clinic settings). 62 Statistically significant treatment effects were found for all the non-pharmacological interventions when the person rating the outcome used was the closest to the intervention setting (e.g. teacher ratings for interventions in school settings). 62 This was the case for all dietary [standardised mean difference (SMD) ranged from 0.21 to 0.48] and psychological treatments (SMD ranged from 0.40 to 0.64) that included cognitive training, neurofeedback and behavioural interventions. However, these treatment effects were not statistically significant for all psychological interventions when raters were blinded to the treatment participants were receiving. 62

School-based interventions

The education system has a front-line role in the management of ADHD. Results from the 2004 British Child Mental Health Survey showed that more families of children with ADHD had sought support from education in the past year than from professionals in specialist health services (74% vs. 51%). 71 Few systematic reviews and meta-syntheses have examined the effectiveness of non-drug interventions in school settings independently of those delivered entirely in home and clinical contexts. One exception is DuPaul et al. who updated a previous meta-analysis to provide a quantitative review of school-based ADHD intervention research that had been conducted between 1996 and 2010. 72 The authors examined the effects of three intervention types labelled as ‘academic’, ‘contingency management’ and ‘cognitive–behavioural’. ‘Academic’ involved study skills training and modification of academic instruction; ‘contingency management’ involved the application of techniques grounded in operant conditioning (such as rewards and punishments) and ‘cognitive–behavioural’ encompassed the development of self-control skills including self-instruction and problem-solving strategies. For the behavioural outcome comprising symptoms related to ADHD, statistically significant positive effects were identified for the within-subjects group (treatment groups assessed pre and post intervention) and single-subject designs but not for the controlled trials (d₊ = 0.18, 95% CI –0.62 to 0.98). Similarly, all mean effect sizes were positive for the academic outcome and statistically significant results were reported for single-subject designs but not for controlled trials (d₊ = 0.43, 95% CI –0.36 to 1.21) and within-subjects designs. However, analyses for the controlled trials were severely limited by the small number of studies included (n = 3).

Although stimulant medication and behaviour modification typically target and have proven to be effective interventions to increase on task, and reduce disruptive behaviour among children with ADHD within the classroom, a literature review conducted in 2006 which focused on academic interventions for ADHD concluded that the manipulation of antecedent conditions, such as academic instruction or materials, often improved both behavioural and academic outcomes. 73 However, in 2007 Trout et al. 74 systematically reviewed non-pharmacological interventions that targeted academic outcomes using single-subject and within-group (treatment groups assessed pre and post intervention) study designs. They categorised interventions as ‘antecedent’ (interventions that target children prior to an academic task); ‘consequence’ [interventions that targeted children post performance of the target behaviour(s)]; ‘peer-mediated’ (intervention that were delivered in part by peers); ‘parent-mediated’ (interventions that were delivered in part by parents); ‘self-regulation’ (interventions that targeted self-regulation of cognition and behaviours); and other interventions that used a combination of treatments. The authors suggested that peer tutoring and self-regulation show some evidence of effectiveness. Nonetheless, the authors reported that there were few systematic lines of research and reached few firm conclusions regarding the effectiveness of behavioural interventions that target children with ADHD. In an earlier review conducted in 2005, Reid et al. 75 reported beneficial effects for interventions based on ‘self-monitoring’ related to ‘self-regulation’ in their review of symptom and scholastic outcomes, providing further support of the beneficial effect of self-monitoring. However, as single-subject and within-group designs were the focus of this review, it is yet to be established if these effects are observed in controlled trials.

A 2010 questionnaire study of first-grade teachers76 points to the fact that interventions of the type reviewed in the studies cited above may not always match the interventions that teachers report using for children with ADHD difficulties. Teachers reported using environmental modifications, assignment modifications and behaviour modification strategies more frequently with inattentive students than with other students. Although some of the interventions mentioned by teachers, such as reward systems and time out, are considered evidence-based treatments for children with ADHD,64 other strategies the teachers reported using, such as preferential seating and additional time, are less often researched.

Reported effectiveness of school-based interventions may vary depending on the outcome of interest. In their 2005 review, Pelham et al. 66 compared a contingency management intervention to methylphenidate or the use of both treatments and reported effect sizes that were four to five times greater for the effectiveness of the educational intervention for classroom rule violations than for teacher ratings of ADHD behaviours. Although school-based behavioural interventions can improve targeted behaviours in the short term, they have been found less useful in reducing the core symptoms of ADHD. 60

Because of its prevalence and at times refractory course, childhood ADHD results in considerable costs for society, particularly to the educational system. In 2013, Telford et al. 77 considered the wider education, health and social care costs of adolescents with ADHD in the UK. The mean cost per adolescent for NHS, social care and education resources used in a 12-month period related to ADHD was £5493 in 2010 prices and the median was £2327. Education resources accounted for approximately three times the cost of health-care costs. The total annual cost of adolescents with ADHD in the UK is estimated to be £670M. 77

Methodological considerations

Reviews of quantitative research in school settings have frequently evaluated the preponderance of within-subject group and single-subject designs, which, although valuable, are more prone to bias than RCTs, which are the scientific ‘gold standard’ for evaluating treatment effects. 78 Many of the findings are difficult to interpret, as they combine results across different contexts (e.g. school, clinical and home), interventions, outcomes and informants. Tests of statistical significance and CIs are often not reported, which makes the findings difficult to interpret, and there are no standardised guidelines for interpreting effect sizes for within-subject group and single-subject designs and comparing these effects with those found in other study designs such as between-group designs where Cohen’s d is often reported. There are differences in the types of outcome measures used across study designs with most single-subject design studies employing proximal outcome measures such as curriculum-based measurements or direct observations of classroom behaviour. 72 In contrast, most controlled trials and within-subject group design studies used more distal measures such as teacher ratings or report card grades. 72

Sociopolitical aspects of educational research in the UK may contribute to a lack of research employing the most rigorous experimental designs. RCTs provide general information about interventions that are particularly useful to policy-makers. In contrast, educational researchers and/or practitioners may be more concerned about knowledge about the application of information to specific cases79,80 or be concerned about aspects of education that are not represented by straightforwardly measurable outcomes,81 for example those who suggest that education involves norms, values and processes of judgement that cannot be separated from extrinsic variables. 82 This may lead them to prefer approaches like case studies and action research. Educators may also be sensitive to moves towards a more central control of education, which some may associate with calls for evidence-based practice. 81

The measurement of attention-deficit/hyperactivity disorder symptoms and outcomes

A range of constructs have been used to evaluate the effectiveness of treatments that target children with or at risk of ADHD in school settings. These constructs can be categorised into three higher-order groups: (1) core ADHD diagnostic symptom categories (including inattention, hyperactivity/impulsivity, ADHD combined); (2) symptoms commonly associated with ADHD (including externalising symptoms, internalising symptoms, social skills); and (3) scholastic behaviours and achievement (including ‘perceptions of scholastic adjustment’, ‘standardised achievement’ and ‘curriculum’-based achievement). Core ADHD diagnostic symptom categories include the three diagnostic presentations of ADHD as specified in DSM-5: age-inappropriate ‘inattention’; ‘hyperactivity/impulsivity’; and ‘combined’ inattention and hyperactivity/impulsivity. The second category of ADHD-related symptoms encompass difficulties that frequently coexist with the core symptoms of ADHD and complicate its management, but are not relevant to diagnosis. Scholastic behaviours and achievement encompass perceived adjustment to school such as achievement, motivation, academic skills and abilities. These three higher-order constructs have been assessed by a range of behavioural measures, which include ratings and observations, as well as more relatively objective neurocognitive assessments and tests of scholastic achievement with objective performance criteria.

Assessments

When measuring the effectiveness of school-based interventions, the child’s core ADHD symptoms, ADHD-related symptoms and scholastic behaviours and outcomes are typically assessed by teacher and parent perception-based measures [e.g. Conners’ Rating Scale (CRS)],83 although independent observers (who have had no previous relationship with the study participant) are sometimes used (e.g. observer-based assessments of on-task behaviour). Neurocognitive assessments such as the test of variables of attention (TOVA)84 have also been employed to assess ‘inattention’ and ‘hyperactivity/impulsivity’ and a range of standardised achievement tests are used to measure academic outcomes (e.g. Wide Range Achievement Test). 85 Most studies include a range of outcomes assessed by a number of raters or informants.

Sonuga-Barke et al. ’s 2013 systematic review of non-pharmacological interventions found significant treatment effects for all of the non-pharmacological interventions when the person rating the outcome used was the closest to the intervention setting (e.g. teacher ratings for interventions in school settings). 62 However, these treatment effects were not statistically significant for all psychological interventions when using the most methodologically rigorous blinded assessments, which indicates the potential for bias in outcomes from raters who are involved in intervention delivery and/or expect the intervention to be successful. Blinding, however, was sometimes inferred from the study design rather than taken from the reported use of blinding. Moreover, findings were limited to a composite outcome combining core ADHD symptom measures and delivery settings, which may mask important effects and account for some of the significant heterogeneity in findings across the primary studies. 62 In addition, there may plausibly be limited generalisation of the impact of an intervention in one setting to perceptions of the child’s behaviour in another setting. Nonetheless, Sonuga-Barke et al. 62 highlight the importance of methodological evaluation alongside evidence synthesis of interventions that target children with or at risk of children with ADHD.

Triangulation of data in quantitative versus qualitative research

Mixed methods in primary studies are considered to enable additional grounds for inference owing to triangulation of data, methods and analysis. 86 The benefits of mixed-methods systematic review are similar. 87 However, triangulation of data, when defined as the additional confirmation of a finding through repetition from different studies, can be problematic in qualitative studies. Unlike experimental evaluations, where repetition increases finding strength, an interpretive approach does not seek a true answer; rather, it explores the meanings people make of their experiences. 88 In interpretive studies it is expected that different people will make sense of things in different ways. By contrast, similar sense of a topic made by different people is not taken as additional evidence of truth, but rather that the participants are drawing from a similar cultural ideology to make sense of their experiences. 89 Therefore, an isolated finding may be more important than repeated findings, for example because it illuminates a previously implicit and overlooked meaning. 88,90 In interpretive research, triangulation of data can be understood as the compilation of multiple perspectives, where the resulting representation of complexity of perspective and depth of meaning is a sign of study rigour. 91

Rationale

The research questions of a systematic review guide the criteria for included studies. 87,92 Our research questions ask about effectiveness of ADHD interventions in schools, factors that enhance or limit the delivery of school interventions for ADHD, and the experience of ADHD in schools. We were unable to locate any systematic reviews of either the experience of non-pharmacological interventions for ADHD, or the experience of ADHD more generally; therefore the synthesis of qualitative primary studies holds important potential to contribute new information on this topic. In this systematic review, we synthesise qualitative research in addition to experimental evaluation studies in order to explore attitudes, experiences and factors that may help explain how or why interventions for ADHD in school settings are or are not effective. Through the overarching synthesis of both quantitative and qualitative reviews, we will explore similarities, contradictions and gaps between these syntheses, further informing the research questions and implications for further research. 87

This review also holds the potential to contribute important new information about the effectiveness of non-pharmacological interventions for ADHD in schools. A range of non-pharmacological interventions have been developed and delivered in school settings by parents, teachers or other professionals. As outlined above, few published reviews have considered the effectiveness of non-pharmacological interventions in school settings independently of those delivered predominantly in other settings such as at home and in clinics. Therefore, a gap remains for a systematic review that considers the effectiveness and cost-effectiveness of such non-pharmacological interventions that are delivered primarily in school settings.

The reviews reported have typically taken a narrow focus in terms of the interventions and outcomes included and few have distinguished between different types of informants (see Fabiano et al. 63 for exception). Moreover, the focus has been on single-subject and within-group designs rather than on controlled trials, which compromises evidence synthesis as study design, intervention type, delivery context, outcomes and informant type may produce important differential effects. 62 Despite the current clinical recommendation for non-pharmacological ADHD interventions,46 there is a lack of clarity regarding their effectiveness and in particular their effectiveness in school settings. Moreover, their cost-effectiveness has not been systematically reviewed.

To build evidence-based guidelines for the treatment of ADHD in school settings, knowledge of the effectiveness of interventions is required. However, heterogeneity among findings across studies of intervention effectiveness is a common conclusion for many reviews that assess non-pharmacological interventions that target children with or at risk of ADHD. 62,63,72,74 Although average, weighted, effect sizes may help to establish the importance of non-pharmacological interventions in improving outcomes for children with or at risk of ADHD, it is critical that the components linked with the most successful interventions are identified so that the design, implementation and replication of future interventions can make the best use of scarce resources.

Repeated calls have been made for precise specification of what makes one behaviour change intervention more effective than another and how this can be understood theoretically (e.g. Rothman93). A range of programme features have been highlighted as potentially moderating the effectiveness of interventions that target children with or at risk of ADHD; these include characteristics of the child participant (e.g. age, medication status, sex), characteristics of the intervention (i.e. the change techniques constituting intervention content) and characteristics related to the delivery of the intervention (e.g. intensity of the intervention, fidelity of delivery). 64 The identification of programme features that are necessary for effective intervention implementation in school contexts will facilitate links between particular intervention components and effectiveness, and in turn could help resources to be used more efficiently and benefit children displaying ADHD symptoms, their carers and service providers.

Through the consideration of relevant qualitative research alongside the synthesis of quantitative studies this review has the potential to provide explanations of why particular interventions are effective and what factors operate as catalysts and barriers to effectiveness. The review will also identify any significant areas of uncertainty with regard to school-based interventions for ADHD and recommend any future research needed to address them.

Aim and research questions

The broad aim of this series of systematic reviews and their overarching synthesis is to evaluate the effectiveness and cost-effectiveness of non-pharmacological interventions delivered in school settings for children with, or at risk of, ADHD and to explore the factors that may enhance, or limit, the delivery of such interventions.

Four reviews were conducted. Review 1 (see Chapter 2) synthesises the effectiveness and the cost-effectiveness of non-pharmacological interventions delivered in school settings for children with or at risk of ADHD. Review 2 (see Chapter 3) considers quantitative studies that explore attitudes towards school-based non-pharmacological interventions for pupils with ADHD. Review 3 (see Chapter 5) synthesises the attitudes and experiences of children, teachers, parents and others using ADHD interventions in school settings. Review 4 (see Chapter 6) explores the experience of ADHD in school among children, their parents and teachers more generally. The four reviews are subsequently brought together in an overarching synthesis (see Chapter 7). Each review addresses particular research questions as outlined below.

Overarching synthesis

The aim of the overarching synthesis is to synthesise findings from reviews 1–4.

Research questions

This chapter describes systematic review 1 and addresses the following three research questions:

1. Are non-pharmacological interventions delivered in school settings for children with, or at risk of, ADHD effective in improving:

   1. Core ADHD symptoms (inattention, hyperactivity/impulsivity, ADHD combined)?

   2. ADHD-related symptoms (externalising symptoms, internalising symptoms, social skills)?

   3. Scholastic behaviours and outcomes (perceptions of school adjustment, curriculum achievement, standardised achievement)?

2. Is effectiveness moderated by particular programme features?

3. Have such interventions been shown to be cost-effective?

Methods

Results

Number of studies included

The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) diagram152 in Figure 1 summarises the search process.

FIGURE 1.

Preferred Reporting Items for Systematic Reviews and Meta-Analyses diagram showing search process and study selection for review 1.

After the removal of duplicates, a total of 15,481 records were screened at title and abstract stage and a total of 655 potentially relevant articles were identified for which full texts were required. Of these, 567 (87%) were successfully retrieved, and of those that were not obtainable, 24 were foreign-language papers.

Research questions 1 and 2: evaluations that assess child outcomes

Fifty-four English-language papers met our full-text screening criteria and were included in the synthesis. Of the seven foreign-language papers retrieved and translated by BG, AJ, AV, SI, HK, none met our inclusion criteria.

Research question 3: evaluations that assess economic outcomes

Of the 54 studies that met the full-text screening criteria, there were no studies that included economic outcomes. Consequently, there are no further results in relation to research question 3 about the cost-effectiveness of interventions that target children with or at risk of ADHD in school settings.

Please refer to Appendix 3 for a complete list of reasons for the exclusion of 601 papers after full-text screening.

Analysis

Discussion

This review synthesised studies that spanned 32 years of research about the effectiveness of non-pharmacological interventions that target children with, or at risk of, ADHD in school settings. Fifty-four evaluations that reported on one or more of nine commonly measured child-related constructs assessing aspects of the condition were synthesised. Informant types (i.e. parent, teacher, child and observer) were also distinguished. In addition to the investigation of whether or not non-pharmacological interventions are effective (research question 1), the review identified and tested a wide range of programme features that could impact on effectiveness, including the study design, participant characteristics, intervention package(s) and delivery elements of non-pharmacological interventions that target children with, or at risk of, ADHD in school settings (research question 2). The cost-effectiveness of non-pharmacological interventions could not be assessed as no studies were identified (research question 3).

Overall, the results provide support for the beneficial effects of non-pharmacological interventions on outcomes related to ADHD. Positive effects were observed for relatively objective assessments (including neurocognitive assessments and tests with objective performance criteria) and for some teacher perception-based measures but not for parental and child perception-based measures. The results indicate that interventions in this field are complex; they are typically composed of multiple features and few interventions consist of common sets of intervention elements. The heterogeneity of the interventions studied was compounded by the generally low methodological quality of included studies and the absence of an agreed set of outcome measures for assessing aspects related to ADHD. Applying Cohen’s guidelines for interpreting effect sizes, mean weighted effect sizes across outcomes ranged from very small (d₊ < 0.20) to large (d₊ ≥ 0.80), but 95% CIs were wide and substantial heterogeneity in effect size estimates across studies was reported. Moderator analyses did not clarify which of the particular programme features are linked with effectiveness (research question 2). The meta-analysed RCTs (n = 36) offer the highest level of evidence and therefore provide the most methodologically robust basis for the following discussion. We will, therefore, (1) review the magnitude of effect sizes across constructs and informants; (2) review the meta-regression results and the effect of various programme features on the success of interventions; (3) compare our findings with those of previous reviews; (4) identify limitations of this review; (5) reflect on the design and evaluation of interventions to optimise outcomes for children with, or at risk of, ADHD; and (6) highlight key conclusions for research and possible implications for practice.