Digital interventions in mental health: evidence syntheses and economic modelling

When a waiting list goes beyond ‘no intervention’

By default, we have classified WL as ‘no intervention’, assuming no regular input from the research team and no changes in usual care because of the research study. If a WL (or usual care) was enhanced through regular contact or therapeutic materials from the research team then we classified it as ‘non-therapeutic active control’. For example, in Johansson et al. ,37 participants on the WL received weekly assessments and substantial non-directive support via an online system by therapists, so this was classified as SDC. Teng et al. 38 invited all WL participants to have a weekly face-to-face assessment with a research assistant in a laboratory, which was classified as SNoDC.

In Pham et al. ,5 patients on the WL also received a weekly newsletter with curated content on breathing retraining exercises, matching content to the intervention they were to receive after coming off the WL, and e-mail reminders to complete assessments, albeit without any personal interaction; this was classified as UDC, although we could argue that the extent and therapeutic content of the information was on a par with an intervention. As the authors called this a ‘waiting list’, and the participants knew that they were being given information while waiting to receive an intervention, irrespective of the potential therapeutic effect of the WL, it could be classified only as a control.

In a study by Lovell et al. ,39 WL was a period of NI followed by individual high-intensity CBT. As the majority of the participants randomised to WL/usual care received treatment within the duration of the study, the comparator for internet CBT was more akin to a standard therapy than to NI. This raises the importance of monitoring and reporting what usual care is within RCTs so that sensitivity analyses can take into account the actual interventions that were received by the control group, rather than making assumptions about what the control group received.

There are occasions40,41 when participants on the WL engage in a research activity that would not be available to those receiving usual care (e.g. the provision of weekly online ratings or the option to telephone the research team if they encounter problems). In this case, ‘no intervention’ is still the appropriate classification because there participants in such a group receive no regular input from the research team (e.g. measurement of outcomes or scheduled ‘check-ins’ with participants).

When a comparator labelled as ‘therapy’ is better classified as a ‘control’

It can be difficult to classify an activity as ‘therapy’ or ‘control’ when the same activity could be therapeutic in some contexts but not in others. For example, in a study reported by McCrone et al. ,42 one of the randomisation arms was a computerised relaxation programme that was delivered in the same way as the intended intervention (i.e. cCBT for OCD) in a clinic using a self-administered software-based programme. Although applied relaxation can be used as a treatment for some conditions, such as GAD, there is no evidence or indication that it is a bona fide treatment for OCD, so the study used this as a psychological placebo.

In another study by Andersson et al. ,43 internet CBT was compared with ‘internet support therapy’, which was intended as a ‘control over attention effects and possible alleviating effects in having contact with a professional therapist’. The internet CBT was a self-help intervention that included 100 pages of materials and worksheets with established therapy components as well as weekly homework and written feedback from therapists through an integrated treatment platform. Internet support therapy included supportive communication with a therapist via the same integrated treatment platform but without the CBT content or homework. Interacting with a therapist via an online platform in a supportive way could have been classified as a therapeutic activity for a condition such as depression but not for OCD; therefore, the internet support therapy on this occasion was a control rather than an intervention.

When a technology-enabled intervention is classified as ‘non-digital’

We have made the distinction between DIs and telecommunication media that may use a digital interface (e.g. a computer or a mobile phone) without information processing. For example, short message service (SMS) texts or e-mails between a therapist and a user are not classified as ‘DIs’, whereas an online or telephone-based messaging system, as used in Johansson et al. ,37 that includes specially constructed backend software to provide a bespoke therapy environment with message encryption and centralised monitoring is classified as a DI. In another scenario, videoconferencing is not a DI unless it is part of a platform with software processing that guides some elements of the intervention.

When unsupported interventions/controls include some type of ‘contact’

In their most basic form, unsupported interventions do not involve any interpersonal contact but are entirely self-administered. Some unsupported interventions may include one-way reminders, postcards or telephone messages, or a helpline to call if there are any problems. What differentiates these from supported interventions is that, in the case of supported interventions, there is a regular and expected two-way interaction between the user and a facilitator or between peers.

Mixed interventions

Interventions could be a mix of medication and psychosocial/behavioural interventions (e.g. when a DI is used to assist a participant in taking their medication and to monitor adherence). Medication that is a usual/routine care intervention may still be part of a ‘psychosocial intervention’ group, a ‘no intervention’ group or an ‘active control’ group. If participants are offered psychosocial support as part of a controlled medication trial, then the intervention is classified as ‘medication’. Participants may still be offered medication as part of usual care in addition to the psychosocial intervention (the medication is not a ‘research intervention’).

Granularity of taxonomy

Digital interventions are complex and may include different therapeutic components (e.g. psychoeducation, cognitive techniques, behavioural techniques and motivational interviewing) and different layers of intensity (e.g. weekly 1-hour sessions or brief interventions). In addition, DIs increasingly follow a mixed model in which one intervention may include different types of technologies (e.g. telephone, website, biofeedback) and different types of personal support (e.g. face-to-face sessions with a therapist, telephone calls for technical support or standardised texts and e-mails from an assistant).

Searches

Material in this section has been reproduced with permission from Jankovic et al. , Systematic review and critique of methods for economic evaluation of digital mental health interventions, Applied Health Economics and Health Policy, published 2020. 45 Copyright © 2020, Springer Nature Switzerland AG.

In December 2018, the following databases were searched to identify published and unpublished studies: MEDLINE, PsycInfo^® (American Psychological Association, Washington, DC, USA), Cochrane Central Register of Controlled Trials (CENTRAL), Cochrane Database of Systematic Reviews (CDSR), Cumulative Index to Nursing and Allied Health Literature (CINAHL) Plus, Database of Abstracts of Reviews of Effects (DARE), EMBASE™ (Elsevier, Amsterdam, the Netherlands), Web of Science™ (Clarivate Analytics, Philadelphia, PA, USA) Core Collection, NHS Economic Evaluation Database (NHS EED), the Health Technology Assessment database and the National Institute for Health Research (NIHR) Journals Library and the Database of Promoting Health Effectiveness Reviews (DoPHER). The full search strategy is presented in Report Supplementary Material 1.

We also searched two clinical trial registries for ongoing studies (ClinicalTrials.gov and the WHO’s International Clinical Trials Registry Platform portal), searched the NIHR portfolio and conducted web searches using Google (Google Inc., Mountain View, CA, USA) and Google Scholar (Google Inc.) using simplified search terms. After searches were completed, we searched the studies included in the relevant systematic reviews identified as well as the references cited in the included studies and conducted forward citation chasing on all identified protocols and conference abstracts. We also contacted researchers in the field and searched the reference lists of the selected studies.

The searches were conducted from 1997, as no relevant studies of DIs could have been published before this date. Searches were restricted to studies written in English, as we anticipated that most economic studies written in other languages would also have a version published in English (e.g. the South Asia Cochrane Group). 46

Selection criteria

Material in this section has been reproduced with permission from Jankovic et al. , Systematic review and critique of methods for economic evaluation of digital mental health interventions, Applied Health Economics and Health Policy, published 2020. 45 Copyright © 2020, Springer Nature Switzerland AG.

Eligible studies included participants with symptoms or at risk of mental health problems as defined by the International Classification of Diseases, Eleventh Revision,1 criteria for mental, behavioural or neurodevelopmental disorders, with the exception of the conditions listed under the categories of neurodevelopmental, neurocognitive and disruptive behaviour or dissocial disorders. Studies were also excluded if the primary diagnosis of the participants was a physical or other condition other than those listed (e.g. cancer or insomnia). All DIs that expressly targeted mental health outcomes and patient-facing therapeutic activities were included, with the exception of those that were simply a communication medium (e.g. telephones or videoconferencing). A broad range of studies was considered in the review, including economic evaluations conducted alongside trials, modelling studies and analyses of administrative databases. Only full economic evaluations that compared two or more options and considered both costs and consequences (i.e. CMA, CEA, CUA and CBA) were included in the review. No studies were excluded on the basis of their comparator group. Study protocols, abstracts and reviews were marked to facilitate follow-up as described in Searches.

Study selection

Two reviewers, one of whom had expertise in health economics, independently assessed all titles and abstracts for eligibility. If either reviewer indicated that a study could be relevant, the full text for that study was sought and again assessed independently by two reviewers, with disagreements resolved through discussion or with a third reviewer.

Data extraction

The purpose of data extraction was to summarise methodology and identify challenges common in the evaluation of DIs. To do so, we extracted information reported in the studies on the population, intervention (underpinning principles, delivery mode, level of support, treatment duration) comparators, outcomes (clinical and economic outcomes, and the economic end point), study design, analytical approach (within-trial analysis, decision model, statistical model, epidemiological study), analysis time horizon, setting (country and analytical perspective), analytical framework (CMA, CEA, CUA, CCA or CBA) and methods employed to characterise uncertainty.

Critical analysis

Material in this section has been reproduced with permission from Jankovic et al. , Systematic review and critique of methods for economic evaluation of digital mental health interventions, Applied Health Economics and Health Policy, published 2020. 45 Copyright © 2020, Springer Nature Switzerland AG.

The identified studies were critically reviewed, to assess whether or not the existing evidence meets the requirements for decision-making in health care44 and to assess the challenges in generating cost-effectiveness evidence in this context. The following questions were asked of the methods used in the included studies:

• Does the economic analysis estimate both costs and effects?

• Does the analysis appropriately synthesise all of the available evidence?

• Are the full ranges of possible alternative interventions and clinical strategies included?

• Are costs and outcomes considered over an appropriate time horizon?

Quality assessment

Checklists are useful tools to assess the quality and applicability of economic evaluations. They also provide a framework for summarising the methods and results of economic evaluations in a consistent manner. Having an appropriate quality assessment checklist for economic evaluations of DIs in mental health can improve reporting standards and encourage harmonisation of key methods likely to drive heterogeneity in results from different economic evaluations. Following a checklist can help structure the narrative synthesis of results and critique of the methods used. This process helps the comparison of results across different studies, which are likely to be driven by methods employed to determine cost-effectiveness.

There are several checklists used in reviews of health economic evaluations, as summarised by Watts and Li. 47 Their paper notes that the Drummond48 checklist [or BMJ (British Medical Journal) checklist, as Watts and Li call it] and the Philips49 checklist are the most commonly used and are well regarded by the health economics community. Using the Drummond48 checklist, we assessed the included studies according to the clarity of their research questions, the quality and completeness of data used in the economic evaluation, the methods used to characterise uncertainty in the evaluation model, and the interpretation of their results. We have also used the Philips49 checklist, which is specific to model-based economic evaluations and describes attributes of good practice and questions for critical appraisal according to the model’s structure, data and consistency.

Summary of available economic evaluations of digital interventions

Material in this section has been reproduced with permission from Jankovic et al. , Systematic review and critique of methods for economic evaluation of digital mental health interventions, Applied Health Economics and Health Policy, published 2020. 45 Copyright © 2020, Springer Nature Switzerland AG.

Our systematic literature search and study selection identified 63 primary studies,15,39,42,43,50–108 whose results were reported in 67 papers,15,39,42,43,50–112 as shown in Figure 6. After the removal of duplicates, 6931 of the 10,764 records originally identified remained and were screened, of which 6645 were excluded by title and abstract and 286 were assessed for eligibility by reviewing their full text. A total of 219 papers were excluded because the primary diagnosis was not a mental health problem (n = 27); the intervention was not a mental health intervention (n = 13); the study did not include health economic outcomes (n = 36); it was not an economic evaluation, or it was a review rather than a primary study (n = 10); it was a conference abstract rather than a peer-reviewed paper (n = 26); or it was a duplicate reference (n = 4) or a protocol (n = 103). Report Supplementary Material 2 gives the references of the excluded studies and the reasons for exclusion.

FIGURE 6.

The PRISMA flow diagram for the identification and selection of cost-effectiveness studies relating to DIs in mental health (search 1: up to November 2018). HE, health economic. Reproduced with permission from Jankovic et al. , Systematic review and critique of methods for economic evaluation of digital mental health interventions, Applied Health Economics and Health Policy, published 2020. 45 Copyright © 2020, Springer Nature Switzerland AG.

Reproduced with permission from Jankovic et al. , Systematic review and critique of methods for economic evaluation of digital mental health interventions, Applied Health Economics and Health Policy, published 2020. 45 Copyright © 2020, Springer Nature Switzerland AG.

Two papers89,109 reported the results of identical analyses from the same study; therefore, the paper by Duarte et al. 109 was incorporated in the summary with Littlewood et al. 89 to avoid duplicate reporting. Two papers62,112 reported results from the same economic evaluation using different perspectives: the employer’s62 and the societal. 112 Three papers69,110,111 used the same sample and interventions but different economic evaluation methods: El Alaoui et al. ’s110 paper was based on the provider’s perspective and a 4-year time horizon, Hedman et al. 69 adopted a societal perspective and a 6-month time horizon, and Hedman et al. ’s111 paper was based on a societal perspective and a 4-year time horizon. It is worth highlighting that two papers43,51 were considered as reporting separate studies, although they used the same sample, as the study reported in the second paper51 was an extension of that reported in the first. 43 In conclusion, there were 66 papers with unique economic analyses and 63 studies with separate samples and interventions (hereafter referred to as 66 economic evaluations and 63 studies).

Approximately two-thirds of the studies (45/63) evaluated interventions that target anxiety and/or depression. Other conditions included suicidal ideation (n = 1), child disruptive behaviour (n = 1), eating disorders (n = 3), schizophrenia (n = 3) and addiction, including drug and alcohol addiction (n = 3 and n = 2, respectively) and smoking cessation (n = 8) (Table 5).

The interventions varied both between and within individual conditions, in terms of their underlying principles (e.g. CBT, guided relaxation, self-help, exposure therapy, motivational support for smoking cessation), content (e.g. the number of modules), mode of delivery (e.g. mobile, computer or text based, or completed at home or at clinic), type of support (e.g. online chat, telephone, face to face), frequency of support (e.g. weekly, ad hoc), person delivering support (e.g. clinician, assistant, lay person) and extent of support (e.g. administrative support only, additional counselling) (see Table 10).

Comparators can be broadly categorised as NI, which could be WL or usual care, active non-therapeutic controls or standard therapy. There is limited reporting of what interventions patients had access to, or received, when allocated to a WL or to usual care. Non-therapeutic controls were designed as ‘psychological placebos’ or attention controls; they encourage patients to spend the same amount of time on treatment as active interventions, but without the ‘active’ component, such as CBT or problem-solving therapy. Non-therapeutic comparators included websites, printed reading and online relaxation not indicated as a ‘treatment’ for the condition under study (see Table 10).

Methods for costing interventions varied. The majority of the evaluations (50/66)15,39,42,43,50–75,77,80,82–84,88–90,92,95–98,100–106,110–112 included the cost of staff time required to deliver the intervention. Some studies included variable equipment costs and website maintenance and hosting, whereas very few considered the cost of development, capital costs or patient recruitment (or technology dissemination).

Most evaluations (52/66)15,39,42,43,50–75,77,80,82–84,88–90,92,95–98,100–106,110–112 were within-trial analyses, and one59 included a temporal extrapolation. A further three studies76,93,107 were within-pilot evaluations76,93 and feasibility trials,107 although one study79 used observational data. Ten studies67,78,81,85–87,91,94,99,108 used decision models to evaluate interventions. Eight78,81,85–87,91,94,99 of the 10 studies67,78,81,85–87,91,94,99,108 that used modelling to evaluate DIs did so because of the absence of head-to-head trial data, and used individual trials or non-comparative data sources to inform the treatment effect of DIs. The model for eating disorders used synthesised evidence of the treatment effect to derive the cost of different treatment options.

The vast majority of the papers reported some form of sensitivity analysis, with only 11 studies57,63,74,77,79,80,85,97,100,102,107 not reporting having conducted any. In total, 50 studies15,39,42,43,50–75,77,80,82–84,88–90,92,95–98,100–106,110–112 used both probabilistic and deterministic sensitivity analyses. Deterministic sensitivity analysis involved exploring alternative scenarios and assumptions; the most common parameter value that was varied was the cost of intervention. Other common scenarios included alternative methods for dealing with missing data or for estimating costs and effect.

Quality of the included studies under each item of the Drummond checklist

Table 6 reports the outcomes of the quality assessment for each economic study based on the Drummond et al. 48 checklist, which comprises 10 items that assess the following domains: (1) question definition, (2) description of competing alternatives, (3) established effectiveness, (4) inclusion of costs and consequences, (5) measurement of costs and consequences, (6) valuation of cost and consequences, (7) adjustment of costs and consequences for differential timing, (8) incremental analysis, (9) allowance for uncertainty and (10) presentation and discussion of results.

Quality of the included studies under each item of the Philips checklist

Tables 7–9 summarise the quality assessment for each model-based study based on the Philips et al. 49 checklist. The analysis methods are generally good; for example, in the majority of the studies the objective was clear (item S1), the choice of model was appropriate (item S6), structural assumptions were transparent and appropriate (items S3 and S4) and the sources of data for baseline outcomes were clear and justified (item D2a). The most common limitations were associated with the comparators (item S5), the time horizon (item S7) and the treatment effect (D2b). Specifically, none of the studies provided a systematic comparison of all possible comparators; the time horizon was either incomplete (potentially failing to capture the long-term effect of DIs) or unclear in 878,81,85–87,91,94,99 out of the 10 studies,67,78,81,85–87,91,94,99,108 and the treatment effect was synthesised from multiple trials in only one study78 (although the methods were not explicitly reported). In the remaining models, four studies87,91,99,108 informed the treatment effect using a single RCT, three studies81,85,86 used indirect evidence and two studies67,94 used a hypothetical treatment effect.

Estimation of costs and outcomes

The included studies use a variety of methods to measure costs and outcomes of DIs. Benefits were measured in terms of QALYs, disability-adjusted life-years, life-years, disease-free days, disease-specific outcome measures, response or clinical improvement, inpatient days avoided or days of abstinence in interventions that target addiction. Costs attributed to DIs included the cost of staff time required to deliver the intervention, a range of equipment costs, website maintenance and hosting, the cost of development, capital costs and the costs of patient recruitment (or technology dissemination).

The optimal method for measuring outcomes ultimately depends on the analysis perspective. An employer may be interested in measuring the effect of the intervention on productivity, a mental health-care provider may include a narrow range of benefits specific to the mental health condition targeted by the intervention and costs that fall on that provider, whereas a health system may aim to improve overall health, and so requires a broader health measure such as health-related quality of life (HRQoL) to allow comparison across different fields of medicine.

Although the majority of the studies that were identified in this review reported a range of different costs and outcomes, seven studies,52,54,59,66,74,86,90 evaluated interventions from a health-care system or payer perspective, but measured outcomes in terms of changes in clinical scores,59,66,86,90 clinical improvement/response/remission52,54 or disease-free days. 74 It is not clear how health gains in such disease-specific outcome measures can be used by decision-makers to allocate resources across different disease areas.

Similarly, the appropriate methods for measuring costs depend on the analysis perspective (e.g. whether to include the cost to employer, service provider, broader health system or the society), as well as the role of the intervention. When interventions target undiagnosed patients who would not have sought care otherwise, dissemination (e.g. advertising or public health campaigns) is an integral part of the intervention that is likely to affect its outcomes, and so the cost of dissemination should be included in the cost of the intervention. Conversely, when an intervention targets diagnosed patients, and is prescribed by clinicians, the dissemination costs are likely to be negligible. Studies that recruit self-referred patients through advertising rarely include recruitment costs in their analysis. Furthermore, the costs of developing and maintaining DIs are highly uncertain because very few studies included capital costs (e.g. computers, staff training or one-off software purchases), or costs for website maintenance and hosting. The subsequent cost per patient depends on the scale of roll-out, where wider delivery (e.g. providing an intervention nationally) is likely to dilute such fixed costs.

Use of all available evidence

Evidence synthesis is more complex for DIs than for interventions such as medication, as DIs are multilayered and subject to external factors, and, therefore, it is difficult to ensure uniform delivery or to disentangle the impact of each layer on outcomes. There is significant heterogeneity likely between interventions, the populations they target and the settings in which they are delivered; even when interventions target the same condition, they tend to vary in their underlying principles, content, and the type and extent of support. It is not clear whether each of the characteristics affect the treatment effect, or whether evidence from similar interventions can be reasonably pooled to make an overall recommendation about their cost-effectiveness.

Furthermore, interventions for the same mental health disorder can target different patient populations (e.g. according to disease severity). The population in which an intervention is evaluated can affect its comparability with other trials (i.e. it may not be appropriate to generalise costs and treatment effect observed in one target population to another, or to attempt to synthesise effectiveness of interventions observed in different patients populations).

Finally, DIs as well as comparators that involve behavioural therapy are likely to vary between settings (clinics and countries) in the referral system, capacity, waiting times and frequency of contact, and so synthesising evidence on resource use may not be appropriate.

Specification of comparators

Comparators included supported and unsupported DIs, medication, different types of face-to-face therapy, WL and usual care. The majority of economic evaluations were based on two-arm RCTs. WLs and usual care were the most common comparators. Their description was often limited and the distinction between them was not always clear. Treatment of mental health conditions tends to vary between health providers, and between different patient populations (e.g. diagnosed vs. undiagnosed), so a lack of understanding of a comparator in a trial can limit generalisability of the findings, as well as comparability of results across trials.

Time horizon for analysis

The majority of the evaluations were conducted alongside a trial or used retrospective data from a single study. Most economic evaluations did not explore the results beyond the trial end point, potentially failing to capture long-term costs and effects of DIs. This is considered inadequate for decision-making owing to the truncated time horizon. Mental illness is a lifetime condition for many patients, with periods of respite and relapse during which costs and outcomes can be influenced by any potential treatment received. Limiting the time horizon of an analysis can generate inaccurate estimates of cost-effectiveness. The lack of long-term modelling is likely to be due, in part, to the lack of reliable data about the long-term performance of DIs. In many cases, there are no empirical data on the duration of treatment effects and how these relate to a changing baseline (i.e. how illness progresses in the population that does not receive the treatment). This issue is likely to be confounded by comorbidities and future events, making long-term extrapolation challenging.

Digital interventions were dominant against alternatives

• DIs were dominant against NI for:

  • depression97,106

  • health anxiety52 (only from societal perspective and not from a health-care perspective)

  • OCD39,88 (only at 12 months’ follow-up)

  • stress62,112

  • smoking. 67,99

• DIs were dominant against non-therapeutic controls for:

  • anxiety53 (against attention placebo)

  • suicidal ideation104 (against information websites)

  • health anxiety68 (against online forum without therapeutic elements)

  • schizophrenia63 (against watching a video; this study suggested lower costs with better outcomes but did not calculate ICERs).

• DIs were dominant against NoDIs for:

  • panic disorder (Bergström et al. 54 against group CBT, McCrone et al. 90 against individual CBT)

  • social anxiety69,110,111 (against face-to-face CBT; note, the three economic evaluations were based on the same clinical trial)

  • GAD85 (against face-to-face CBT or medication)

  • mixed depression and anxiety86 (against medication and face-to-face CBT)

  • binge eating disorder83 (against face-to-face CBT when outcomes were expressed in QALYs rather than number of binge-free days)

  • OCD39 (against manual-based self-help at the 12-month but not 3-month follow-up).

Digital interventions achieved better outcomes with higher costs against alternatives

• DIs achieved better outcomes with higher costs against NI for:

  • addiction80,95

  • depression15,55,64,71,72,81,84,89,103,105

  • eating disorders50,78

  • health anxiety52 (from a health-care but not a societal perspective)

  • OCD43,51

  • panic disorder91

  • smoking59,93,94

  • social anxiety102

  • stress/worry/anxiety. 61,74

• DIs achieved better outcomes with higher costs against non-therapeutic controls for:

  • depression58 (against non-specific education)

  • smoking108 (against non-specific education).

• DIs achieved better outcomes with higher costs against NoDIs for:

  • health anxiety52 (against manualised self-help).

Digital interventions were dominated by their alternatives

• DIs were dominated by NI for:

  • depression89 (Beating the Blues only)

  • substance misuse92 (when outcomes were measured in QALYs rather than abstinent years).

• DIs were dominated by NoDIs for:

  • health anxiety52 (against manual-based self-help)

  • depression82 (against bibliotherapy).

It is worth noting that the following studies found that DIs did not confer any added value in terms of outcomes against their alternatives:

• DIs compared with NI for:

  • OCD39 (at 3-month follow-up).

• DIs compared with non-therapeutic controls for:

  • depression and/or anxiety96,107 (against attention control)

  • schizophrenia100 (against a ‘dummy’ version of the DI).

• DIs compared with NoDIs for:

  • child disruptive disorder76 (against clinic-based parenting programme)

  • depression and anxiety79 (against individual CBT)

  • eating disorder83 (against individual CBT when outcome was number of binge-free days)

  • OCD39 (against manualised self-help at 3-month follow-up)

  • schizophrenia77 (against clinician sessions)

  • severe depression81 (against audio-based self-help)

  • smoking60 (against specialist clinic smoking cessation programme).

This panoramic overview of the results of individual studies reiterates the complexity in interpreting the results of economic evaluations; even within the same study, results were different depending on how outcomes were expressed (e.g. QALYs rather than abstinent years92 or binge-free days83), the different time horizons (e.g. 12 months rather than 3 months39) or the different perspectives (e.g. societal vs. health care52). When DIs were compared with NI or non-therapeutic controls, individual studies suggested that the DIs either were dominant or achieved better outcomes with higher costs. When DIs were compared with NoDIs, such as individual or group CBT, several studies indicated dominance of the DIs studied, but equally as many studies found that DIs did not confer any added value.

Search strategy

In December 2018, the following databases were searched to identify published and unpublished studies: MEDLINE, PsycInfo, CENTRAL, CDSR, CINAHL Plus, DARE, EMBASE, Web of Science Core Collection, DoPHER and ProQuest^® (ProQuest LLC, Ann Arbor, MI, USA).

We searched two clinical trial registries for ongoing studies: ClinicalTrials.gov and the WHO’s International Clinical Trials Registry Platform portal. We also searched the NIHR portfolio, and conducted web searches using Google and Google Scholar using simplified search terms. After these searches were complete, we scanned the lists of included studies of relevant systematic reviews identified by the searches and the reference lists of all included studies; conducted forward citation chasing on all identified protocols, conference abstracts and the included studies using Google Scholar for any relevant publications; and contacted the authors of included studies for information on any other work in the field they were aware of.

The searches were conducted from 1997, as we did not anticipate finding relevant DIs from before that date, and were restricted to those written in English, as we anticipated that most studies written in other languages would also have a version published in English (e.g. the South Asia Cochrane Group46).

In June 2019, the searches were updated and widened to included terms based on unspecified anxiety disorders. At this time, we also conducted an additional pilot search using terms based on ‘worry’ and ‘anxiety prevention’ using only the Cochrane Library and PsycInfo databases to ensure that no articles were being missed. As no new included articles emerged from this pilot search, it was not deemed necessary to expand this to all the databases. The full search terms and outputs of the database searches are provided in Report Supplementary Material 3 and 4.

Study identification and selection

Two reviewers independently screened all titles and abstracts of the identified studies against our inclusion/exclusion criteria. If either reviewer thought that a study could be relevant, we retrieved the full text. The same two reviewers independently assessed the full texts against our inclusion/exclusion criteria. A third reviewer resolved any disagreements through discussion and arrived at the final list of included and excluded studies.

Eligible studies included those that featured the following:

• Participants with symptoms or who were at risk of GAD within mental health populations or within the general population; we defined this as a certified diagnosis using a standardised diagnostic interview or a score above an accepted cut-off point for diagnosable GAD in standardised questionnaires.

• Software-based systems and technology platforms designed for patient-facing delivery of a mental health intervention (i.e. an intervention to improve mental health outcomes).

• All comparisons relevant to DIs, even when two or more DIs were compared with each other without other comparators.

• GAD-specific measures of anxiety or worry, reported for GAD populations or GAD subsamples within mixed populations.

• Randomised controlled trials to minimise risk of bias and confounding variables.

We excluded studies that featured the following:

• Mixed populations of GAD with other conditions, when the outcomes were not reported separately for GAD subgroups.

• Technology used as a means for telecommunication (e.g. e-mail, telephone or video) without any software-based processing.

• Software-based systems designed for the training of health professionals or for administration without any patient-facing intervention components.

• Outcomes that were not mental health related.

• Study protocols, abstracts and reviews; these were marked so that we could check for RCTs that we may have missed in the database searches.

Data extraction and risk-of-bias assessment

Two researchers independently extracted data from published and unpublished study reports. Data were extracted on the sample, study design, intervention and comparator characteristics, baseline characteristics and results. Any discrepancies were resolved by a third reviewer. The same two reviewers who completed the data extraction (DM and HM) independently assessed the risk of bias of each study using the Cochrane risk-of-bias tool (RoB 2)137 and resolved disagreements by discussion with a third reviewer (RC). We assessed the risk of bias for each outcome measure reported in a study in the following six domains: randomisation process, deviations from intended interventions, missing outcome data, measurement of the outcome, selection of the reported results and overall bias. The risk-of-bias tool137 classifies each study as high risk, posing some concerns, or low risk based on responses to each domain.

Data synthesis and statistical analysis

An analysis of covariance (ANCOVA) modelling framework was used, in which the final measurement is synthesised, adjusting for baseline outcome measurements. With the ‘change from baseline’ approach, the ANCOVA avoids guessing within-patient correlation to enable the calculation of the standard error of change and subsequent sensitivity analyses for different correlation values. Treatment effect estimates based on ANCOVA methods have been shown to be more efficient, less biased and robust to chance baseline imbalance. 138–143 Hence, ANCOVA is the preferred method for estimating treatment effects from continuous outcomes. 144–147

A modelling approach in line with parameterisation for continuous data with normal likelihood and identity link used by Dias et al. 138,139 is taken throughout. Fixed-effects (FE) and random-effects (RE) models (accounting for potential correlation within multiarm trials) were fitted to the data, with each outcome of interest being modelled independently. In the model, patients who did not receive any treatment are expected to neither improve nor worsen during treatment (i.e. null placebo effect). In addition, it was assumed that the effect of the baseline measurement is common across all treatments, in other words that, when two active treatments are compared in a trial, the baseline effects are cancelled out.

All analyses were conducted within a Bayesian Markov chain Monte Carlo (MCMC) approach and fitted using WinBUGS software version 1.4.3, 2007 [Medical Research Council (UK) and Imperial College (UK)]148 and linked to the freely available software R (version 3.6.0; The R Foundation for Statistical Computing, Vienna, Austria) through the package R2WinBUGS. 149 In all models the MCMC Gibbs sampler was initially run for 10,000 iterations, and these were discarded as ‘burn-in’. Models were run for at least a further 5000 iterations, on which inferences were based. Chain convergence was checked using autocorrelation and Brooks–Gelman–Rubin diagram diagnostics. 150–152 Goodness of fit was assessed using the DIC (deviance information criterion) (with differences of ≥ 3 assumed to be important) and posterior mean residual deviance. 153,154

We presented the estimated results as relative treatment effect scores [and associated 95% credible intervals (CrIs)] in the selected outcome measures. We have estimated the probability of a treatment being the ‘best’ (i.e. being the most clinically effective),155 and presented rankograms for all interventions, which provide the probabilities of an intervention being ranked 1 (the highest) to 7 (the lowest). Finally, we reported the surface under the cumulative ranking curve (SUCRA), which is a numerical presentation of the overall ranking of each intervention. SUCRA values range from 0% to 100%: interventions are more likely to be placed towards the top ranks the closer their SUCRA value is to 100% and towards the bottom ranks the closer their SUCRA value is to 0%. 138,156

Appendix 1 gives further details on the analysis, including annotated synthesis WinBUGS code, sample data and initial values for the main synthesis model used (see Table 19).

Assessment of heterogeneity and consistency

For both end points, the model was extended to include study-level covariates as potential treatment effect modifiers. Clinical expectations were that potential sources of heterogeneity could include disease severity, concomitant medication and comorbidities. Meta-regression is the most commonly employed method to explore the influence of particular study-level covariates on the relative effect. However, this method requires that all studies report data on the covariate(s) in question. For the trials informing the NMA, complete data for disease severity (as a binary covariate mild to moderate/moderate to severe) were obtained, but not for the other two potential effect modifiers. To preserve all studies (and treatments), when a covariate was not reported by some studies, we allowed the model to impute missing covariate information (multiple imputation procedure assuming ‘missingness’ mechanism of ‘missing at random’).

As per guidance by Dias et al. ,156,157 inconsistency was assessed by comparing the DIC of our primary analyses (based on NMA models that assume consistency between direct and indirect evidence) and the DICs yielded by inconsistency models (which provide effect estimates based on direct evidence only). Results were assessed for coherence by qualitatively comparing estimates of pairwise ANCOVA meta-analysis (direct) and ANCOVA RE NMA (direct and indirect).

Sensitivity analysis

We conducted two types of sensitivity analysis. First, we evaluated how sensitive the networks were to individual trials. When network links were informed by more than one trial, we removed each trial one at a time (giving n – 1 for each analysis) and investigated the impact on the probability of each intervention being ‘best’. Second, we assessed the robustness of the synthesis results by repeating the analysis while excluding studies of fewer than 30 patients.

Included and excluded studies

Initial systematic searches of bibliographic databases identified 16,272 records; in addition, 32 records were identified through secondary searches (e.g. citation searching of protocols and abstracts). After duplicates were removed, a total of 8920 records were screened by title and abstract, and 8543 records were excluded. We retrieved the full-text papers for the remaining 377 records and, as a result of further screening, 352 articles were excluded. In total, 21 studies (reported in 25 papers) were included in the review. The PRISMA flow diagram (Figure 8) summarises the number of records retrieved and selected at different stages of identification and screening.

FIGURE 8.

The PRISMA flow diagram for the identification and selection of clinical trials relating to DIs for GAD. Reproduced with permission from Saramago et al. 124 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: https://creativecommons.org/licenses/by/4.0/. The figure includes minor additions and formatting changes to the original figure.

Reproduced with permission from Saramago et al. 124 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: https://creativecommons.org/licenses/by/4.0/. The figure includes minor additions and formatting changes to the original figure.

At the stage of full-text screening, we excluded 18 records because they were reviews, 14 that were abstract only and 116 that were protocols, and 10 duplicates that had not been picked up in the previous screening. We also excluded six studies that did not recruit GAD populations and 88 studies that recruited mixed populations, which may have included GAD among other conditions (e.g. depression) but did not report separate outcomes for the GAD sample. We excluded a further 61 studies in which the participants did not meet GAD criteria, 10 studies because the intervention was not digital (software based), 11 studies that reported no anxiety outcomes and 18 studies that were not RCTs. Report Supplementary Material 5 provides a full reference list of the excluded studies grouped according to reasons for exclusion.

Sample characteristics in randomised controlled trials of digital interventions for generalised anxiety disorder

The 21 RCTs included in the review,4,5,37,38,40,41,158–172 as detailed in Table 11, were conducted over 10 years, between 2009 and 2019, in 10 countries [Sweden, Australia, USA, UK, Canada, Spain, Italy, Ireland, Taiwan (Province of China) and the Netherlands] and involved 2547 randomised participants. Most participants were recruited from the general adult population, except in four studies161,163,168,172 that recruited students/young adults and one study164 involving the over-60s. GAD populations were defined as either meeting the criteria of an established diagnostic tool, such as the Mini-International Neuropsychiatric Interview (MINI),177 or scoring above an accepted cut-off value for diagnosable GAD in standardised questionnaires, such as the GAD-7 questionnaire178 or the Penn State Worry Questionnaire (PSWQ). 179

Selection of studies and outcomes for the network meta-analyses

A total of 45 different outcome measures were reported in the included RCTs, as shown in Appendix 2. All 21 RCTs4,5,37,38,40,41,158–172 used either the GAD-7 (14 studies)4,5,37,41,159,160,162–165,167–169,171 or the PSWQ (14 studies),4,38,40,41,158,161,162,166–172 with seven studies4,41,162,167–169,171 using both to measure symptoms at baseline and outcomes at follow-up. Table 11 shows which studies reported GAD-7 and/or PSWQ scores. Apart from GAD-7 and PSWQ, the two most frequently reported outcomes were for depression, the Patient Health Questionnaire – 9 items (PHQ-9)180 and the Beck Depression Inventory, version 2 (BDI-II),181 which were reported in eight and six RCTs, respectively (see Appendix 2, Table 20). We focused on GAD-7 and PSWQ as our outcomes of choice for the NMAs because other commonly used GAD outcomes, such as the Beck Anxiety Inventory (BAI) and State–Trait Anxiety Inventory (STAI), were used in only five RCTs. An additional 25 outcome measures, including the Hamilton Anxiety Rating Scale (HAM-A)182 used in a recent NMA on medication for GAD,129 appeared only once in the included RCTs (see Appendix 2, Table 21).

Our first NMA was based on the GAD-7, a seven-item anxiety scale described in the literature as a valid and efficient tool to screen for GAD and assess symptom severity in clinical practice and research. 178 Our second NMA was based on the PSWQ, a measure that focuses on worry, which is one of the central features of GAD. 183 The measure is designed to capture the generality, excessiveness, and uncontrollability dimensions of pathological worry.

Our NMA for GAD-7 included 13 studies. 4,5,37,41,159,160,162,164,165,167–169,171 One study163 used GAD-7 but it was not included in the meta-analysis because it reported only categorical outcomes (i.e. mild, moderate, severe) rather than continuous scores. Another study164 reported the Penn State Worry Questionnaire – abbreviated (PSWQ-A), the abbreviated version of the PSWQ, so it was included only in the GAD-7 model. The measurement period ranged from 3 to 12 weeks because longer follow-ups were available for only a very few studies (e.g. for GAD-7, only four studies4,35,159,160 reported outcomes at 6 and/or 12 months, and one study160 reported outcomes at 24 months). Even when longer follow-ups were available, the control group had already crossed over to the intervention, so the randomisation was lost.

Risk-of-bias assessment

All but one161 of the 21 included studies were judged to have a high risk of bias in at least one domain of assessment for at least one outcome measure. This was largely because of outcome measurement, as all studies used self-reported (albeit standardised) questionnaires. The reason why self-reported outcomes are considered to have a high risk of bias in these studies is that participants are not masked to their allocation group, namely whether they have been allocated to a psychological intervention or to a WL or medication or a psychological placebo that is self-evidently non-therapeutic (e.g. a website with advice on general health rather than a sophisticated web-based programme specific to GAD).

The one study using self-reported measures that was assessed to be at low risk of bias involved the inclusion of a ‘sham arm’ very similar to the experimental arm,161 which made it more difficult for the participants to guess whether the arm to which they were allocated was an intervention or a control. One other study37 was assessed to be at low risk of bias but only for its researcher-administered diagnostic measure, the MINI. 177 Although researcher-administered outcomes are less open to inter-rater bias because they are completed by one person, or a few researchers who receive standardised training, a high risk of bias for the study remains if the researchers who administer the interviews are not masked to the participant’s allocation.

With regard to risk of bias, most studies were considered to be at a low risk for the randomisation process in general, although over half did not provide enough detail of allocation concealment to make an assessment. Similarly, most studies were rated as being at low risk of bias as a result of deviations from the intended intervention. Missing outcome data were of considerable concern with respect to bias, with over half the studies being rated as having a high risk of bias in this domain. Concerns about missing outcome data included high rates of withdrawals or exclusions at follow-up, differential attrition between groups and limited use of appropriate statistical procedures to mitigate these issues. The risk-of-bias outcome for each study under each domain is shown in Table 12 and a visual description of the risk-of-bias assessment across all studies is shown in Figure 9.

FIGURE 9.

Proportion (%) of RCTs of DIs for GAD with ‘low risk’, ‘high risk’ or ‘some concerns’ in the six domains of the risk-of-bias tool. Reproduced with permission from Saramago et al. 124 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: https://creativecommons.org/licenses/by/4.0/. The figure includes minor additions and formatting changes to the original figure.

Reproduced with permission from Saramago et al. 124 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: https://creativecommons.org/licenses/by/4.0/. The figure includes minor additions and formatting changes to the original figure.

Classification of digital interventions and comparators

We classified DIs and their alternatives according to three criteria: (1) whether they were a psychological/behavioural intervention (I) or a non-therapeutic psychological/behavioural control (C); (2) whether they were digital (D) or non-digital (NoD); and (3) whether they were supported (S) or unsupported (U). WLs and usual care were classified under NI unless an active component (e.g. monitoring, sham activity) was introduced, in which case the WL/usual care was classified as non-therapeutic psychological/behavioural control. An additional classification group was included for pharmacological interventions, called medication (M).

The interventions and controls of the 21 included RCTs were allocated to one of the following eight classification groups: medication (M), NI, SDC, SDI, SNoDC, SNoDI, UDC and UDI. There were no available clinical studies that included UNoDIs or UNoDCs.

Table 13 describes all the interventions and controls included in each classification group for each study. The majority of DIs were supported (in 19 RCTs)4,37,38,40,41,158–162,164–172 and were compared against NI (in 12 RCTs). 5,40,41,158,164,166–172 Only three RCTs4,5,160 included UDIs; two were web-based CBT4,160 and one5 was a mobile game to practise breathing retraining. The only NoDI represented in two RCTs165,172 was group therapy [one CBT and one mindfulness-based intervention (MBI) and there was only one RCT that included medication (an antidepressant, sertraline)]. 159 Most of the non-therapeutic active controls reported in eight RCTs4,5,37,38,159,161–163 included a digital element, whereas only one RCT38 had a non-digital control in the form of a weekly face-to-face assessment with a research assistant in a laboratory.

Network meta-analysis results: Generalised Anxiety Disorder-7 scores at follow-up

Generalised Anxiety Disorder-7 scores were reported in 13 out of 20 studies (n = 1613 patients). 4,5,37,38,40,41,158–162,164–172 Each of these studies reported short-term outcomes (up to 12 weeks), with very few studies reporting outcomes beyond 12 weeks post treatment initiation. In those that did, crossovers were allowed at follow-up, biasing any long-term treatment effect. 184 Ten direct treatment comparisons were made in the 13 trials included in the GAD-7-based NMA; four of the 13 trials were multiarm trials (three three-arm trials159,167,169 and one five-arm trial4); five comparisons were informed by more than one trial when a pairwise ANCOVA meta-analysis was conducted (ANCOVA FE and ANCOVA RE models when n > 3).

We constructed a network plot to illustrate which interventions had been compared head to head (direct pairwise comparisons) for GAD-7 in the 13 included RCTs. An overview of these pairwise comparisons is shown in Appendix 3, Table 22. The structure of the network for GAD-7 is shown in Figure 10.

FIGURE 10.

Network plot for direct pairwise comparisons between all interventions and controls for GAD populations in RCTs with GAD-7 scores as an outcome. Line thickness around the node is proportional to the number of patients contributing to each intervention/control group. Line thickness connecting nodes is proportional to the number of patients contributing to each pairwise comparison between interventions/controls. n = number of trials informing each comparison. M, medication. Reproduced with permission from Saramago et al. 124 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: https://creativecommons.org/licenses/by/4.0/. The figure includes minor additions and formatting changes to the original figure.

Reproduced with permission from Saramago et al. 124 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: https://creativecommons.org/licenses/by/4.0/. The figure includes minor additions and formatting changes to the original figure.

Fixed-effects and RE models were employed with minimal difference in mean residual deviances and DIC between the models tested. However, posterior estimates of between-study heterogeneity suggested considerable variability across studies, which was in line with the assessment made of the studies within the evidence base. Hence, a RE approach was preferred. There was a high degree of uncertainty in the network, especially links that were not informed by direct comparisons. Table 14 presents the full results of the NMA based on GAD-7 scores. Negative values suggest a positive effect for the first intervention in the direct pairwise comparisons, and that the intervention on the left column is ‘better’ in the NMA. GAD-7 score differences should be assessed in the light of the GAD-7 score range 0–21, from mild to severe.

Medication was the intervention associated with the largest decrease in GAD-7 median scores, although uncertainty was high in the NMA estimates, with all 95% CrIs including zero. These results were driven by the outcomes of a small (n = 21), three-arm trial159 that compared medication supplemented with scheduled face-to-face meetings with psychologists and general practitioners (GPs) with SDC (a general health website with scheduled meetings with psychologists and GPs) and SDI (a web-based CBT self-help programme with scheduled meetings with psychologists and GPs).

Based on SUCRA rankings and rankograms for each intervention (as detailed in Appendix 4), SDIs were estimated to be more effective than UDIs, which included unsupported web-based CBT4,160 and an unsupported mobile breathing retraining game;5 however, SDIs were less effective than SNoDI, which was a weekly group MBI with a therapist. 165 The adjustment for baseline scores indicated that the baseline effect on the final outcome was small with a 95% CrI including zero (median –0.14, 95% CrI –1.10 to 0.82).

The results of independently pooling direct evidence for each contrast (but not pooling when n = 1), were found to be generally consistent with the NMA results in terms of both direction and magnitude of the estimates (see Table 14; upper-right triangle, shaded). Of note are the differences in the estimates found when applying a FE and a RE ANCOVA meta-analysis model to direct evidence for the comparisons of SDIs and SDCs (n = 4) and of SDIs and NI (n = 8), evidencing non-negligible variability across the studies and the importance of accounting for between-study heterogeneity.

Network meta-analysis results: Penn State Worry Questionnaire scores at follow-up

The PSWQ follow-up scores were reported in 14 out of 20 studies4,31,40,41,158,161,162,166–172 (n = 1776 patients). Using these studies, we constructed a network plot to illustrate which interventions have been compared head to head (direct pairwise comparisons) for PSWQ score. An overview of these pairwise comparisons is shown in Appendix 3, Table 23. The structure of the network is shown in Figure 11.

FIGURE 11.

Network plot for direct pairwise comparisons between all interventions and controls for GAD populations in RCTs with PSWQ scores as an outcome (n = number of comparisons available across all RCTs). Line thickness around the node: proportional to the number of patients contributing to each intervention/control group. Line thickness connecting nodes: proportional to the number of patients contributing to each pairwise comparison between interventions/controls. n = number of trials informing each comparison. Reproduced with permission from Saramago et al. 124 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: https://creativecommons.org/licenses/by/4.0/. The figure includes minor additions and formatting changes to the original figure.

Reproduced with permission from Saramago et al. 124 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: https://creativecommons.org/licenses/by/4.0/. The figure includes minor additions and formatting changes to the original figure.

Eleven direct treatment comparisons were made in the 14 trials4,31,40,41,158,161,162,166–172 included in the NMA; 6 of the 14 trials were multiarm trials (five three-arm trials38,158,167,169,172 and one five-arm trial),4 five comparisons were informed by more than one trial in which pairwise ANCOVA meta-analysis was conducted (ANCOVA FE models and ANCOVA RE when n > 3).

Fixed-effects and RE models were employed with minimal difference in mean residual deviances and DIC between them. However, posterior estimates of between-study heterogeneity suggested considerable variability across studies, which was in line with the assessment made of the studies within the evidence base. Hence, a RE approach was preferred. Table 15 presents the full results of the NMA based on PSWQ scores. Negative values suggest a positive effect for the first intervention in the direct pairwise comparison, and that the intervention on the left column achieves better outcomes in the NMA. The PSWQ score differences should be assessed in the light of the PSWQ score ranging from 16 to 80, that is from mild to severe.

Uncertainty was high for all comparisons, with all 95% CrIs including zero. We observed no difference in median PSWQ scores between the SNoDI, which was a weekly group CBT with a therapist,172 and the UDI, an internet CBT self-help programme without therapist support. 4 Based on SUCRA rankings and rankograms for each intervention, as detailed in Appendix 4, SNoDI and UDI were estimated to be more effective than the remaining interventions and controls, including SDIs; however, SDIs were associated with larger median score reduction on the PSWQ than NI and UDC, which was a general health website without any personal communication. 4

Unexpectedly, SNoDC, a WL control supplemented by weekly face-to-face assessment with research assistant,38 was ‘worse’ than all other comparators, with similar effectiveness to NI. One explanation may be that repeated assessment without any supportive elements during interactions with the researcher may have accentuated participants’ awareness of their anxiety symptoms.

Adjustment for baseline in the PSWQ indicated that baseline effect on final outcome is negligible, with a 95% CrI including zero (median 0.01, 95% CrI –0.44 to 0.45). The results of independently pooling direct evidence on the PSWQ for each contrast (but not pooling when n = 1) were generally consistent with the NMA results (see Table 15; upper-right triangle, shaded).

Results of between-study heterogeneity and inconsistencies

Three sources of heterogeneity were considered relevant: disease severity, concomitant medication and comorbidities. Using data relating to disease severity and comorbidities was not feasible (see Appendix 1 for further details); therefore, only data on concomitant medication were included as a covariate in the synthesis modelling. For both outcomes, when this covariate was included, the between-study heterogeneity parameter was not reduced, suggesting that heterogeneity is not explained by this covariate. Crucially, even if the proportion receiving concomitant medication was found to be an important effect modifier, the meta-regression model is not necessarily suited to detect this intervention–covariate interaction, as patients were receiving medication before trial entry. Therefore, medication may have already exerted an effect on patients, being captured by the ANCOVA baseline adjustment component.

Several data loops existed in the networks for GAD-7 and PSWQ, where both direct and indirect data informed intervention effectiveness estimates; the possibility of inconsistencies was investigated. Tables 14 and 15 show no evidence of substantial discrepancies between the direct and the NMA results for both outcomes; given the uncertainty in the data, only very large differences were likely to result in statistical significance. The results of the consistency and inconsistency models for both outcomes indicated the existence of overall model consistency, as detailed in Appendix 5.

Sensitivity analysis results

The sensitivity of networks to specific studies was investigated. In total, 10 analyses with 12 (rather than the total 13) included studies for GAD-7, and 11 analyses with 13 (rather than the total 14) included studies for PSWQ were performed, and the probability of each intervention being the best was assessed. For GAD-7, the selective serotonin reuptake inhibitor (SSRI) and group CBT continued to have the highest chances of being ‘best’, with probabilities of around 43% and 30%, respectively. Similar results to the primary analysis were obtained for PSWQ. The exception was when a three-arm study comparing SDIs with NI was removed,158 which resulted in increased uncertainty around PSWQ scores for all active interventions.

Two studies159,167 in the evidence base informing the network for the GAD-7 outcome had included fewer than 30 patients. Not considering these studies from the network implied the exclusion of medication from the comparator set, altering the network structure. As expected, with the reduction in the number of studies informing the network, the uncertainty in the posterior effect distributions increased further. However, no significant changes were observed compared with the main model results. For PSWQ, two studies161,167 were also excluded from the network on the basis of lower patient numbers, with the comparator set prevailing. The ranking of active interventions in terms of median PSWQ score decrease compared with NI was unaltered, although higher score decreases were estimated.

Comparison of network meta-analysis results on Generalised Anxiety Disorder-7 and Penn State Worry Questionnaire for digital interventions

To compare the results of the NMAs across both outcome measures (i.e. GAD-7 and PSWQ), and only for comparisons between DIs and their alternatives (rather than between alternatives, e.g. medication vs. group therapy), we present an abridged version of the NMA results in Table 16. The table shows the network comparisons and the direct pairwise comparisons between DIs and alternatives for post treatment (3–12 weeks) scores on GAD-7 and PSWQ median scores (adjusted for baseline).

Comparing the direction and magnitude of differences in median scores at follow-up between the GAD-7 and PSWQ results (where available for both) in Table 16, we make three observations. First, the difference in GAD-7 median scores at follow-up between medication and DIs is the largest across all comparators, and favours medication. Second, no data were available for comparisons between DIs and individual therapy, either face to face or by telephone, or between DIs and manualised guided self-help (which is the non-digital counterpart of most DIs). Third, the direction of effect favoured SDIs for GAD-7 and UDIs for PSWQ.

To compare the ranking of DIs relative to their comparators across both GAD-7 and PSWQ we produced two figures. Figure 12 shows the ranking of DIs (SDI and UDI) for GAD relative to NoDIs and controls based on their likelihood of being ‘best’ for outcomes up to 12 weeks’ follow-up (adjusted for baseline). Figure 12 shows the average ranking of each intervention based on SUCRAs. Appendix 4 shows in detail the SUCRA graphs and rankograms for each intervention and control for GAD-7 and PSWQ.

FIGURE 12.

Ranking of DIs (SDI and UDI) for GAD relative to NoDIs and controls based on their likelihood of being ‘best’ for outcomes up to 12 weeks’ follow-up (adjusted for baseline). Dotted line, top rank; bold line, middle rank; and dashed line, bottom rank. M, medication. Reproduced with permission from Saramago et al. 124 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: https://creativecommons.org/licenses/by/4.0/. The figure includes minor additions and formatting changes to the original figure.

Reproduced with permission from Saramago et al. 124 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: https://creativecommons.org/licenses/by/4.0/. The figure includes minor additions and formatting changes to the original figure.

From Figure 13 we make three observations. First, medication and group therapy have a higher probability than DIs of being ‘best’ (i.e. they are associated with lower GAD-7 and PSWQ scores at 3–12 weeks’ follow-up). Second, UDIs are more likely than SDIs to be ‘best’. Third, several non-therapeutic controls (SDC, UDC and SNoDC) rank above SDIs for both GAD-7 and PSWQ.

FIGURE 13.

Ranking of DIs (SDI and UDI) for GAD relative to NoDIs and controls based on SUCRAs for outcomes up to 12 weeks’ follow-up (adjusted for baseline). Dotted line, top rank; bold line, upper-middle rank; double line, lower-middle rank. M, medication. Reproduced with permission from Saramago et al. 124 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: https://creativecommons.org/licenses/by/4.0/. The figure includes minor additions and formatting changes to the original figure.

Reproduced with permission from Saramago et al. 124 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: https://creativecommons.org/licenses/by/4.0/. The figure includes minor additions and formatting changes to the original figure.

Summary and interpretation

Our systematic review retrieved 21 studies4,5,37,38,40,41,158–172 and our analysis was based on 20 RCTs,4,5,37,38,40,41,158–162,164–172 which included 2325 adults with emerging or diagnosable GAD allocated to DIs or an alternative pathway of care, including NI. Comparisons varied depending on whether the intervention or control was digital or non-digital and supported or unsupported by clinicians or laypeople; the majority of comparisons were between SDIs and NI. Using an ANCOVA framework, our two NMAs pooled together post-treatment scores (adjusted for baseline) for two outcome measures: GAD-7 and PSWQ. In addition, the existence of treatment effect modifiers was assessed, several sensitivity analyses were carried out and network consistency was evaluated.

Our NMA results suggest that medication is associated with lower anxiety scores at follow-up relative to all other interventions and controls. Medication also ranks first in terms of its likelihood of being most effective, which considers the uncertainty in relative effect estimates. Medication results are supported by data from one study reporting GAD-7 and not PSWQ. Antidepressant medication as a treatment for GAD is supported by clinical guidelines2 and previous evidence syntheses. A large NMA129 of six trials comparing setraline with placebo for the treatment of GAD found that sertraline (the same antidepressant used in the study by Christensen et al. 159 included in our NMA) resulted in a greater improvement in HAM-A scores compared with baseline than did placebo (mean difference –2.88, 95% CrI –4.17 to –1.59). Another meta-analysis131 favoured a combined treatment of psychological therapy and medication for all anxiety disorders and depression, except GAD, where the direction of effect favoured antidepressant medication (venlafaxine) alone.

Our NMA makes best use of all currently available RCT-based evidence on DIs for GAD. Despite the sparsity and low quality of the data, a statistical synthesis can still be useful for mental health professionals who use DIs for GAD anyway, so that they can be informed about the current evidence base, understand which DIs have been shown to be more effective in reducing GAD and prioritise future research. Previous reviews of DIs that reported GAD-related outcomes131,132 used mixed samples of anxiety disorders and depression without reporting separate outcomes for GAD subgroups. There are no RCTs carried out in GAD populations comparing DIs with non-digital self-help interventions based on a manual rather that a web-based program. Nor are there any RCTs comparing DIs with individual therapy, either face to face or by telephone, as the only available comparisons within the literature are between DIs and group therapy.

Owing to very wide confidence intervals, our NMA results were inconclusive as to whether DIs for GAD are better than NI and non-therapeutic active controls, or whether they confer an additional benefit to standard therapy. Previous meta-analyses suggested that supported DIs could be as good as face-to-face therapy across depression, anxiety and somatic disorders. 185 Yet the mixed samples in these meta-analyses without separate analysis or reporting for GAD subsamples do not allow any conclusions to be drawn about the relative efficacy of DIs specific to the treatment and prevention of GAD.

Our NMA results about the comparative effectiveness of supported and unsupported DIs for GAD were counterintuitive, as we would expect SDIs to rank higher in terms of likelihood of being ‘best’, based on a previous meta-analysis in which SDIs were found to be four times more effective than DIs without any therapist contact. 186 We found that UDIs rank higher than SDIs in terms of being best, but the converse is true based on all rankings (SUCRAs). This is consistent with a recent review187 that reported mixed findings regarding guided versus unguided DIs and human versus automated support for DIs. This suggests that the design, content, technology platform and type of reinforcement offered in lieu of personal support in UDIs may bear more weight and account for greater variability in their outcomes.

The evidence base available in this setting is complex. In particular, the sheer volume of anxiety metrics (45 in total) being reported across the available studies suggests a lack of consensus on which measures to use in evaluating GAD outcomes. GAD-7 and PSWQ are commonly used and well-validated outcomes to capture change in GAD symptoms. The use of GAD-7 or PSWQ or both as outcome measures across all the included studies further strengthens our review, as we did not need to exclude any of the retrieved studies from the meta-analysis on the basis that it did not use GAD-7 or PSWQ. The only excluded study163 used GAD-7, but reported only categorical outcomes. It is worth noting that the HAM-A used in the large meta-analysis on medication for GAD129 featured in only one167 of our included studies, showing a discrepancy in the choice of outcomes for psychological and pharmacological interventions.

Limitations

There was substantial uncertainty around effect estimates of DIs against alternatives for GAD-7 and PSWQ. This was driven by the small number of studies informing most comparisons, as well as the small samples in some of these studies and their high risk of bias, limiting our confidence in any observed differences in anxiety scores between interventions and controls. These observed differences may simply be due to chance; therefore, we cannot make clear recommendations about the relative value of DIs compared with their comparators, but we can make recommendations for future research (both for primary studies and evidence synthesis) in view of the current evidence.

Caution is needed when interpreting the results of our NMA across all different interventions for GAD. Our review has been completed in the context of DIs; it included only RCTs in which at least one of the randomisation arms was a DI. Therefore, we cannot draw any conclusions about the comparative merit of NoDIs (psychological or pharmacological) for GAD when these are considered separately from DIs (e.g. group CBT vs. medication). To be able to do this, we would need to include RCTs in a NMA that would enable second or third order contrasts (e.g. RCTs comparing NI and medication), which was beyond the scope of this review. In addition, ranking based on likelihood of being best and on SUCRAs does not reflect differences in effectiveness estimates between interventions and controls and CrIs, that is, we cannot tell whether or not the differences between ranking position (e.g. between first, second, third and fourth) are clinically meaningful.

Another point of caution, as it is with all evidence synthesis, relates to pooling DIs and their alternatives into groups for analysis based on our classification criteria. Any classification implies interpretation and judgement, which is conditional on attained information from studies. We note the insufficient reporting of details about ‘non-therapeutic controls’ and WLs in some studies. Furthermore, we could have split DIs into further categories according to the technology used (e.g. VR, internet, mobile app), the function of the technology (e.g. adjunct to clinician-delivered therapy vs. patient self-help) or the type of support (e.g. telephone calls vs. meetings). This would have created more ‘nodes’ in the NMA models but also more uncertainty because comparisons between DIs and their alternatives within each subgroup would have been informed by fewer studies.

Many of our included RCTs had small samples and multiple arms, comparing different versions of the same intervention, thereby reducing the power of the study. Our evidence synthesis also shows that in the majority of RCTs either the time frame for follow-up is short (up to 12 weeks) or the control group has already crossed over to the intervention at the point of a longer follow-up (up to 2 years), which cancels the randomised comparison. Consequently, we did not include observations for further follow-up time points, when these were available, nor did we account for time differences in the short-term reporting (the timing of assessments post treatment varied from 3 to 12 weeks). Our NMA results reflect the short-term impact of DIs as there is scant evidence to inform randomised comparisons about effects beyond 12 weeks.

Recommendations

As GAD is the most prevalent and least studied condition among other common mental health problems, future evidence syntheses will be helpful to focus on GAD populations and GAD subgroups within mixed populations as a means of informing GAD-specific future research and clinical guidelines. 2 Feasibility and pilot studies, as well as user involvement in the development of the intervention and delivery protocols, could ensure that the final RCT tests the best possible intervention for GAD. Adaptive designs with improved intervention features and boosted recruitment numbers to a fully powered RCT are preferable to underpowered studies with multiple arms testing increments of the same DI.

Our NMAs and previous literature suggest that antidepressants are an important factor to consider in future studies on DIs for GAD. Studies of psychological interventions (whether digital or non-digital) often include participants who are taking medication as part of their routine care. It is difficult to disentangle the effects of medication and psychological support for GAD, and future RCTs need to report medication details (name, dose and duration) and include it as a covariate in their analysis to establish how outcomes with DIs and controls are influenced by concurrent medication use.

Having a consensus about GAD-specific outcome measures can prevent participant fatigue from completing batteries of different questionnaires and enable comparisons across studies and data syntheses. GAD-7 is more sensitive to changes associated with treatment and therefore may be more suitable for longitudinal clinical research. 188 Reporting continuous data on GAD-7 as a common measure in RCTs with GAD populations will make more studies available for a future statistical synthesis. Including HAM-A in studies of psychological therapies will enable us to compare results with those of pharmacological studies. Future analyses using multivariate models may be in a position to make better use of the available evidence by borrowing strength across different outcomes.

Many studies that include long-term follow up of participants offer the intervention to those randomised to the control group at a crossover point. Participants are also likely to receive some treatment as part of usual care the longer they remain on WLs or non-therapeutic controls, so studies cannot withhold interventions to enable long-term follow-up. As the typical duration of DIs is 3–12 weeks, the follow-up period of future RCTs needs to be longer than this, for example at least 6 months, to get a sense of the ‘stickiness’ of DIs beyond their delivery period. Usual care and WLs are poorly reported in RCTs, and the RCTs do not include data on concurrent interventions accessed by participants, including openly available self-help, which can influence the difference in outcomes between DIs and NIs.

Classification of digital interventions and their alternatives

We compared seven types of interventions and controls for GAD based on our classification criteria: medication (M), SNoDI, SDI, UDI, SDC, UDC and NI. There were no available clinical studies that included UNoDIs or UNoDCs, and no studies of SNoDCs that used GAD-7 as an outcome. DCs identified in WP2 (see Classification of digital interventions and comparators) were included in the analysis, as patients are occasionally signposted to information sources in reputable websites when waiting times are long. Their exclusion from the analysis was explored in Scenario analysis.

Model structure

The CEA methods followed the NICE reference case. 11 An NHS and Personal Social Services perspective was used for the analysis. A full incremental analysis was undertaken comparing all seven interventions/controls simultaneously over a patient’s lifetime.

The model structure was based on GAD severity determined by the GAD-7 questionnaire, in which the scores denote no (scores 0–4), mild (5–9), moderate (10–14) or severe (15–21) anxiety. 178 A Markov cohort model was used, following the structure adapted from Kumar et al. 85 The health states included are shown in Figure 14.

FIGURE 14.

Decision model for GAD health states based on GAD-7 scores. Reproduced with permission from Jankovic et al. 189 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution-NonCommercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for non-commercial use, provided the original work is properly cited. See: https://creativecommons.org/licenses/by-nc/4.0/. The figure includes minor additions and formatting changes to the original figure.

Reproduced with permission from Jankovic et al. 189 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution-NonCommercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for non-commercial use, provided the original work is properly cited. See: https://creativecommons.org/licenses/by-nc/4.0/. The figure includes minor additions and formatting changes to the original figure.

At the start of the model, patients are in one of four health states. At each subsequent cycle of the model, they can remain in this health state or transition to another, better or worse, health state. Health-care resource use and HRQoL are driven by GAD severity, both directly and through comorbidities. The intended effect of DIs was to reduce the severity of GAD, thereby reducing the number of patients in moderate and severe GAD states, with an associated impact on costs and HRQoL.

Model parameters

Sensitivity analysis

Value-of-information analysis

The results of the probabilistic uncertainty analysis were used to estimate the VOI. The maximum value of further research per patient treated [expected value of perfect information (EVPI)] was derived from the difference between the NMB under perfect information (i.e. where the optimum treatment is known with certainty) and existing information (i.e. the expected net benefit from the treatment that is the most cost-effective under current knowledge). 202

The value of further research at population level [expected value of perfect information at the population level (EVPI_P)] was then estimated by multiplying EVPI by the effective population size (Equation 1):202

where I_t represents GAD incidence in year t, T is the total number of years for which information from the research would be useful (usually representing the technology lifetime) and dr is the discount rate, set at 3.5%.

The incidence of GAD was 4.9% of the population in the UK in 2008203 and 4.3% in Germany in 2008. 204 Therefore, in the VOI analysis, the incidence was set at 250,000, representing a conservative estimate in which < 10% of patients in England (population of ≈ 55.9 million) who acquire GAD would receive the intervention. The relevant technology lifetime was assumed to be 5 years.

Finally, the expected value of partial perfect information at the population level (EVPPI_P) was derived to understand the parameters in the model driving uncertainty relevant to the adoption decision. The non-parametric method developed by Strong et al. 205 was used. The method, conducted using the Sheffield Accelerated Value of Information (SAVI) interface (URL: http://savi.shef.ac.uk/SAVI/; accessed 7 September 2021), regresses random samples of all uncertain parameters against the net benefit of each intervention to derive the impact of uncertainty about individual parameters on the model results. All uncertain parameters in the model were used to derive EVPPI_P for the following five groups of parameters: treatment effect (GAD-7 scores after the first cycle, seven parameters), state-specific costs (four parameters), state-specific utilities (four parameters), excess death [three parameters – relative risk (RR) of death in mild, moderate and severe anxiety] and age-related utility decrements (54 parameters, utility decrements every year until all patients in the model die).

The opportunity cost was assumed to be £15,000 per QALY, in line with the empirical estimate by Claxton et al. 206

Generalised anxiety disorder score trajectory

Figure 15 shows the proportion of patients in each GAD-7 health state for the initial 5 years, without treatment. At the start of treatment almost 80% of patients had moderate GAD, a further 20% had mild GAD and the proportion with no and severe GAD was < 0.05. In the base case, for the first 3 years of the model, patients’ GAD was assumed to improve spontaneously, resulting in a decrease in the proportion who had moderate and severe GAD, and an increase in the proportion with no or mild GAD. After 3 years the proportions remained constant (in the living population). The proportion in each anxiety state after treatment is shown in Appendix 8.

FIGURE 15.

Movement through Markov states for the initial 5 years without treatment. Reproduced with permission from Jankovic et al. 189 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution-NonCommercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for non-commercial use, provided the original work is properly cited. See: https://creativecommons.org/licenses/by-nc/4.0/. The figure includes minor additions and formatting changes to the original figure.

Reproduced with permission from Jankovic et al. 189 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution-NonCommercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for non-commercial use, provided the original work is properly cited. See: https://creativecommons.org/licenses/by-nc/4.0/. The figure includes minor additions and formatting changes to the original figure.

The reduction in GAD-7 scores after receiving treatment with the seven treatment options compared in the model is shown in Figure 16. The initial reduction in GAD-7 reflects the effect of treatment, as determined by our meta-analysis. The treatment effect was assumed to remain constant indefinitely, and after 1 year GAD-7 scores decreased further as patients’ symptoms continued to improve at the same rate as without treatment. However, the GAD-7 reduction decreased, as, following treatment, the number of patients who could recover spontaneously was lower than if no treatment had been received. The rate of decrease in GAD-7 score varied between interventions, depending on the magnitude and uncertainty of the treatment effect. More effective treatments were associated with fewer patients in mild, moderate and severe anxiety states, and so the spontaneous symptoms improvement affected fewer patients. Wider confidence intervals led to a higher probability of patients being in the no anxiety state, and so the spontaneous symptoms improvement affected fewer patients; this is why the reduction in GAD-7 scores after 1 year was higher in SDIs than in UDIs, despite SDIs being more effective. After 3 years, the reduction in GAD-7 remained constant.

FIGURE 16.

Change in mean GAD-7 score for the initial 5-year period after starting treatment. M, medication. Reproduced with permission from Jankovic et al. 189 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution-NonCommercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for non-commercial use, provided the original work is properly cited. See: https://creativecommons.org/licenses/by-nc/4.0/. The figure includes minor additions and formatting changes to the original figure.

Reproduced with permission from Jankovic et al. 189 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution-NonCommercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for non-commercial use, provided the original work is properly cited. See: https://creativecommons.org/licenses/by-nc/4.0/. The figure includes minor additions and formatting changes to the original figure.

Cost and outcomes of digital interventions compared with alternatives

Table 17 shows the costs and outcomes associated with each of the seven comparators. The differences in QALY and life-years follow the same pattern; the differences between comparators were small and uncertain, reflected in wide and overlapping confidence intervals. The QALY and life-year gains reflect the results from the meta-analysis: on average, the greatest reduction in post-treatment anxiety scores was associated with medication, followed by SNoDIs, then by SDIs and then by UDIs. Both SDIs and UDIs were associated with lower scores post treatment than SDCs, UDCs and no treatment.

Health-care costs were highly uncertain, with overlapping confidence intervals. Differences in health-care costs largely followed the reverse order of QALY gains; health-care costs were lowest for patients taking medication and highest for patients who received NI. This was due to the correlation between health-care costs and GAD severity (see Appendix 7).

Supported digital interventions and SNoDIs were associated with the same intervention costs, as they include the same level of human resources. Medication was more expensive, as it is administered over 5 years (and so requires contact with health-care professionals for 5 years), and, unlike other interventions, the costs were uncertain because the proportion of patients who were alive and taking medication was uncertain. The total costs of different comparators follow the same order as health-care costs, except SDIs that incur a higher total cost than UDIs owing to the higher intervention cost.

Table 17 shows the incremental costs and effects and the net benefit of all seven comparators, whereas Figure 17 shows the cost-effectiveness acceptability curves for each intervention and control, and how their probability of cost-effectiveness changes for different opportunity costs. In Table 17, net benefit and the probability of cost-effectiveness are shown for two opportunity costs: £0 per QALY, representing a decision-maker who will implement only cost-saving interventions, and £15,000 per QALY, close to the empirical estimate of the opportunity cost in England. 9 The model results did not change significantly at opportunity costs higher than £15,000 per QALY (as seen in Figure 17). The intervention most likely to be cost-effective depends on the opportunity cost. The most cost-effective intervention is NI for an opportunity cost of £0 per QALY, SNoDIs for an opportunity cost of £1000 per QALY and medication for an opportunity cost of £2000 per QALY or higher.

FIGURE 17.

Cost-effectiveness acceptability curves for each intervention and control. M, medication. Reproduced with permission from Jankovic et al. 189 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution-NonCommercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for non-commercial use, provided the original work is properly cited. See: https://creativecommons.org/licenses/by-nc/4.0/. The figure includes minor additions and formatting changes to the original figure.

Reproduced with permission from Jankovic et al. 189 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution-NonCommercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for non-commercial use, provided the original work is properly cited. See: https://creativecommons.org/licenses/by-nc/4.0/. The figure includes minor additions and formatting changes to the original figure.

Medication was dominant, resulting in the highest NMB at all opportunity costs, followed by group therapy, as both led to lower total costs and greater QALY gains than DIs. NI had the lowest NMB because of the lower QALY gains and higher health-care costs. DIs (UDIs and SDIs) had a higher NMB than DCs (UDC and SDC) as they were associated with better outcomes and lower health-care costs. SDIs had greater QALY gains but also higher costs than UDIs, so their NMB depended on the opportunity cost of the health system. When the opportunity cost is £0 per QALY, UDIs are ranked above SDIs, whereas the opposite is the case when the opportunity cost increases to £15,000 per QALY.

These results are uncertain, which is reflected in the wide confidence intervals associated with NMB and the probability of each intervention being the most cost-effective (see Table 17 and Figure 18). For example, although medication has the highest NMB, there is a high probability that it is not the most cost-effective comparator: 0.949 probability at £0 per QALY opportunity cost and 0.658 at £15,000 per QALY.

FIGURE 18.

Cost-effectiveness ranking based on NMB for DIs and controls. Values in brackets are 95% CIs. k, opportunity cost; M, medication.

Accumulation of costs and outcomes over time is shown in Appendix 10. The longer the analysis time horizon, the greater the differences in the QALY gain (see Appendix 10, Figure 29), as the benefits accrue over time. Differences in health-care cost are driven by the clinical effectiveness of interventions, and so, as for QALY gains, the longer the time horizon the greater the cost differences (see Appendix 10, Figure 30), but the rate of change is diminishing. Medication is the most expensive intervention in the short term, but, as health-care cost savings accrue over time, and treatment cost reduces after 5 years, the total cost for medication increases at a lower rate than for other comparators, eventually becoming the second-cheapest treatment option, after SNoDI.

Value-of-information analysis

Figure 19 shows EVPI_P over a range of opportunity costs. Even at its lowest, when opportunity cost is £4000 per QALY, the value of uncertainty is high (£11.4B), increasing to £16.2B and £19B when opportunity costs are £15,000 and £20,000 per QALY, respectively. The EVPPI_P analysis suggested that uncertainty in the treatment effect had the greatest value, £12.9B. Parameters defining the effect of GAD on costs and HRQoL (state-related costs and utilities, and excess mortality) had low or negligible value, whereas age-related utility decrements were uncertain, valued at £6.8B.

FIGURE 19.

The EVPI_P for a range of opportunity costs. Reproduced with permission from Jankovic et al. 189 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution-NonCommercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for non-commercial use, provided the original work is properly cited. See: https://creativecommons.org/licenses/by-nc/4.0/. The figure includes minor additions and formatting changes to the original figure.

Reproduced with permission from Jankovic et al. 189 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution-NonCommercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for non-commercial use, provided the original work is properly cited. See: https://creativecommons.org/licenses/by-nc/4.0/. The figure includes minor additions and formatting changes to the original figure.

Scenario analysis

Scenario analyses were performed to explore the sensitivity of the findings to assumptions made regarding the GAD score trajectory, health-care costs and interventions costs. Appendix 11 shows the movement through Markov states, and the changes in GAD-7 scores in the initial 10-year period is shown in Appendix 11, Figures 31–33. The results were not sensitive to any of the alternative scenarios; the order of cost-effectiveness did not change, only the magnitude of the difference. Pharmacotherapy dominates all other interventions, and SDIs are more effective and costlier than UDIs (see Appendix 11). As DIs were costlier than face-to-face therapy and medication, the effect of increasing their cost further was not explored.