Methods to identify postnatal depression in primary care: an integrated evidence synthesis and value of information analysis

Article history

The research reported in this issue of the journal was commissioned by the HTA programme as project number 05/39/06. The contractual start date was in October 2006. The draft report began editorial review in May 2008 and was accepted for publication in January 2009. As the funder, by devising a commissioning brief, the HTA programme specified the research question and study design. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The HTA editors and publisher have tried to ensure the accuracy of the authors’ report and would like to thank the referees for their constructive comments on the draft document. However, they do not accept liability for damages or losses arising from material published in this report.

Declared competing interests of authors

None

Copyright statement

© 2009 Queen’s Printer and Controller of HMSO. This monograph may be freely reproduced for the purposes of private research and study and may be included in professional journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be addressed to: NETSCC, Health Technology Assessment, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK.

2009 Queen’s Printer and Controller of HMSO

Definitions and epidemiology of postnatal depression

For the purposes of this research, PND was defined as a non-psychotic depressive episode meeting standardised diagnostic criteria for a minor or major depressive disorder, beginning in or extending into the postnatal period. 9 Within our research we distinguish postnatal depression from other types of mental health problems that can also occur during pregnancy and the postnatal period. Baby blues and puerperal psychosis are two such examples and are important health problems within their own rights. Baby blues and puerperal psychosis were not addressed specifically within this body of research. From a clinical perspective, PND includes three subgroups of women whose management may differ: (1) those who develop depression only after childbirth; (2) those who have developed antenatal depression, which continues into the postnatal period; and (3) those women with pre-existing chronic or relapsing depression. A meta-analysis of 59 studies (including 12,810 women, mainly from first world countries) found that the prevalence of depression within the first few postnatal months was 13% [95% confidence interval (CI) 12.3% to 13.4%]. 10 Most cases develop within the first 3 postnatal months,11 with a peak incidence at around 4–6 weeks. 9 Although one study11 showed that most cases last around 3 months and resolve spontaneously without treatment, another study12 demonstrated the presence of depression with over 50% of cases lasting over 6 months and some being still present at 4 years.

Clinical and social consequences of postnatal depression

Depression in all populations is associated with profound decrements in quality of life, social functioning and economic productivity. 13 However, in the case of PND, the adverse consequences are felt beyond the individual with depression, affecting the family and development of the infant. In particular, the severity and chronicity of maternal depression are predictive of disturbances in child development. 14 Physical health and risk of childhood injury also seem to be adversely affected as a consequence of PND. 15

Strategies to improve the detection of postnatal depression

Given the adverse consequences of PND and the general underidentification of this problem, a number of strategies have been proposed to improve PND identification. These broadly fall into five categories:

• postnatal identification using specially developed standardised postnatal questionnaires [such as the Edinburgh Postnatal Depression Scale (EPDS)16 and the US Postpartum Depression Screening Scale (PDSS)17

• postnatal identification using standardised generic questionnaires for depression [such as the Beck Depression Inventory (BDI)18]

• prenatal screening using standardised depression questionnaires to identify those with pre-existing depression and those at risk of developing significant depression in the postnatal period19

• prenatal screening using known risk factors for PND (such as previous history of depression and lack of social support) to identify those who are likely to subsequently develop depression in the postnatal period20

• the use of training packages targeted at health-care professionals designed to enhance awareness and recognition of clinical signs of PND and to ensure that a thorough psychosocial assessment is provided on a routine basis. 21

Current policy and practice within the UK

Following publication of authoritative recommendations in a number of national policy documents,22,23 the use of case-finding and screening strategies has accelerated rapidly. For example, the National Service Framework made an explicit requirement that all areas should have local protocols for the management of PND. 23 In practice, screening and case-finding strategies have dominated and have tended to focus upon the routine or ad hoc administration of the EPDS in the postnatal period, such that it has become the most widely used of the above strategies. The application of this measure has tended to fall upon health visitors, who are responsible for monitoring the well-being of the mother and newborn after 14–28 days postnatally. The de facto implementation of a national screening strategy in the UK, based around the EPDS, has attracted substantial discussion. Criticisms have been levelled at this strategy based upon the ethics of mass screening; the psychometric properties of available instruments (especially the EPDS); the acceptability (to both patients and health-care professionals) of screening and case-finding strategies; and the absence of any evidence that screening, per se, leads to effective management and improved mother and infant outcomes. 8

Screening is only one way in which recognition and management of PND might be improved and there are clear criteria laid down that should guide the adoption of a screening strategy as part of national health policy. Screening tests can be justified only if the instrument is accurate, results in a more effective treatment than would otherwise be the case and does so with a favourable ratio of costs to benefits. 24 These criteria have been codified in the UK by the establishment of the National Screening Committee (NSC) and the publication of clear criteria that must be satisfied before adoption of a screening strategy. 25 When these criteria were applied to the case of screening for PND,26 insufficient evidence was found to support the implementation of this strategy. This NSC recommendation has not been without controversy, but genuine concerns remain regarding the acceptability, validly, clinical effectiveness and cost-effectiveness of identification methods for PND.

Several reviews have been undertaken to identify, review and assess the performance of methods to identify PND. 27–29 The most recent of these is a comprehensive review of antenatal and postnatal care that was undertaken by the National Institute for Health and Clinical Excellence (NICE) in 2007. 30 On the basis of this review, clinical guidance on the management of antenatal and postnatal mental health care was issued. The guidance included a review of methods to identify mental health problems during the postnatal period. The review considered two methods of identification, the EPDS and case-finding questions (Whooley plus help question).

A literature search was undertaken to investigate the psychometric properties of the EPDS. Eight validation studies16,31–37 and a recent systematic review27 were identified from the literature; the evidence from the systematic review27 alone was used to establish the diagnostic performance of the EPDS. In the review three studies were retrieved that had used the EPDS to identify major depression during the postnatal period. Pooling of two of these studies was undertaken at a cut point of ≥ 13 and the EPDS was found to have a sensitivity of 0.91 (95% CI 0.84 to 0.99) and a specificity of 0.91 (95% CI 0.88 to 0.94). Based on these figures, together with a prevalence rate of 6.8% (based on a review of prevalence studies performed by the same authors), an overall positive predictive value of 57% and a negative predictive value of 99% was calculated. Furthermore, pooling of three studies was undertaken at a cut point of ≥ 10 to identify women with major or minor depression and the EPDS was found to have a sensitivity of 0.68 (95% CI 0.58 to 0.78); pooling of specificity values was not undertaken because of significant heterogeneity although a value of 0.80 is reported in the discussion. Based on these figures, together with a prevalence rate of 11.3% (based on a review of prevalence studies performed by the same authors), an overall positive predictive value of 30% and a negative predictive value of 95% were calculated.

Evidence of the psychometric properties of case-finding questions came from two studies. 38,39 In the first study38 the potential advantages of two brief questions (termed ‘Whooley questions’) compared with usual measures for identifying depression (depression in general not PND) were explored. The two brief questions were derived from the 2-item Patient Health Questionnaire (PHQ)-9 and were: (1) ‘During the past month, have you often been bothered by feeling down, depressed or hopeless?’ and (2) ‘During the past month, have you often been bothered by little interest or pleasure in doing things?’ The two questions, Center for Epidemiologic Studies Depression Scale (CES-D), Medical Outcomes Study depression measure (MOS), BDI, Symptom-Driven Diagnostic System for Primary Care (SDDS-PC) and Diagnostic Interview Schedule (DIS) were administered to male participants attending a Veterans Affairs Medical Center in San Francisco. A positive response to the two case-finding questions had a sensitivity of 0.96 (95% CI 0.90 to 0.99) and a specificity of 0.57 (95% CI 0.53 to 0.62). Based on these figures, together with a prevalence rate for major depression of 18.1%, an overall positive predictive value of 33% and a negative predictive value of 98% were calculated.

The second study39 used the same questions as the previous study with an additional ‘help’ question: ‘Is this something with which you would like help?’ The three questions (termed ‘Whooley questions plus the help question’) were administered to 936 patients through 19 general practitioners in six clinics in New Zealand and were validated against the standardised psychiatric interview CIDI (Composite International Diagnostic Interview). A positive response to either question plus the ‘help’ question had a sensitivity of 0.96 (95% CI 0.86 to 0.99) and a specificity of 0.89 (95% CI 0.87 to 0.91) when identifying major depression. Based on these figures, together with a prevalence rate for major depression of 5%, an overall positive predictive value of 32% and a negative predictive value of 99.7% were calculated.

The current NICE guidance issued on antenatal and postnatal mental health recommends the use of the Whooley questions plus the additional help question and states:

• At a woman’s first contact with primary care, and at her booking visit and postnatally (usually at 4–6 weeks and 3–4 months), health-care professionals (including midwives, obstetricians, health visitors and GPs) should ask the two questions to identify possible depression:

  1. – ‘During the past month, have you often been bothered by feeling down, depressed or hopeless?’

  2. – ‘During the past month, have you often been bothered by little interest or pleasure in doing things?’

• A third question should be considered if the woman answers ‘yes’ to either of the initial questions:

  1. – ‘Is this something you feel you need or want help with?’

• Health-care professionals may consider the use of self-report measures such as the EPDS, Hospital Anxiety and Depression Scale (HADS) or PHQ-9 as part of subsequent assessment or for the routine monitworing of outcomes.

The Whooley case-finding questions, derived from the Primary Care Evaluation of Mental Disorders (PRIME-MD), are brief and easy to use in routine practice and are currently recommended in the NICE guidance as the identification method of choice for case finding minor and major depression in any type of non-postnatal population. However, no research literature currently exists of studies that have considered, or have validated, case-finding questions in samples of women in the postnatal period. The NICE research recommendations based on the review propose that a validation study should be undertaken of the Whooley questions plus the help question in women in the first postnatal year, examining the effectiveness of the questions when used by midwives and health visitors compared with a psychiatric interview.

Search strategy

Searches were undertaken on the following databases to identify relevant clinical and cost-effectiveness literature:

• MEDLINE (Ovid Online – www.ovid.com/)

• CINAHL (Cumulative Index to Nursing and Allied Health Literature) (Ovid Online – www.ovid.com/)

• PsycINFO (Ovid Online – www.ovid.com/)

• EMBASE (Ovid Online – www.ovid.com/)

• Maternity and Infant Care (Ovid Online – www.ovid.com/)

• CENTRAL and DARE (Database of Abstracts of Reviews of Effects) (Cochrane Library – CD-ROM)

• CDSR (Cochrane Database of Systematic Reviews) (Cochrane Library – CD-ROM)

• Science Citation Index (SSCI) (Web Of Knowledge – http://wos.mimas.ac.uk/)

• NRR (National Research Register) (www.nrr.nhs.uk/)

• ReFeR (Research Findings Register)

• metaRegister of Controlled Trials (mRCT) (via Current Controlled Trials – http://controlled-trials.com/)

• Health Services Research Projects in Progress (HSRProj) (www.nlm.nih.gov/hsrproj/)

• LILACS (http://bases.bireme.br/cgibin/wxislind.exe/iah/online/?IsisScript=iah/iah.xis%26base=LILACS%26lang=I)

• Inside Conferences – accessed via Dialog (file 65) using DialogLink 5

• Dissertation Abstracts – accessed via Dialog (file 35) using DialogLink 5.

In addition, the following databases were searched specifically for cost-effectiveness studies:

• NHS Economic Evaluation Database (NHS EED) (CRD – www.york.ac.uk/inst/crd/crddatabases.htm)

• Health Economic Evaluations Database (HEED) (CD-ROM)

• IDEAS (http://ideas.repec.org/)

• EconLit (ERLWebSPIRS5 – http://arc.uk.ovid.com/).

Forward citation searching

For the validation review (Chapter 5), forward citation searching was undertaken for the original EPDS16 and PDSS44 validation studies. This process was undertaken using the Web of Science (WoS) Institute of Scientific Information (ISI) citation database. Each study was entered separately and all citations to the paper since publication were identified. Titles and, when available, abstracts of the papers that had cited the selected trials were downloaded.

Terminology

The terms for the search strategies were identified through discussion between an information officer and the research team, by scanning the background literature and by browsing the MEDLINE thesaurus (MeSH). All databases were searched from their inception to the date of the search. Searches took place during February 2007 (see Appendix 1 for dates of individual searches). No language or other restrictions were applied. A broader strategy was used on the economic databases to capture primary economic evaluations relating to depression. Full details of the search strategies are reported in Appendix 1.

Inclusion and exclusion criteria

All records were imported into a bibliographic referencing software program (endnote version 9). Duplicate references were identified using the inbuilt function within endnote and subsequently deleted. Two reviewers screened titles and abstracts to identify potentially eligible studies; any disagreements were resolved by consensus or deferred to a third party if necessary. Studies were assessed for inclusion across all phases of the review. Further details regarding the inclusion and exclusion criteria for each individual review can be found in the following chapters. Full papers of potentially eligible studies were obtained and assessed for inclusion independently by two reviewers.

Summary of literature searching

The study selection process is outlined in Figure 1. In total, 11,945 potentially relevant studies were identified from the searches, of which 225 were selected for full assessment. Of these, 108 studies met the inclusion criteria and were included in one or more of the reviews outlined in the following chapters.

FIGURE 1.

Flow diagram of literature searches.

Reference lists of all reviews were inspected to ensure that all potentially relevant studies had been identified. Authors were contacted when studies were published as abstracts or when there was insufficient information to assess eligibility or extract the relevant data. In addition, authors of ongoing and recently completed research projects were contacted to enquire if the research had been completed and if there were any subsequent publications. In total, 32 authors were contacted for further information and 20 authors responded providing further information.

Stakeholder involvement

Postnatal depression is an important health problem and PND identification has proved to be an especially controversial area of practice and policy. Our research sought to engage with, rather than ignore, this area of controversy. Important stakeholders were engaged in helping us frame our research questions and in understanding the results of our evidence synthesis.

Stakeholders can be thought of as groups of individuals who have specific interests and concerns with respect to a particular issue. Stakeholders are not a homogeneous group and there may be important differences between stakeholders in terms of their understanding of the issue and expectations of the actions that should be taken by others (e.g. regulators, policy-makers, professionals, members of the public). Stakeholder engagement requires an explicit analysis of the ‘power’ and ‘stake’ that are inherent in different stakeholder constituencies. A meaningful engagement of stakeholders requires that the following are included: (1) high stake/influence and high power (in this case Department of Health policy-makers; members of the NSC; commissioners of primary care and maternity services; national professional organisations – Community Practitioners and Health Visitors Association, Royal College of Psychiatrists, Royal College of Midwives) and (2) high stake, low influence [in this case frontline primary health-care workers – midwives, GPs, psychiatrists and health visitors – and women (and their partners) with experience of maternity services and PND].

We held a single stakeholder consultation day in November 2007. At the beginning of the consultation day, participants were provided with an overview of the background to PND from an epidemiological perspective, and to the principles of screening, with illustrations of the main screening, research, policy and practice issues for PND. Subsequently, stakeholders were asked to participate in the first focus group, which examined stakeholders:

• general perceptions of PND and screening

• awareness of methods to identify PND

• methods used in practice

• awareness of the recent NICE guidelines

• views about the Whooley + help case finding questions

• views about the EPDS.

The programme of research was then outlined and the emerging results presented. Following the presentations, the second focus group was undertaken, which examined:

• stakeholders views of the emerging research evidence presented

• influence of research evidence on the stakeholders prior beliefs

• importance of validity, acceptability, outcome and costs in defining good measures

• opinions on the predefined themes for the acceptability review

• research priorities.

Stakeholders were involved in the design, refinement, conduct and analysis of this programme of reviews. Health professionals, service users and researchers have contributed to and commented on the research at key stages in its development. A full list of the invited and attending stakeholders can be found in Appendix 2.

Introduction

To produce a comprehensive list of potential PND identification methods we undertook a scoping literature review. This review contextualises the systematic reviews presented in the following chapters. It provides an overview of the different classification systems, the criteria for diagnosis of major depression and the differences between the classification systems, the criteria for diagnosis of PND, and the different generic and postnatal-specific measures that are available.

Historical background of classification systems

Formal classification systems of mental health problems came to the forefront of psychiatry during the mid-twentieth century because of the need to provide a consistent and standardised approach to the classification of the heterogeneous range of symptoms associated with mental health problems such as depressive syndrome. 45 The aim of classification systems is to promote increased physician communication, understanding and consensus of diagnosis, enhanced understanding of the distinct differences between disorders such as unipolar and bipolar conditions, and therefore ultimately select the most effective and appropriate treatment available for a specific condition. 45

The Diagnostic and Statistical Manual of Mental Disorders, 1st edition (DSM-I)46 published in 1952 by the American Psychiatric Association and the addition of the mental disorders section to the International Classification of Diseases, Injuries and Causes of Death, 6^th revision (ICD-6)47 in 1948 by the World Health Organization represented the first attempts to achieve these aims. The DSM-I focused on the concept of ‘reactive’ aspects of psychiatric conditions. These were limited to the classification of disturbances of mental functioning and were designated as groups of related psychiatric syndromes, termed as disorders. The ICD-6 mental disorders section was limited to psychosis, psychoneurosis and disorders of character, behaviour and intelligence. Lack of international acceptance of these classification systems and a shift in the concept of psychiatric nomenclature led to subsequent revisions of both systems, which, through close collaboration, resulted in substantial similarity between the DSM-II48 and ICD-8. 49 By the mid-1970s, a crucial problem was highlighted with the DSM-II. There was a lack of explicit criteria for diagnosis, whereby a diagnostic category was selected based on whichever one most closely resembled the characteristics of the patient. The perceived need for consistent sets of criteria for clinical work and selection of research samples, and concerns that the diagnostic approaches in the development of earlier classification systems lacked reliability, led to the development of the Research Diagnostic Criteria (RDC). 50 The RDC, which elaborated on the earlier diagnostic work of the Feighner criteria,51 was based on the concept that for each disorder explicit inclusion and exclusion criteria should exist based on symptoms or signs of illness, the level of severity or impairment experienced and the duration or course of illness.

The concepts of the RDC formed the basis for specified diagnostic criteria for all of the mental disorders included in the development of the DSM-III. 52 In contrast to the earlier editions, the DSM-III made the distinction between a diagnosis of major depressive episode and bipolar disorder, and distinguished between the presence and absence of mania. In addition, diagnoses of depressive ‘reaction’ and neurotic depression were withdrawn. The DSM-III, published in 1980, was adopted 1 year later as the official classification system within mental health facilities in the USA. The DSM-III represented a major shift from the ICD-9,53 which did not contain explicit criteria until the development of the ICD-9-Clinical Modification54 (ICD-9-CM), which provided a glossary of descriptions of abnormal mental behaviour that represented a consistent framework of reference. Subsequent revisions – DSM-IV,55 ICD-1056 – and the RDC form the current diagnostic framework for classification of mental disorders, representing the most widely accepted reference case or ‘gold standard’ diagnostic procedure for establishing a psychiatric diagnosis.

The DSM-IV is a multiaxial system that organises each psychiatric diagnosis into five independent levels: axis I – major mood disorders including depression, anxiety and bipolar disorder; axis II – personality disorders; axis III – relevant general medical conditions and physical disorders; axis IV – relevant psychosocial and environmental stressors; and axis V – global assessment of functioning. DSM-IV reflects a global evaluation in which a person may be diagnosed with a disorder on more than one independent axis. The ICD-10 contains 22 chapters each with multilevel categories. Various criticisms have been directed at both classification systems for the unnecessary complexity of the categories, arbitrary and unvalidated boundaries between categories, lack of clarity in the precise meaning of manic states and the failure to give recognition to the close relationship between anxiety and depression. 57 Despite this the DSM-IV and ICD-10 numeric diagnostic codes associated with each mental disorder are considered appropriate methods for the collection and dissemination of psychiatric morbidity and mortality data throughout the world. Nevertheless, the codes are rarely utilised outside of research, health service administrative and insurance purposes (e.g. in the USA). 57

Diagnosis and classification of depressive disorder

In the diagnosis and classification of mood disorders such as depression the RDC, DSM-IV and ICD-10 display many similarities, which is unsurprising given the development and use of the RDC, the short timescale between development of subsequent editions of each system and collaboration between the global psychiatry communities. Eight key symptoms are common between the three classification systems for a depressive episode: depressed mood, loss of interest or pleasure, disturbed sleep, altered appetite, decreased energy, inability to concentrate, psychomotor agitation or retardation and thoughts of suicide or death.

The DSM-IV uses the term ‘major depressive episode’ and the ICD-10 the term ‘depressive episode’ and neither attributes a clear aetiology to underlying biochemical processes. Structurally the systems differ. In the ICD-10 two sets of items are presented, described as ‘typical’ and ‘common’ symptoms; one set contains three typical symptoms and the other seven common symptoms. In addition, a third set presents somatic symptoms. Mild, moderate or severe episodes are based on separate diagnostic thresholds dependent on the number, type and severity of the symptoms presented.

In the DSM-IV nine items are presented in one set and severity assigned after the criteria for major depressive diagnosis have been met. In contrast the RDC allows for cases to be defined as either ‘probable’ or ‘definite’, and requires a patient to have experienced impairment in daily activities of living, with help maybe being required, and explicit exclusion of schizophrenia for a diagnosis of major depression to be met.

Table 1 displays the diagnostic criteria for the classification of depressive disorder for the DSM-IV, ICD-10 and RDC and highlights the differences between the systems in terms of the type, duration and number of symptoms required and the method of classification of severity of depressive episode. The main difficulty with the differences between the systems is that potentially it may result in an individual being classified differently on the basis of severity or by recurrence, dependent on which diagnostic criteria are applied.

Diagnosis and classification of postnatal depression

Postnatal depression is classified in the ICD-10 under the category ‘Behavioural syndromes associated with physiological disturbance and physical factors’ and in the DSM-IV under the category of ‘Mood disorders’. Limitations within current classifications for postnatal mental disorder are acknowledged. 58 In particular, limitations with the classification of PND according to the three main gold standard diagnostic systems available are that they differ with respect to the time frame of onset of the depressive episode after delivery. In addition, PND is not considered as an independent category or entity, but defined as an episode of depression that must occur within a relevant time frame post delivery. Accordingly, within the ICD-10, RDC and DSM-IV the symptoms and criteria for postnatal onset of a major or minor depressive episode do not differ from those of non-postnatal mood episodes. The set of criteria for classification of major depression are applied and if these are met then the subject receives the relevant diagnostic code plus an additional code that specifies postnatal onset.

The DSM-IV does describe specific features of postnatal mood disorder in its accompanying text. The common symptoms associated with postnatal onset are described as mood lability, fluctuation in mood, guilt due to dissonance between the mother’s mood and society’s expectation of happiness, and disinterest or preoccupation with infant well-being. However, these descriptions do not form specific inclusion and exclusion criteria. Classification for diagnosis of PND is specified by the DSM-IV if onset of the depressive episode is within 4 weeks of the birth. In contrast, the ICD-10 defines PND as mental and behavioural disorders associated with the puerperium that commence within 6 weeks of delivery, and only if they cannot be classified elsewhere. The RDC, however, has never developed a specifier for postnatal major depressive disorder (Professor Jean Endicott, Columbia University Medical Centre, 2008, personal communication) and therefore does not specify criteria or a time frame for diagnosis of PND. Researchers often select their own time frame based on the DSM-IV or ICD-10 specifiers or consider use of an expert consensus opinion as to the time period that defines postnatal onset when applying the RDC.

With the future development of new classifications (the DSM-V is under development)59 revisions suggested for inclusion in the new ICD classification system by a panel of international experts at the 1999 classification workshop in Sweden60 were the introduction of a specifier for onset within 3 months postnatally that would cover all diagnoses of mood disorder, psychosis and adjustment disorder; omission of ICD-10 code F53 (mental and behavioural disorders associated with the puerperium, not elsewhere classified); the introduction of a further psychotic diagnostic category; and the introduction of a defined diagnostic category in the mood disorders section for subsyndromal or minor depression, also permitting the postnatal specifier. Nevertheless, the suggested revisions did not go so far as to suggest more specific inclusion and exclusion criteria for the classification of PND based on the specific descriptive symptoms for postnatal mood disorder observed in the DSM-IV text. The absence of a specific postnatal classification within any of the current diagnostic manuals is thought to reflect the underlying uncertainty in the entity of PND as a distinct diagnosis. 61

Standardised clinical interview schedules

A range of psychiatric diagnostic interview schedules are available, which may be used to establish a diagnosis of depression or other psychiatric disorder based on the gold standard classification systems described previously. The need for clinical interview schedules arose because of the unreliability of psychiatric clinician diagnoses, for example during trials of the DSM-III, inter-rater reliability for major disorders ranged from kappa values of 0.28 to 0.92. 52 The aim of clinical psychiatric interviews is to distinguish using a standardised method between significant symptoms and the ordinary concerns and worries of daily life by setting requirements for clinical significance and distinguishing psychiatric symptoms from symptoms caused by drugs, alcohol and physical illness. Interview schedules may be structured or semistructured and provide a standardised method to increase the confidence in the diagnostic process and reliability of psychiatric diagnoses compared with open or unstructured interviews. Interview schedules provide greater inter-rater agreement between researchers and diagnosticians. Two types of interview schedule have been developed. The first type of interview schedule gives structure to the questions; however, the interviewer must make clinical judgements as to whether there is a need to probe with further questions based on whether the answers fulfil diagnostic criteria and therefore this type of interview schedule may be more suitable for administration by trained clinicians and clinically trained researchers. In the second type of interview schedule the questions are fully structured and interviewers are required to follow a fully specified route of questions; scoring is based on the subject’s response and clinical judgement is not required, therefore these interviews are more suitable for trained lay interviewer administration. Standardised interview schedules are usually utilised within the research context rather than within the clinical context. Asking explicit questions about the symptoms based on stringent criteria ensures that a systematic and reliable diagnosis is established, and a replicable method ensures that any comparisons subsequently made across various research studies may be considered meaningful. 62–64 A number of interview schedules have been developed and brief summaries of six are given below.

Clinician-rated identification strategies

Clinician-rated scales are measures of depression used to standardise clinical judgements and provide ratings of duration and severity of symptoms. The measures are designed for use during a clinical consultation and are not suitable for large-scale population-based screening. Several clinician-rated scales are available to assess depression and monitor treatment response.

Self-report identification strategies

A wide range of self-report instruments are available for the identification of PND. These include generic depression strategies and postnatal-specific instruments. Generic depression (and sometimes anxiety) instruments are those designed and validated for the identification of depression in non-postnatal populations. Measures have also been developed specifically for use in postnatal populations. Both generic depression measures and postnatal-specific measures assess self-reported depressive symptoms and subjects rate their symptoms in terms of their frequency and severity.

Case-finding questions

In some areas of research there has been a shift away from using self-report strategies to using case-finding questions (e.g. for depression89). The PRIME-MD90 was designed to identify depression in primary care and classifies patients according to the DSM-IV criteria. The PRIME-MD contains two initial questions about depressed mood and anhedonia that may be asked during consultation: (1) ‘During the past month, have you often been bothered by feeling down, depressed or hopeless?’ and (2) ‘During the past month, have you often been bothered by little interest or pleasure in doing things?’ A positive response to one of these questions prompts the clinician to ask the patient to complete the patient questionnaire of the PRIME-MD, a one-page self-report questionnaire that assesses five dimensions of psychiatric disorders, including mood disorders, and comprises 26 items regarding symptoms and one item regarding general health.

An alternative way of using these questions is to simply ask patients to give ‘yes’ or ‘no’ responses and use this as the identification strategy rather than as a prompt for further investigation. Whooley et al. 38 have demonstrated the use of the two brief questions, developed from the 2-item PHQ-9, compared with usual measures for identifying depression in a group of male participants attending a Veterans Affairs Medical Center.

Summary

Measures used in the identification of PND are numerous. Diagnostic measures such as the DSM, ICD and RDC are considered the gold standard reference case for classification of mood disorders within the postnatal period. However, the current classification of PND as a mood disorder within the postnatal period reflects underlying uncertainty regarding the entity of PND as a distinct diagnosis. In addition, disparities in diagnosis may arise because of differences in the time frame specified for onset and in the criteria for a major depressive disorder between the classification systems, for example more symptoms must be established using the RDC than with the DSM. There are numerous generic and postnatal-specific measures that may be used to identify possible cases of PND.

Reflection on current policy and practice within the UK

As part of the NICE guidance30 issued on antenatal and postnatal mental health a survey of primary care trusts (PCTs) in England and Wales was undertaken. The guideline development group sent a brief questionnaire to all PCTs in England and Wales with the aim of gaining an understanding of current service provision within primary care. As part of this survey, 64% of respondents included free-text comments; within these comments, 40% reported using the EPDS as an assessment tool (93% of those mentioning such tools). Despite the widespread use of the EPDS, the NICE guidance recommends using the case-finding questions developed by Whooley and colleagues with the additional help question (‘Is this something you feel you need or want help with’) if women respond yes to either of the two Whooley questions.

Key concepts in diagnostic accuracy studies

In the previous chapter a number of methods to identify PND were outlined. In this chapter we summarise the available evidence regarding the validity of these methods. Diagnostic accuracy studies aim to measure the amount of agreement between index test results and the outcome of the reference standard. When we focus on PND, for the purposes of this review, the reference standard was a standardised diagnostic interview conducted according to internationally recognised criteria. Hence, the identification strategy would be administered to a series of women in the pre- or postnatal period, and the presence or absence of PND would be determined by the outcome of the diagnostic interview.

In general, when any test is used there are four possible outcomes:

• when a person has the condition the test may be positive (true positive)

• when a person has the condition the test may be negative (false negative)

• when a person does not have the condition the test may be negative (true negative)

• when a person does not have the condition the test may be positive (false positive).

The results of a diagnostic accuracy study are often summarised in a 2 × 2 table, as shown in Table 4.

Several measures of a test’s performance can be calculated from this summary information. Two frequently used measures are sensitivity and specificity. Sensitivity of the test is the probability of a positive test result using the index test given the individual has the target condition. For example, the sensitivity of the EPDS is the proportion of women who score above a predefined threshold who have PND as classified using the diagnostic interview. Specificity of the test is the probability of a negative test result using the index test given the individual does not have the target condition. For example, the specificity of the EPDS is the proportion of women who score below a predefined threshold who do not have PND as classified using the diagnostic interview.

Sensitivity and specificity of an identification strategy vary as a function of the cut point used. A cut point is used to indicate which individuals are likely to have the target condition (e.g. those scoring at or above the cut point) and which individuals are unlikely to have the target condition (e.g. those scoring below the cut point). For example, if a cut point of 13 is used with the EPDS then women scoring 13 or above would be grouped as having PND, whereas those women scoring 12 or below would be grouped as not having PND. The EPDS is scored on a scale from 0 to 30 and any value in this range could be used as a cut point. Sensitivity and specificity are dependent upon one another – if one value decreases the other value increases. Hence, increasing the cut point increases or decreases the sensitivity and specificity of the test. Youden’s index is one way of attempting to summarise test performance into a single numeric value to aid decision-making regarding cut points and was the method chosen to define the optimum cut point in this chapter. 91

Methods

Results

A total of 64 articles met the inclusion criteria and provided sufficient data to calculate full 2 × 2 cross-tabulations. Four articles either provided data from the same sample of women, but at different time points or used different instruments or provided additional data at different cut points. Hence, the 60 studies were reported in 64 articles (see Tables 7 and 9). Of the 60 studies, four were published in languages other than English: two in Spanish,103,104 one in German105 and one in Japanese. 106

Prenatal results

This section focuses on the 10 retrieved studies (4236 women) that administered identification strategies in the prenatal period to identify women with depression and/or women at risk of developing depression in the postnatal period.

Quality assessment

Quality assessment was undertaken using QUADAS and the results are shown in Figure 2. There was variability in the results of the quality assessment. Studies did well in five out of the eight questions focusing on bias (questions 3, 5, 6, 7,and 14) and in all three questions relating to reporting quality (questions 8, 9 and 13). Over 70% of studies scored ‘yes’ in answer to these questions.

FIGURE 2.

Summary of quality assessment for prenatal studies.

Several QUADAS items were poorly described in the diagnostic studies. This included both questions relating to variability: in 50% of studies the spectrum of participants was not representative and in 60% of studies it was unclear how participants were selected for the study. Question 10 had the lowest quality rating. It was unclear from 80% of the studies whether the index tests were interpreted without knowledge of the reference standard. Interestingly, the studies performed better on question 11, a related question; for this question it was unclear in 40% of studies whether the reference standard was interpreted without knowledge of the index test.

Characteristics of included studies

Studies were published between 1990 and 2006 and were undertaken in a variety of countries: two in Nigeria,108,109 two in the UK,19,110 and a single study each in France,111 Australia,112 the USA,113 Tanzania,114 Japan115 and Malta. 116 The percentage of women with PND in these studies ranged from 5% to 25%, according to the reference standard criteria. Eight of the studies19,108,109,111,113–116 aimed to validate identification strategies for prenatal use by administering the identification strategy and reference standard during pregnancy. The two remaining studies110,112 also administered the identification strategy prenatally, but administered the reference standard postnatally to measure the predictive strength of the identification strategy.

Eight different instruments were reported as being used prenatally across the diagnostic accuracy studies included in this part of the review. Figure 3 displays the number of studies utilising the different instruments. It is clear that the EPDS was the most frequently used instrument. A number of the instruments have been translated into other languages and validated. For example, the EPDS has been translated from English into three other languages (Table 7). As previously stated there are multiple reference standards that can be used to establish whether an individual may be depressed or not and the type of depression that they have. In this review, five (50%) of the studies used DSM, two (20%) used ICD and two (20%) used RDC classifications. It was unclear in the final study which criteria were used; it was only reported that the CIDI interview schedule was used.

FIGURE 3.

Number of included studies using different instruments prenatally. BDI, Beck Depression Inventory; EPDS, Edinburgh Postnatal Depression Scale; GHQ, General Health Questionnaire; HADS, Hospital Anxiety and Depression Scale; HSCL, Hopkins Symptom Checklist; PI, Predictive Index; PRQ, Pregnancy Risk Questionnaire; SDS, Zung’s Self-rating Depression Scale.

TABLE 7 - Characteristics of included studies using identification strategies prenatally

BDI, Beck Depression Inventory; CIDI, Composite International Diagnostic Interview; CIS, Clinical Interview Schedule; DSM, Diagnostic and Statistical Manual of Mental Disorders; EPDS, Edinburgh Postnatal Depression Scale; GHQ, General Health Questionnaire; HADS, Hospital Anxiety and Depression Scale; HSCL, Hopkins Symptom Checklist; ICD, International Classification of Diseases; MINI, Mini-International Neuropsychiatric Interview; NIMH-DISC, National Institute of Mental Health Diagnostic Interview Schedule; PI, Predictive Index; PRQ, Pregnancy Risk Questionnaire; PSE, Present State Examination; RCT, randomised controlled trial; RDC, Research Diagnostic Criteria; SADS, Schedule for Affective Disorders and Schizophrenia; SCID, Structured Clinical Interview for DSM; SDS, Zung’s Self-rating Depression Scale; SPI, Standardised Psychiatric Interview.

Studies also validated identification strategies postnatally.

Study, country, study type	Version	Study setting, validation sample	Identification strategy	Time of admin., setting if different	Interview type, gold standard	Type of depression	n (%) with depression	Sample size
Abiodun, 1994,108 Nigeria, validation	English or Yoruba	Antenatal clinics at teaching hospital, all	GHQ-12	Unclear	PSE, ICD-9	Other	11 (4.6)	240
			HADS-depression			Major or minor
Adewuya et al., 2006,109 Nigeria, validation	English or Yoruba	Antenatal clinics at teaching hospital, EPDS ≥ 12 + 10% EPDS < 6	EPDS	32–36 weeks’ gestation	MINI, DSM-IV	Major only	9 (10.5)	86
						Major or minor	15 (17.4)
Adouard et al., 2005,111 France, validation	French	Antenatal clinic at hospital specialising in high-risk pregnancies, all	EPDS	28–34 weeks’ gestation	MINI, DSM-IV	Major only	15 (25.0)	60
Felice et al., 2006,116 Malta, validationa	Maltese	Antenatal clinics at hospital, all	EPDS	19 weeks’ gestation	CIS-R, ICD-10	Major or minor	32 (14.3)	223
Holcomb et al., 1996,113 US, validation	English	Women’s health centre, obstetric complications clinic, centre for diabetes and pregnancy, fetal testing unit of teaching hospital, all	BDI	32 weeks’ gestation, unclear	NIMH-DISC-III, DSM-III-R	Any psychiatric disorder	12 (11.4)	105
Kaaya et al., 2002,114 Tanzania, Validation	Kiswahli	Women included in RCT, first 50 HSCL-25 > 1.75 + random 50 HSCL-25 ≤ 1.75	HSCL-15, HSCL-25, HSCL-R	18 weeks’ gestation	SCID, DSM-IV	Major only	11 (11.1)	99
Kitamura et al., 1994,115 Japan, validationa	Japanese	Obstetric department of general hospital, all	SDS	First trimester	SADS, RDC	Major or minor	11 (9.9)	111
				34 weeks’ gestation			10 (9.8)	102
			GHQ-30	First trimester		Other	18 (16.7)	108
				34 weeks’ gestation			13 (13.3)	98
Murray and Cox, 1990,19 UK, validation	English	Antenatal clinic of large maternity hospital, all	EPDS	28 and 34 weeks’ gestation	SPI, RDC	Major only	6 (6.0)	100
						Major or minor	14 (14.0)	100
Austin et al., 2005,112 Australia, prediction	English	Midwife-based clinic at large obstetric hospital, ≥ 12 or reported felt so miserable or sad that interfered with ability to get things done or their relationships with family/friends + 7% random others	EPDS, PRQ – antenatal section	32 weeks’ gestation, posted back, and postal questionnaire 2–4 months postnatally	Computerised CIDI, unclear	Major only	69 (5.3)	1296
Cooper et al., 1996,110 UK, prediction	English	Antenatal clinics at teaching hospital, EPDS > 8	PI	32 weeks’ gestation and postal questionnaire 6–8 weeks postnatally	SCID, DSM-III-R postnatal	Major only	293 (15.3)	1916

In the diagnosis of post-natal depression, as with other mental health conditions, cut points on the scores from paper and pencil based questionnaires are often chosen to distinguish between cases and non-cases or major and minor episodes. In practice, careful consideration needs to be given to decide what the most appropriate cut point to be used is, as the cut point chosen affects the accuracy of the test. Choosing a lower cut point to distinguish between post-natal depression cases and non-cases will lead to higher sensitivity but lower specificity values. A more sensitive test will result in fewer women with PND being unidentified, however, the lower specificity of the test will result in more women being wrongly identified as having PND.

In the original validation study of the EPDS a cut point of 13 was recommended as the most likely cut point to identify women suffering from a depressive illness of varying severity. 16 It was also recommended that a cut point of 10 should be used if the scale was considered for routine use. Table 8 summarises the cut points reported in the studies using the EPDS to identify women likely to have the target condition. The studies were summarised by the type of disorder the gold standard diagnostic interview was used to classify (i.e. major depression only, major or minor depression, any psychiatric disorder and other categories). A variety of cut points were reported across all studies using the EPDS, ranging from 0 to 26; however, for all classifications the most frequently reported cut point was 13.

Table 8 - Summary of the type of depression by the cut points used for the EPDS

Cut point	Major only	Major or minor	Any psychiatric disorder	Other
0	0	0	1	0
1	1	0	0	0
2	1	0	1	0
3	1	1	0	1
4	1	2	1	1
5	1	2	0	1
6	1	3	2	1
7	4	10	2	2
8	9	14	3	2
9	10	20	4	2
10	14	22	4	6
11	12	21	4	2
12	15	20	5	3
13	18	24	7	7
14	8	12	4	2
15	7	10	2	0
16	4	6	3	0
17	2	3	2	0
18	1	1	2	0
19	1	1	1	0
20	1	0	2	0
21	1	0	1	0
22	1	0	2	0
23	1	0	0	0
24	1	0	1	0
25	1	0	1	0
26	1	0	0	0

Major depression (DSM or equivalent) only

Results for the diagnosis of major depression only were reported in six out of the 10 prenatal studies. 19,109–112,114 Two of these studies110,112 were the prediction studies and were not combined with the results of the validation studies. Both studies administered the identification strategy at 32 weeks’ gestation and posted women the identification strategy to complete in the postnatal period. The first study used the PI during pregnancy. 110 At 6–8 weeks postnatally women were posted the EPDS to complete and return. Women scoring 8 or above on the EPDS were subsequently visited at home and a diagnostic interview undertaken. The sensitivity and specificity were reported for a range of cut points (Figure 4). Sensitivity ranged from 0.05 (specificity 0.98) to 1 (specificity 0.06), with the optimal cut point at 23, in terms of the trade-off between sensitivity and specificity, giving a sensitivity of 0.59 and a specificity of 0.67.

FIGURE 4.

Single study using the Predictive Index prenatally. Values in parentheses on the left-hand side of the graph represent the various cut points used to define major depression.

The second study used the EPDS and the antenatal section of the PRQ. 112 Women were administered the EPDS and the PRQ during pregnancy and then reviewed by postal questionnaire at 2 and 4 months postnatally using the EPDS. Women scoring 12 or above on the EPDS or reporting that in the last 2 months there had been a period of 1 week or more when they felt so miserable or sad that it interfered with their ability to get things done or interfered with their relationships with friends/family were given a diagnostic interview over the telephone. The authors reported the sensitivity and specificity at three cut points for each identification strategy (Figure 5). Sensitivity for the EPDS ranged from 0.22 (specificity 0.91) to 0.78 (specificity 0.49). Sensitivity for the PRQ ranged from 0.44 (specificity 0.92) to 0.81 (specificity 0.60).

FIGURE 5.

Single prediction study using the PRQ and EPDS prenatally. Values in parentheses on the left-hand side of the graph represent the various cut points used to define major depression. EPDS, Edinburgh Postnatal Depression Scale; PRQ, Pregnancy Risk Questionnaire.

Three of the remaining four validation studies used the EPDS19,109,111 and the final study used the Hopkins Symptom Checklist (HSCL). 114 Unfortunately, there were insufficient data at each cut point for the EPDS and the HSCL to permit pooling. A summary of the sensitivity and specificity of the studies using the EPDS and the HSCL are summarised in Figures 6 and 7 respectively. Sensitivity for the EPDS ranged from 0.56 (specificity 1.00) to 1.00 (specificity 0.79). Sensitivity for the HSCL was 0.89 with specificity ranging from 0.79 to 0.85.

FIGURE 6.

Validation studies using the EPDS prenatally to diagnose major depression. Values in parentheses on the left-hand side of the graph represent the various cut points used to define major depression.

FIGURE 7.

Single validation study using the HSCL prenatally. Values in parentheses on the left-hand side of the graph represent the various cut points used to define major depression.

Major or minor depression (DSM or equivalent)

Five out of the 10 prenatal validation studies focused on major or minor depression. Three studies used the EPDS,19,109,116 one study used the HADS108 and the final study used the Zung’s SDS. 115 There were insufficient data for each instrument at the various cut points to permit pooling of diagnostic data. A summary of the sensitivities and specificities for the EPDS and Zung’s SDS are shown in Figures 8 and 9 respectively. Sensitivity for the EPDS ranged from 0.50 (specificity 0.99) to 1.00 (specificity 0.32). For the study using the Zung’s SDS women were examined twice during the prenatal period – during the first trimester and during the third trimester. Sensitivity and specificity were reported for a range of cut points for both time points. Sensitivity ranged from 0.46 (specificity 0.93) to 1.00 (specificity 0.03) during the first trimester and from 0.10 (specificity 0.97) to 1.00 (specificity 0.33) during the third trimester. The authors set an optimal cut point of 23, giving a sensitivity of 0.91 (specificity 0.70) during the first trimester and a sensitivity of 0.70 (specificity 0.76) during the third trimester. The single study utilising the HADS reported a single cut point of 8 giving a sensitivity of 0.90 and a specificity of 0.91.

FIGURE 8.

Studies using the EPDS prenatally to diagnose major or minor depression. Values in parentheses on the left-hand side of the graph represent the various cut points used to define major or minor depression.

FIGURE 9.

Single study using Zung’s Self-rating Depression Scale prenatally. Values on the left-hand side of the graph represent the time of administration (first or third trimester) and the values in parentheses display the various cut points used to define major or minor depression.

From Figure 10 we can see that, from the studies identified, the EPDS had, on average, higher sensitivity and specificity values than those for the Zung’s SDS, irrespective of the cut point used. It is also clear that the single study using HADS demonstrated relatively high sensitivity and specificity values compared with the other two instruments.

FIGURE 10.

Summary of identification strategies used prenatally at varying cut points to diagnose major or minor depression.

Any psychiatric disorder

One study113 administered the BDI during pregnancy (32 weeks’ gestation) to identify women with current or remitting depression. Sensitivities and specificities were reported for a range of cut points (Figure 11). Sensitivities ranged from 0.67 (specificity 0.96) to 1.00 (specificity 0.68). The author recommended a cut point of 16 giving a sensitivity of 0.83 and a specificity of 0.89.

FIGURE 11.

Single study using the BDI prenatally. Values in parentheses on the left-hand side of the graph represent the various cut points used to define major or minor depression.

Other classifications

Two studies108,115 used the General Health Questionnaire (GHQ) to identify other types of disorders. Pooling was not undertaken as each study classified different disorders. The first study108 focused on identifying cases of anxiety and depression. Sensitivity and specificity were reported for a single cut point of 3 and were found to be 0.83 and 0.81 respectively. The second study115 focused on identifying all RDC diagnoses. Women were examined twice during the prenatal period – in the first trimester and in the third trimester. Sensitivity and specificity were reported for a range of cut points for both time points. Sensitivity ranged from 0.28 (specificity 0.93) to 1 (specificity 0.03) during the first trimester and from 0.08 (specificity 0.95) to 0.92 (specificity 0.13) during the third trimester. The authors set an optimal cut point of 8, giving a sensitivity of 0.83 (specificity 0.71) during the first trimester and a sensitivity of 0.39 (specificity 0.82) during the third trimester.

Postnatal results

This section focuses on 55 studies (10,651 women) meeting the inclusion criteria that administered identification strategies in the postnatal period.

Quality assessment

Quality assessment was undertaken using QUADAS and the results are shown in Figure 12. There was variability in the results of the quality assessment. Studies demonstrated high quality in five out of the eight questions focusing on bias (questions 3, 5, 6, 7 and 14), with over 70% of studies scoring ‘yes’ in answer to these questions. Furthermore, for three questions (questions 3, 6 and 7) 100% of studies scored ‘yes’. Scores were also favourable for the three questions relating to reporting quality (questions 8, 9 and 13); over 80% of studies scored ‘yes’ in answer to these questions.

FIGURE 12.

Summary of quality assessment for postnatal studies.

Several QUADAS items were poorly described in the diagnostic studies. This included one of the questions relating to variability: in 56% of studies it was unclear how participants were selected for the study. The other question relating to variability (question 1) performed much better with 78% of studies scoring ‘yes’ in answer to this question. Question 10 had the lowest quality rating. It was unclear from 78% of the studies whether the index tests were interpreted without knowledge of the reference standard. Interestingly, the studies performed better on the related question 11; for this question it was unclear in 38% of studies whether the reference standards were interpreted without knowledge of the index test. Finally, the question focusing on disease progression bias also scored poorly with only 53% of studies administering the index test and reference standard within 2 weeks of each other. It was unclear in the majority of studies when instruments were actually administered.

Characteristics of included studies

Studies were published between 1987 and 2007 and were undertaken in a variety of countries: eight in the UK,16,33,34,36,117–120 four in Australia,32,35,121,122 three each in Canada,37,123,124 Japan,106,115,125 Nigeria126–128 and Spain,103,129,130 two each in Austria,105,131 France,132,133 Italy,134,135 Norway,136,137 Sweden,138,139 Thailand,140,141 Turkey142,143 and the USA,31,44,144 and single studies in Chile,145 Hong Kong,146–148 Malaysia,149,150 Malta,116 Morocco,151 Nepal,152 New Zealand,153 Peru,104 South Africa,154 Taiwan155 and United Arab Emirates. 156 The percentage of women with PND in these studies ranged from 0.3% to 76% according to the reference standard criteria. In total, 13 different instruments were reported as being used postnatally across the diagnostic accuracy studies included in this part of the review. Figure 13 shows the number of studies utilising the different instruments. It is clear that the EPDS was the most frequently used instrument. The other identification strategies were used in two or fewer studies and included the Zung’s Self-rating Depression Scale (SDS) (n = 2), HRSD (n = 1), MADRS (n = 1), Raskin (n = 1), SCL-90-R (n = 1) and EPDS–GHQ double test (n = 1). A number of the instruments have been translated into other languages and validated. For example, the EPDS has been translated from English into 18 other languages (Table 9). As previously stated there are multiple reference standards that can be used to establish whether an individual may be depressed or not and the type of depression that they have. In this part of the review, 29 (57%) of the studies used DSM, 12 (24%) used RDC classifications, six (12%) used ICD, and three (6%) used Bedford College or Catego classifications. In the final study it was unclear which criteria were used; it was reported only that interviews were undertaken using SCID.

FIGURE 13.

Number of included studies using different instruments postnatally.

TABLE 9 - Characteristics of included studies using identification strategies postnatally

BDI, Beck Depression Inventory; CIDI, Composite International Diagnostic Interview; CIS, Clinical Interview Schedule; DIS, Diagnostic Interview Schedule; DSM, Diagnostic and Statistical Manual; EPDS, Edinburgh Postnatal Depression Scale; GHQ, General Health Questionnaire; HAMD, Hamilton Rating Scale for Depression; ICD, International Classification of Diseases; MADRS, Montgomery-Asberg Depression Rating Scale; MINI, Mini-International Neuropsychiatric Interview; PAS, Psychiatric Assessment Schedule; PDSS, Postpartum Depression Screening Scale; PND, post natal depression; PRIME-MD, Primary Care Evaluation of Mental Disorders; PSE, Present State Examination; RDC, Research Diagnostic Criteria; SADS, Schedule for Affective Disorders and Schizophrenia; SCID, Structured Clinical Interview for DSM; SCL-90-R, Symptom Checklist-90-Revised; Zung’s SDS, Zung’s Self-rating Depression Scale; SPI, Standardised Psychiatric Interview.

Studies also validated identification strategies prenatally.

Study	Version	Study sample, validation sample	Identification strategy	Time of admin., setting if different	Interview type, gold standard	Type of depression	Cases (%)	Sample size	Mean age (years)
Abiodun, 2006,126 Nigeria	English or Yoruba	Postnatal clinics at three primary health-care centres, all sample	EPDS	6 weeks postnatally	PSE, ICD-10	Any psychiatric disorder	67 (18.6)	360	27.9
Adewuya et al., 2005,128 Morocco	English or Yoruba	Postnatal clinics and immunisation clinics at five health centres, EPDS ≥ 9 + BDI ≥ 10 + random sample with EPDS < 9 + BDI < 10	EPDS, BDI	6 weeks postnatally	SCID-NP, DSM-III-R	Major or minor	128 (14.6)	876	28.8
Agoub et al., 2005,151 Nigeria	Arabic	Maternal and infant health unit in a primary health-care setting, all sample	EPDS	2–3 weeks postnatally	MINI, DSM-IV	Major only	27 (18.8)	144	30.3
Aslan et al., 1997,143 Turkey	Turkish	Primary health-care centre, unclear	EPDS	Unclear	SCID-NP, unclear	Any psychiatric disorder	19 (25.0)	76	–
Aydin et al., 2004,142 Turkey	Turkish	Women attending primary health-care clinics, all sample	EPDS	Within 1 year postnatally	SCID-I, DSM-IV	Major or minor	49 (14.4)	341	26.6
Bagedahl-Strindlund et al., 1998,138 Sweden	Swedish	All Swedish speaking mothers visiting 15 Well Baby Clinics, EPDS > 12	EPDS	3 months postnatally, mean = 13.5 weeks	Unclear, RDC	Major only	1 (0.3)	303	28.2
						Major or minor	19 (6.3)	–	–
Barnett et al., 1999,122 Australia	Vietnamese	Four antenatal clinics, all sample	EPDS	6 weeks postnatally, at home	DIS, DSM-III-R	Major only	5 (4.4)	113	27.0
	Anglo-Celtic						7 (6.7)	105	–
	Arabic						9 (9.5)	95	–
Beattie-Clarke, 2003,124 Canada	English	Advertisements and women attending prenatal, postnatal and parenting groups, all sample	EPDS, PDSS, BDI-II	1–12 months postnatally: 1–2 months, n = 18; 2–4, n = 31; 4–6, n = 12; 6–9, n = 14; 9–12, n = 27	SCID, DSM-IV	Major only	17 (16.5)	103	23.8
						Major or minor	25 (24.3)	–	–
Beck and Gable, 20 01,31 USA; Beck and Gable, 200144 USA	English	Preparation for childbirth classes or newspaper advert, all sample	PDSS, EPDS, BDI-II	2–12 weeks postnatally, mean = 39 days	SCID, DSM-IV	Major only	18 (14.8)	122	31.0
						Major or minor	46 (30.7)	150	–
Beck and Gable, 2005,144 USA	Spanish	Acute care, community teaching hospital and perinatal clinic, all sample	PDSS-7, PDSS-35	2–12 weeks postnatally, mean = 27 days	SCID, DSM-IV	Major or minor	55 (36.7)	150	25.8
Benvenuti et al., 1999,134 Italy	Italian	Obstetric clinic at large university hospital, all sample	EPDS	8–12 weeks postnatally	MINI, DSM-III-R	Any psychiatric disorder	18 (15.9)	113	31.9
Bergant et al., 1998,105 Austria	German	Randomly selected women after childbirth in university hospital, all sample	EPDS	4 days postnatally	Unclear, ICD-10	Major or minor	22 (20.0)	110	28.6
Berle et al., 2003,136 Norway	Norwegian	Women attending routine postnatal visits, EPDS ≥ 8 + every tenth EPDS < 8	EPDS	6–12 weeks postnatally	MINI, DSM-IV	Major only	27 (27.0)	100	30.3
						Major or minor	41 (41.0)	–	–
Boyce et al., 1993,32 Australia	English	Mothers’ Advisory Clinics (baby health clinics staffed by community nurses) + women referred to hospital psychiatric department for outpatient treatment of PND, all sample	EPDS	Within 6 months postnatally, mean = 12 weeks postnatally at home	DIS, DSM-III-R at home	Major only	9 (8.7)	103	28.4
Carpiniello et al., 1997,135 Italy	Italian	Consecutive admissions for delivery at obstetric clinic in university hospital, all sample	EPDS	4–6 weeks postnatally, at home	PSE, ICD-Catego at home	Any psychiatric disorder	9 (14.8)	61	31.6
Cox et al., 1987,16 UK	English	Part of larger study, health visitors identified women as potentially depressed + 12 non-depressed women, all sample	EPDS	3 months postnatally, at home	SPI, RDC at home or clinic	Major or minor	35 (41.7)	84	26.0
Cox et al., 1996,120 UK	English	Part of controlled study including non-postnatal women, recruited from GP age/sex registers mailed EPDS, EPDS ≥ 9 + one-third EPDS < 9	EPDS	6 months postnatally, at home	CIS, RDC at home	Major only	8 (6.2)	128	26.3
						Major or minor	21 (16.3)	–	–
Eberhard-Gran et al., 2001,137 Norway	Norwegian	Women from a community-based questionnaire study, EPDS ≥ 10 + a control group; control group women included those with EPDS < 10 and who delivered closest in time to the high scoring woman	EPDS	10 weeks postnatally, local primary health-care centre	PRIME-MD, DSM-IV	Major only	9 (16.1)	56	–
						Major or minor	21 (37.5)	–	–
Felice et al., 2006,116a Malta	Maltese	Antenatal clinic at hospital, all sample	EPDS	8 weeks postnatally, home	CIS-R, ICD-10	Major or minor	18 (8.1)	223	27.1
Garcia-Esteve et al., 2003,129 Spain	Spanish	Maternity hospital, EPDS ≥ 9 + 10% random EPDS < 9	EPDS	6 weeks postnatally	SCID, DSM-III-R	Major only	36 (3.2)	1123	30.0
						Major or minor	100 (8.9)	–	–
Ghubash et al., 1997,156 United Arab Emirates	Arabic	Postnatal ward at hospital, all sample	EPDS	1 week postnatally	PSE, Catego, 8 weeks postnatally	Any psychiatric disorder	13 (13.7)	95	28.6
Guedeney and Fermanian, 1998,133 France	French	Random sample of women involved with public service mainly staffed with community nurses, provides home visits, consultations in baby health clinics + identified as probably depressed by nurses, all sample	EPDS	Within first 4 months postnatally, mean = 7 weeks at home	PSE, RDC	Major or minor	45 (51.7)	87	30.4
Harris et al., 1989,33 UK	English	Part of larger population study for postnatal thyroid dysfunction, all sample	BDI	6 weeks postnatally, routine postnatal follow-up clinic	Unclear, DSM-III	Major only	19 (14.7)	129	24.6
			Raskin				22 (15.0)	146	–
			MADRS				22 (15.1)	147	–
			EPDS				22 (17.5)	126	–
Holt, 1995,153 New Zealand	English	All women attending 6-week postnatal examinations, all sample	EPDS	6 weeks postnatally	Review of case notes, DSM-III	Major only	7 (5.8)	121	26.0
Jadresic et al., 1995,145 Chile	Spanish	Antenatal clinic at university hospital, all sample	EPDS	2–3 months postnatally	PAS, RDC	Major or minor	11 (10.2)	108	27.7
Jardi et al., 2006,132 France	French	Maternity unit at university hospital, EPDS > 8 + same number randomly selected EPDS ≤ 8	EPDS	3–5 days postnatally	MINI, DSM-IV, telephone at 8 weeks postnatally	Major or minor	112 (30.8)	364	28.8
Kitamura et al., 1994,115a Japan	Japanese	Volunteers participating in follow-up study, attending obstetric department at general hospital, all sample	Zung’s SDS	5 days postnatally	SADS, RDC	Major or minor	11 (12.1)	91	28.0
				1 month postnatally			9 (8.9)	101	–
			GHQ-30	5 days postnatally		Other	18 (20.0)	90	–
				1 month postnatally			16 (16.2)	99	–
Lawrie et al., 1998,154 South Africa	Multiple: English, African, Zulu, Tswana, Sotho, Xhosa + others	Mother and child academic hospital serving only women who experienced obstetric complications, required Caesarean section or requested sterilisation for family planning – women seen at 6 weeks, all sample	EPDS	6 weeks postnatally, EPDS read to women	Unclear, DSM-IV	Major only	8 (7.8)	102	28.1
						Major or minor	25 (24.5)	–	–
Lee et al., 2000,147 Hong Kong; Lee et al., 2001,146 Hong Kong; Lee et al., 2003,148 Hong Kong	Chinese	Consecutive admissions to postnatal ward at university-affiliated general hospital, all sample	EPDS, GHQ-12, BDI, EPDS–GHQ double test	6 weeks postnatally	SCID-NP, DSM-III-R at 6 weeks postnatally	Major or minor	17 (11.7)	145	29.0
			EPDS, GHQ-12, BDI	2 days postnatally
Leverton and Elliott, 2000,34 UK	English	Antenatal clinic at hospital for larger prevention study, all sample	EPDS	3 months postnatally, at home	PSE, Catego (last 2 weeks of month)	Other	9 (4.5)	199	–
					PSE, Catego (any time in last month)		10 (5.0)	–	–
					PSE, Bedford (last 2 weeks of month)		14 (7)	–	–
					PSE, Bedford (any time in last month)		16 (8.0)	–	–
Mahmud et al., 2003,150 Malaysia; Mahmud et al., 2004,149 Malaysia	Malay	Women attending health centre, all sample	EPDS	4–12 weeks postnatally, mean = 55.5 days	CIS, ICD-10	Major or minor	9 (14.1)	64	28.7
			BDI-II	4–12 weeks postnatally, mean = 55.1 days			7 (11.5)	61	26.6
Matthey et al., 2001,35 Australia	English	Couples attending Preparation for Parenthood classes in hospital in Australia, all sample	EPDS	6–7 weeks postnatally, at home	DIS, DSM-IV 3 days after completion	Other	37 (15.5)	238	27.2
						Major or minor	24 (10.4)	230	–
Milgrom et al., 2005,121 Australia	English	Women attending 47 maternal and child health centres in Australia, EPDS ≥ 12	EPDS, BDI	4 months postnatally	CIDI, DSM-IV, timing between screening and diagnosis averaged 1–2 weeks	Other	260 (75.6)	344	30.1
Murray and Carothers, 1990,36 UK	English	Postnatal wards at maternity hospitals approached for other study, EPDS ≥ 13 + random sample EPDS 10–12 + 1 in 10 of EPDS < 10	EPDS	6 weeks postnatally, postal	SPI, RDC	Major only	Values estimated by logistic regression	702	–
						Major or minor
Muzik et al., 2000,131 Austria	German	Drawn from larger epidemiological study of PND in Austria, EPDS > 7	EPDS, SDS, SCL-90-R depression	3 or 6 months postnatally	SCID, DSM-III-R	Major only	9 (18.0)	50	28.0
Navarro et al., 2007,130 Spain	Spanish	Routine postnatal follow-up at acute care teaching hospital, random sample 15% EPDS < 7, 50% EPDS between 7 and 9, 60% EPDS between10 and 12, 100% EPDS ≥ 13	GHQ-12, EPDS	6 weeks postnatally	SCID-NP, DSM-IV	Other	180 (44.4)	405	–
Okano et al., 1996,106 Japan	Japanese	Antenatal clinic in the later part of pregnancy, all sample	EPDS	1 month postnatally	SADS, RDC	Major or minor	4 (8.5)	47	–
Pitanupong et al., 2007,141 Thailand	Thai	Women giving birth and receiving follow-up care at university hospital, all sample	EPDS	6–8 weeks postnatally	Unclear, DSM-IV	Major or minor	38 (10.8)	351	27.9
Regmi et al., 2002,152 Nepal	Unclear	Immunisation visit at postnatal clinic at university hospital, EPDS ≥ 13 + every fifth EPDS < 13	EPDS	2–3 months postnatally	SCID, DSM-IV	Major only	5 (5.0)	100	–
Teng et al., 2005,155 Taiwan	Taiwanese	Maternity wards at university hospital, all sample	EPDS, BDI-II	6 weeks postnatally	MINI, DSM-IV	Any psychiatric disorder	24 (11.8)	203	29.0
Terren et al., 2003,103 Spain	Spanish	Women attending postnatal clinic at university hospital, EPDS ≥ 9 + random 16% sample with EPDS < 9	EPDS	6 weeks postnatally	SCID-NP, DSM-IV	Major or minor	100 (29.9)	334	30.1
Thompson et al., 1998,118 UK	English	Consecutive women presenting at 16 weeks’ gestation at hospital, 50% GHQ-30 ≥ 7 + ~50% positive thyroid antibody	EPDS	8 weeks postnatally	Unclear, RDC	Major or minor	8 (7.3)	109	–
				12 weeks postnatally			17 (11.4)	149	–
				20 weeks postnatally			18 (12.8)	141	–
				28 weeks postnatally			15 (10.4)	144	–
			HAMD	8 weeks postnatally			10 (9.3)	108	–
				12 weeks postnatally			15 (10.6)	141	–
				20 weeks postnatally			12 (8.9)	135	–
				28 weeks postnatally			13 (9.6)	135	–
Uwakwe and Okonkwo, 2003,127 Nigeria	English or Igbo	Wards and postnatal clinics at university hospital, all sample	EPDS	6–8 weeks postnatally, 98% within 6 weeks	CIDI adapted, ICD-10, SCL within 48 hours of screening	Any psychiatric disorder	24 (10.7)	225	28.9
Vega-Dienstmaier et al., 2002,104 Peru	Spanish	Family planning centres and routine paediatric follow-up consultations, all sample	EPDS	Within 1 year	SCID, DSM-IV	Major only	19 (5.9)	321	25.1
Vittayanont et al., 2006,140 Thailand	Thai	University hospital, all sample	PDSS	6–8 weeks postnatally	Unclear, DSM-IV	Major only	4 (1.0)	400	27.9
						Major or minor	40 (10.0)
Werrett and Clifford, 2006,117 UK	Punjabi and English	Convenience sample of bilingual mothers from caseloads of health visitors, all sample	EPDS	5–8 weeks postnatally	CIDI, ICD-10, 1 week later at home or health centre	Major or minor	7 (30.4)	23	28.6
Whiffen, 1988,123 Canada	English	Public health prenatal classes, all sample	BDI	6–8 weeks postnatally, mean = 45 days	SADS, RDC interview, average 7.6 days after BDI	Major only	9 (7.5)	120	28.0
						Major or minor	21 (17.5)	–	–
Wickberg and Hwang, 1996,139 Sweden	Swedish	17 child health clinics, EPDS 12 both times + random sample (n = 16) EPDS 10 or 11 + (n = 21) EPDS < 10	EPDS	Twice at 2 and 3 months postnatally	MADRS, DSM-III-R, 1–2 weeks after EPDS at home	Major only	56 (43.8)	128	28.1
Yamashita et al., 2000,125 Japan	Japanese	Mothers admitted for delivery to maternity ward of university hospital, all sample	EPDS	1 month postnatally, routine 1-month postnatal check	SADS, RDC, telephone at 3 weeks postnatally	Major or minor	11 (14.7)	75	30.5
Yoshida et al., 1997,119 UK	Japanese	Antenatal classes for pregnant Japanese women and advertisement in maternity guidebook for pregnant Japanese women in the UK, all sample	EPDS	1 month postnatally, postal	SADS, RDC at home, 3 months postnatally	Major or minor	8 (8.2)	97	30.0
Zelkowitz and Milet, 1995,37 Canada	English	Two community health-care centres, unclear	EPDS	6–8 weeks postnatally, telephone	SCID-NP, DSM-III-R, within 10 days of telephone screening	Any psychiatric disorder	43 (48.3)	89	30.5

Major depression (DSM or equivalent) only

In total, 21 studies were included that compared identification strategies with a gold standard in the postnatal period for the diagnosis of major depression. Eight identification strategies were used: EPDS (n = 18), PDSS (n = 3), BDI (n = 2), BDI-II (n = 2), MADRS (n = 1), Raskin (n = 1), SCL-90-R depression (n = 1) and Zung’s SDS (n = 1). Insufficient data were available to permit pooling for the majority of identification strategies. Only the EPDS had sufficient data to pool at a variety of cut points (7–16). A summary of the sensitivity and specificity of the studies using all of the instruments is shown in Figure 14.

FIGURE 14.

Summary of identification strategies used postnatally at varying cut points to diagnose major depression.

When the studies were combined the pooled sensitivities ranged from 0.60 (95% CI 0.47 to 0.71) to 0.96 (95% CI 0.90 to 0.98) and specificities ranged from 0.45 (95% CI 0.26 to 0.66) to 0.97 (95% CI 0.92 to 0.99) for the diagnosis of major depression at a range of cut points using the EPDS (Table 10). The optimal cut point, in terms of the trade-offs between sensitivity and specificity (i.e. Youden’s index), was 12 (Figure 15). At this cut point a pooled sensitivity of 0.86 (95% CI 0.81 to 0.89) and specificity of 0.87 (95% CI 0.80 to 0.92) were derived. The positive likelihood ratio associated with the sensitivity and specificity was 6.66 (95% CI 4.32 to 10.28) and the pooled DOR was 40.54 (95% CI 24.22 to 67.88). A summary plot of sensitivity and specificity in ROC space, summarising each study at cut point 12, weighted by study size can be seen in Figure 16; additional plots for the other cut points can be found in Appendix 4. Although all of the studies used the EPDS, used a widely recognised gold standard and were focusing on identifying major depression the I² value of 63% was high, indicating high between-study heterogeneity. Thus, care should be taken when interpreting the results.

TABLE 10 - Summary of the sensitivity and specificity of the EPDS postnatally at varying cut points for major depression

DOR, diagnostic odds ratio; LR+, positive likelihood ratio; LR–, negative likelihood ratio.

Note: Values in parentheses are 95% confidence intervals.

Cut point	Sensitivity	Specificity	LR+	LR–	DOR	n	I 2
7	0.96 (0.90 to 0.98)	0.45 (0.26 to 0.66)	1.76 (1.20 to 2.57)	0.09 (0.03 to 0.26)	19.89 (5.39 to 73.43)	4	0
8	0.94 (0.89 to 0.97)	0.58 (0.46 to 0.69)	2.23 (1.68 to 2.96)	0.10 (0.05 to 0.20)	21.45 (9.05 to 50.83)	9	0
9	0.92 (0.87 to 0.96)	0.66 (0.55 to 0.75)	2.71 (1.99 to 3.69)	0.12 (0.06 to 0.21)	23.52 (10.36 to 53.43)	10	21
10	0.92 (0.87 to 0.95)	0.77 (0.70 to 0.83)	4.00 (2.94 to 5.45)	0.11 (0.06 to 0.18)	37.17 (17.61 to 78.44)	14	38
11	0.87 (0.80 to 0.92)	0.84 (0.76 to 0.89)	5.35 (3.50 to 8.17)	0.15 (0.10 to 0.24)	34.91 (16.49 to 73.92)	12	59
12	0.86 (0.81 to 0.89)	0.87 (0.80 to 0.92)	6.66 (4.32 to 10.28)	0.16 (0.12 to 0.22)	40.54 (24.22 to 67.88)	15	63
13	0.79 (0.74 to 0.83)	0.89 (0.85 to 0.92)	7.50 (5.38 to 10.45)	0.23 (0.19 to 0.29)	32.29 (20.76 to 50.22)	18	86
14	0.73 (0.64 to 0.80)	0.92 (0.86 to 0.95)	9.04 (5.04 to 16.21)	0.30 (0.22 to 0.40)	30.37 (13.96 to 66.07)	8	89
15	0.65 (0.55 to 0.74)	0.96 (0.92 to 0.98)	15.10 (8.02 to 28.27)	0.36 (0.28 to 0.48)	41.41 (20.03 to 85.61)	7	88
16	0.60 (0.47 to 0.71)	0.97 (0.92 to 0.99)	22.46 (7.25 to 69.56)	0.42 (0.31 to 0.56)	54.04 (16.55 to 176.48)	4	93

FIGURE 15.

Graphical summary of the sensitivity and specificity of the EPDS postnatally at varying cut points to diagnose major depression.

FIGURE 16.

EPDS SROC plot for diagnosis of major depression at cut point 12.

When the psychometric attributes were pooled across the studies, a variety of levels of between-study heterogeneity were identified (range 0–93%). From Table 10 we can see that moderate or high levels of between-study heterogeneity were identified when a cut point of 11 or higher was used. Meta-regression was undertaken for cut points 11, 12 and 13; because of the low number of studies reporting data for cut points above 13 we were unable to investigate any potential sources of heterogeneity.

Table 11 displays the results of the univariate analyses. As meta-regression was undertaken within each cut point, multiple tests were performed and therefore we must be cautious in interpreting the results. It can be seen that timing seemed to be an important factor across cut points 12 and 13. The DORs for studies administering the EPDS within 6 weeks postnatally were higher than those in studies administering the EPDS after 6 weeks postnatally (cut point 12: 84 versus 27; cut point 13: 68 versus 16). This indicates better discriminatory performance if the EPDS is administered within 6 weeks postnatally. The I² value reduced from 63% to 40% and 86% to 47% for cut points 12 and 13, respectively, when timing was included. For cut point 11, verification bias seemed to be the most important factor in explaining the heterogeneity in the DOR of the primary studies. The DOR for studies that performed a diagnostic interview on all women included in the study (or a random sample) was 26, whereas the DOR for studies that only performed a diagnostic interview on a non-random sample of women was 65. This analysis highlights that the DOR was inflated when a non-random sample of women was given a diagnostic interview, thus indicating the presence of verification bias.

TABLE 11 - Univariate meta-regression analyses

CI, confidence interval; DOR, diagnostic odds ratio.

Cut point	Predictor	DOR1	DOR2	Beta-coefficient (95% CI)	p-value	I 2
11	Prevalence	–	–	5.59 (–8.55 to 19.72)	0.40	62
	Verification bias	26.01	64.58	0.39 (0.08 to 1.98)	0.23	41
	Blinding	73.93	24.18	3.15 (0.66 to 15.10)	0.13	37
	Timing	56.81	21.29	2.08 (0.36 to 12.07)	0.38	53
12	Prevalence	–	–	–1.78 (–6.95 to 3.40)	0.47	59
	Verification bias	45.97	58.92	0.76 (0.22 to 2.70)	0.65	61
	Blinding	83.61	38.02	2.23 (0.68 to 7.36)	0.17	52
	Timing	83.78	26.68	3.14 (1.13 to 8.74)	0.03	40
13	Prevalence	–	–	–4.07 (–7.87 to –0.27)	0.04	58
	Verification bias	40.14	32.37	1.33 (0.30 to 2.92)	0.69	87
	Blinding	54.80	31.07	1.75 (0.45 to 6.86)	0.40	78
	Timing	68.48	15.67	4.10 (1.78 to 9.44)	0.002	47

Table 12 shows the results of the multivariate analyses. As meta-regression was undertaken within each cut point, multiple tests were performed, and, also, because of the low number of studies included we should be cautious when interpreting the results. When all variables were considered simultaneously, timing seemed to be the most important factor in explaining the heterogeneity in the DOR of the primary studies across all cut points, although none of the variables showed statistically significant differences at the p ≤ 0.05 level. Across all cut points the I² value reduced, from 59% to 35%, 63% to 44% and 86% to 44% for cut points 11, 12 and 13 respectively. It is worth noting that the I² values in the multivariate analyses are not substantially reduced compared with the univariate analyses. As the I² values were above 0% there are obviously still some sources of variation that remain unexplained; however, the I² values did reduce and were classified as moderate.

TABLE 12 - Multivariate meta-regression analyses

Cut point	Predictor	Beta-coefficient (95% CI)	p-value	I 2
11	Prevalence	8.09 (–6.46 to 22.65)	0.23	35
	Verification bias	0.43 (0.01 to 20.08)	0.62
	Blinding	1.13 (0.02 to 61.28)	0.94
	Timing	3.48 (0.52 to 23.42)	0.17
12	Prevalence	1.69 (–5.00 to 8.37)	0.59	44
	Verification bias	1.40 (0.15 to 13.10)	0.75
	Blinding	2.42 (0.24 to 24.03)	0.41
	Timing	3.09 (0.75 to 12.83)	0.11
13	Prevalence	–1.06 (–5.81 to 3.70)	0.64	44
	Verification bias	1.61 (0.33 to 7.75)	0.52
	Blinding	2.10 (0.41 to 10.71)	0.34
	Timing	2.65 (0.87 to 8.04)	0.08

Major or minor depression (DSM or equivalent)

In total, 35 studies compared identification strategies with a gold standard in the postnatal period for the diagnosis of major or minor depression. Eight identification strategies were used: EPDS (n = 28), PDSS (n = 4), BDI (n = 4), BDI-II (n = 3), GHQ-12 (n = 2), EPDS–GHQ (n = 1), Hamilton Rating Scale for Depression (HAMD) (n = 1), PDSS-short (n = 1) and Zung’s SDS (n = 1). Insufficient data were available to permit pooling for the majority of identification strategies. The EPDS had sufficient data to pool at a variety of cut points (7–16), the BDI at cut point 10 and the HAMD at cut point 11. A summary of the sensitivity and specificity of all of the studies is given in Figure 17. This shows that, from the studies identified, the EPDS, BDI, BDI-II and HAMD seem to have, on average, higher sensitivity and specificity values than the other identification strategies irrespective of the cut point used.

FIGURE 17.

Summary of identification strategies used postnatally at varying cut points to diagnose major or minor depression.

When the studies were combined the pooled sensitivities ranged from 0.31 (95% CI 0.19 to 0.47) to 0.91 (95% CI 0.80 to 0.96) and specificities ranged from 0.67 (95% CI 0.57 to 0.76) to 0.99 (95% CI 0.98 to 0.99) for the diagnosis of major or minor depression at a range of cut points using the EPDS (Table 13). The optimal cut point, in terms of the trade-offs between sensitivity and specificity (i.e. Youden’s index), was 10 (Figure 18). At this cut point a pooled sensitivity of 0.82 (95% CI 0.76 to 0.86) and specificity of 0.86 (95% CI 0.79 to 0.91) were derived. The positive likelihood ratio associated with the sensitivity and specificity was 5.95 (95% CI 3.80 to 9.32) and the pooled DOR was 28.04 (95% CI 15.35 to 51.22). A summary plot of sensitivity and specificity in ROC space, summarising each study at cut point 10, weighted by study size can be seen in Figure 19; additional plots for the other cut points can be found in Appendix 4. Although all of the studies used the EPDS, used a widely recognised gold standard and were focusing on identifying major or minor depression the I² value of 97% was very high. Thus, care should be taken when interpreting the results.

TABLE 13 - Summary of the sensitivity and specificity of the EPDS postnatally at varying cut points for major or minor depression

DOR, diagnostic odds ratio; LR+, positive likelihood ratio; LR–, negative likelihood ratio.

Note: Values in parentheses are 95% confidence intervals.

Cut point	Sensitivity	Specificity	LR+	LR–	DOR	n	I 2
7	0.91 (0.80 to 0.96)	0.67 (0.57 to 0.76)	2.76 (2.07 to 3.68)	0.13 (0.05 to 0.32)	20.95 (7.65 to 27.34)	10	72
8	0.91 (0.84 to 0.95)	0.75 (0.64 to 0.83)	3.58 (2.46 to 5.23)	0.12 (0.06 to 0.22)	30.34 (12.59 to 73.08)	14	94
9	0.85 (0.77 to 0.91)	0.82 (0.75 to 0.88)	4.78 (3.30 to 6.93)	0.18 (0.11 to 0.29)	26.68 (13.12 to 54.26)	20	96
10	0.82 (0.76 to 0.86)	0.86 (0.79 to 0.91)	5.95 (3.80 to 9.32)	0.21 (0.16 to 0.28)	28.04 (15.35 to 51.22)	22	97
11	0.72 (0.64 to 0.79)	0.91 (0.85 to 0.95)	8.11 (4.86 to 13.55)	0.31 (0.23 to 0.40)	26.60 (14.73 to 48.02)	21	97
12	0.68 (0.62 to 0.74)	0.93 (0.88 to 0.96)	9.81 (5.94 to 16.18)	0.34 (0.29 to 0.41)	28.57 (16.56 to 49.27)	20	98
13	0.66 (0.57 to 0.74)	0.93 (0.90 to 0.95)	9.08 (6.53 to 12.62)	0.36 (0.29 to 0.46)	24.89 (16.36 to 37.87)	24	97
14	0.53 (0.48 to 0.58)	0.96 (0.92 to 0.98)	12.45 (6.47 to 23.97)	0.49 (0.44 to 0.54)	25.52 (12.66 to 51.45)	12	97
15	0.39 (0.32 to 0.46)	0.98 (0.96 to 0.99)	17.45 (9.83 to 30.98)	0.62 (0.55 to 0.70)	28.01 (14.53 to 54.00)	10	79
16	0.31 (0.19 to 0.47)	0.99 (0.98 to 0.99)	29.13 (10.70 to 79.26)	0.69 (0.56 to 0.86)	41.92 (12.90 to 136.2)	6	41

FIGURE 18.

Graphical summary of sensitivity and specificity postnatally at varying cut points for major or minor depression.

FIGURE 19.

EPDS SROC plot for diagnosis of major or minor depression at cut point 10.

When the studies using the BDI at a single cut point were combined the pooled sensitivity was 0.72 (95% CI 0.54 to 0.85) and specificity was 0.91 (95% CI 0.83 to 0.95) for the diagnosis of major or minor depression (Table 14). The positive likelihood ratio associated with the sensitivity and specificity was 7.66 (95% CI 3.34 to 17.60) and the pooled DOR was 24.74 (95% CI 5.95 to 102.77). The I² value was high at 99%, hence care should be taken when interpreting the results. For the HAMD the pooled sensitivity was 0.74 (95% CI 0.60 to 0.84) and specificity was 0.99 (95% CI 0.98 to 1.00) for the diagnosis of major or minor depression (Table 14). The positive likelihood ratio associated with the sensitivity and specificity was 86.83 (95% CI 32.29 to 233.46) and the pooled DOR was 325.93 (95% CI 102.36 to 1037.84). As a single study provided the data to pool for the HAMD (the instrument was administered at different time points on the same women using the same cut point) it is unsurprising that the I² value is 0. A summary plot of sensitivity and specificity in ROC space, summarising each study for the BDI and HAMD, weighted by study size can be seen in Figure 20.

TABLE 14 - Summary of the sensitivity and specificity of the BDI and HAMD for major or minor depression

BDI, Beck Depression Inventory; DOR, diagnostic odds ratio; HAMD, Hamilton Rating Scale for Depression; LR+, positive likelihood ratio; LR–, negative likelihood ratio.

Note: Values in parentheses are 95% confidence intervals.

	Sensitivity	Specificity	LR+	LR–	DOR	I 2
BDI	0.72 (0.54 to 0.85)	0.91 (0.83 to 0.95)	7.66 (3.34 to 17.60)	0.31 (0.17 to 0.58)	24.74 (5.95 to 102.8)	99
HAMD	0.74 (0.60 to 0.84)	0.99 (0.98 to 1.00)	86.83 (32.29 to 233.46)	0.27 (0.17 to 0.42)	325.93 (102.4 to 1037.8)	0

FIGURE 20.

SROC plot for diagnosis of major or minor depression.

When the psychometric attributes were pooled across the studies, high levels of between-study heterogeneity were identified (range 41–98%). From Table 13 we can see that moderate or high levels of between-study heterogeneity were identified across all cut points. Meta-regression was undertaken for cut points 7–15; because of the low number of studies reporting data for cut point 16 we were unable to investigate any potential sources of heterogeneity.

Table 15 shows the results of the univariate analyses. As meta-regression was undertaken within each cut point multiple tests were performed and therefore we must be cautious in interpreting the results. It can be seen that verification bias seemed to be an important factor across cut points 7–13. The DORs for studies undertaking a diagnostic interview for the whole sample (or random sample) were lower than those for studies undertaking diagnostic interviews on a non-random sample. For cut points 14 and 15, blinding seemed to be the most important factor in explaining the heterogeneity in the DOR of the primary studies. The DORs were higher in those studies in which the index test results were interpreted without knowledge of the results of the reference standard than in those studies in which the index test results were interpreted with knowledge of the results of the reference standard. Despite many of these results being statistically significant at the p ≤ 0.05 level, the I² values still remain high.

TABLE 15 - Univariate meta-regression analyses

CI, confidence interval; DOR, diagnostic odds ratio.

Cut point	Predictor	DOR1	DOR2	Beta-coefficient (95% CI)	p-value	I 2
7	Prevalence	–	–	2.25 (–8.99 to 13.49)	0.66	75
	Verification bias	6.07	70.15	0.08 (0.02 to 0.45)	0.01	49
	Blinding	22.77	8.06	2.21 (0.22 to 21.83)	0.45	75
	Timing	10.77	13.23	0.71 (0.06 to 8.91)	0.76	74
8	Prevalence	–	–	–2.99 (–14.29 to 8.31)	0.58	94
	Verification bias	8.25	85.92	0.12 (0.03 to 0.50)	0.008	80
	Blinding	26.94	26.93	1.24 (0.11 to 14.61)	0.85	94
	Timing	32.51	13.64	1.55 (0.12 to 19.12)	0.71	94
9	Prevalence	–	–	–1.54 (–10.02 to 6.94)	0.71	96
	Verification bias	10.25	146.70	0.09 (0.03 to 0.32)	0.001	86
	Blinding	36.30	22.71	2.03 (0.24 to 17.13)	0.50	96
	Timing	28.88	18.30	1.19 (0.21 to 6.68)	0.83	96
10	Prevalence	–	–	–2.22 (–8.04 to 3.59)	0.44	96
	Verification bias	14.99	70.00	0.24 (0.07 to 0.76)	0.02	92
	Blinding	37.45	29.15	1.36 (0.23 to 7.99)	0.72	97
	Timing	38.51	21.15	1.50 (0.32 to 7.05)	0.59	97
11	Prevalence	–	–	0.42 (–4.68 to 5.52)	0.87	94
	Verification bias	15.89	47.10	0.33 (0.10 to 1.08)	0.07	90
	Blinding	36.40	19.06	1.67 (0.31 to 8.91)	0.53	97
	Timing	19.94	27.68	0.64 (0.14 to 2.93)	0.54	98
12	Prevalence	–	–	–0.02 (–5.18 to 5.14)	0.99	96
	Verification bias	22.86	39.25	0.53 (0.15 to 1.93)	0.32	94
	Blinding	33.87	22.74	1.27 (0.27 to 5.84)	0.75	98
	Timing	29.67	26.59	1.00 (0.24 to 4.24)	0.99	98
13	Prevalence	–	–	–1.87 (–5.97 to 2.23)	0.35	92
	Verification bias	17.05	46.43	0.35 (0.15 to 0.81)	0.02	85
	Blinding	38.03	22.93	1.33 (0.27 to 6.47)	0.72	97
	Timing	20.14	29.82	0.56 (0.13 to 2.44)	0.42	97
14	Prevalence	–	–	–2.98 (–13.33 to 7.36)	0.54	97
	Verification bias	14.18	43.59	0.32 (0.07 to 1.51)	0.13	87
	Blinding	66.49	13.85	4.97 (1.18 to 20.99)	0.03	83
	Timing	27.56	22.76	1.22 (0.21 to 6.99)	0.81	93
15	Prevalence	–	–		0.29	74
	Verification bias	34.16	37.72	0.83 (0.15 to 4.59)	0.81	79
	Blinding	66.70	23.75	3.16 (0.88 to 11.37)	0.07	53
	Timing	44.09	29.36	1.58 (0.33 to 7.70)	0.52	71

Table 16 shows the results of the multivariate analyses. As meta-regression was undertaken within each cut point multiple tests were performed and, furthermore, because of the low number of studies included we should be cautious in interpreting the results. Although differences for some variables (verification bias) were statistically significant at some cut points the coefficients were extremely low and I² values high, thus indicating that none of the a priori sources of heterogeneity were predictive when entered as covariates in the meta-regression and that there are obviously still some sources of variation that remain unexplained. At cut point 15, when all variables were considered simultaneously in the model the I² value reduced from 79% to 0% indicating that these variables explained all of the heterogeneity in the DORs of the primary studies.

TABLE 16 - Multivariate meta-regression analyses

CI,confidence interval

Cut point	Predictor	Beta-coefficient (95% CI)	p-value	I 2
7	Prevalence	3.24 (–13.12 to 19.59)	0.63	47
	Verification bias	0.05 (0.002 to 1.27)	0.06
	Blinding	0.42 (0.01 to 22.97)	0.60
	Timing	0.89 (0.08 to 10.37)	0.91
8	Prevalence	–1.32 (–14.43 to 11.79)	0.83	84
	Verification bias	0.09 (0.01 to 1.47)	0.08
	Blinding	0.39 (0.02 to 8.69)	0.51
	Timing	1.37 (0.10 to 18.39)	0.79
9	Prevalence	–0.90 (–7.47 to 5.67)	0.78	87
	Verification bias	0.06 (0.01 to 0.32)	0.003
	Blinding	0.41 (0.06 to 2.62)	0.32
	Timing	0.88 (0.26 to 3.00)	0.82
10	Prevalence	–1.06 (–6.71 to 4.59)	0.70	92
	Verification bias	0.18 (0.04 to 0.92)	0.04
	Blinding	0.52 (0.08 to 3.19)	0.45
	Timing	1.28 (0.31 to 5.22)	0.71
11	Prevalence	–0.49 (–5.66 to 4.68)	0.84	88
	Verification bias	0.39 (0.07 to 2.17)	0.27
	Blinding	1.72 (0.24 to 12.34)	0.57
	Timing	0.38 (0.09 to 1.62)	0.17
12	Prevalence	0.02 (–6.14 to 6.18)	0.995	95
	Verification bias	0.58 (0.09 to 3.61)	0.54
	Blinding	1.21 (0.15 to 9.80)	0.85
	Timing	0.81 (0.15 to 4.31)	0.80
13	Prevalence	–1.40 (–5.49 to 2.70)	0.48	84
	Verification bias	0.44 (0.14 to 1.39)	0.15
	Blinding	1.57 (0.35 to 6.93)	0.53
	Timing	0.45 (0.13 to 1.53)	0.19
14	Prevalence	–4.47 (–14.22 to 5.28)	0.31	85
	Verification bias	1.31 (0.07 to 23.07)	0.83
	Blinding	7.09 (0.28 to 181.88)	0.20
	Timing	0.56 (0.07 to 4.35)	0.52
15	Prevalence	–10.14 (–17.75 to –2.52)	0.02	0
	Verification bias	5.03 (0.94 to 26.97)	0.06
	Blinding	27.18 (4.18 to 176.66)	0.01
	Timing	0.12 (0.02 to 0.81)	0.04

Any psychiatric disorder

Any psychiatric disorder was used to classify studies when the disorder under study was not differentiated between major depression or minor depression or dysthymia, but did not incorporate other types of psychiatric disorder such as anxiety. Eight studies compared identification strategies with a gold standard in the postnatal period for the diagnosis of what we have labelled ‘any psychiatric disorder’. Table 17 summarises the types of disorder that fall into this category. Two identification strategies were used: EPDS (n = 8) and BDI-II (n = 1). Insufficient data were available to permit pooling for the BDI-II. The EPDS had sufficient data to pool at a variety of cut points (9–14). A summary of the sensitivity and specificity of all studies is summarised in Figure 21.

TABLE 17 - ‘Any psychiatric disorder’ classifications

Study	Types of disorder
Abiodun, 2006126	Postnatal depression
Aslan et al., 1997143	Depression
Benvenuti et al., 1999134	Major depression, anxiety and mood disorders
Carpiniello et al., 1997135	Clinically depressed
Ghubash et al., 1997156	Catego definition of depression
Teng et al., 2005155	Major depressive disorder, depressive disorder not otherwise stated and dysthymic disorder
Uwakwe and Okonkwo, 2003127	Affective morbidity
Zelkowitz and Milet, 199537	Any depressive disorder

FIGURE 21.

Summary of identification strategies used postnatally at varying cut points to diagnose any psychiatric disorder.

When the studies were combined the pooled sensitivities ranged from 0.38 (95% CI 0.28 to 0.48) to 0.86 (95% CI 0.75 to 0.92) and specificities ranged from 0.85 (95% CI 0.78 to 0.90) to 0.99 (95% CI 0.97 to 1.00) for the diagnosis of any psychiatric disorder at a range of cut points using the EPDS (Table 18). The optimal cut point, in terms of the trade-offs between sensitivity and specificity (i.e. Youden’s index), was 9 (Figure 22). At this cut point a pooled sensitivity of 0.86 (95% CI 0.75 to 0.92) and specificity of 0.87 (95% CI 0.73 to 0.94) were derived. The positive likelihood ratio associated with the sensitivity and specificity was 6.54 (95% CI 3.19 to 13.43) and the pooled DOR was 39.46 (95% CI 18.98 to 82.06). A summary plot of sensitivity and specificity in ROC space, summarising each study at cut point 9, weighted by study size can be seen in Figure 23; additional plots for the other cut points can be found in Appendix 4. When the psychometric attributes were pooled across the studies, a variety of levels of between-study heterogeneity were identified (range 0–60%). From Table 18 we can see that, apart from cut point 10, the I² values were relatively low, indicating low levels of between-study heterogeneity. Because of the low numbers of studies reporting data at each cut point care should be taken when interpreting the results.

TABLE 18 - Summary of the sensitivity and specificity of the EPDS postnatally at varying cut points for any psychiatric disorder

DOR, diagnostic odds ratio; LR+, positive likelihood ratio; LR–, negative likelihood ratio.

Note: Values in parentheses are 95% confidence intervals.

Cut point	Sensitivity	Specificity	LR+	LR–	DOR	n	I 2
9	0.86 (0.75 to 0.92)	0.87 (0.73 to 0.94)	6.54 (3.19 to 13.43)	0.17 (0.10 to 0.28)	39.46 (18.98 to 82.06)	4	44
10	0.81 (0.64 to 0.91)	0.85 (0.78 to 0.90)	5.34 (3.84 to 7.43)	0.23 (0.12 to 0.44)	23.35 (11.88 to 45.91)	4	60
11	0.70 (0.53 to 0.83)	0.93 (0.86 to 0.97)	10.32 (5.23 to 20.36)	0.32 (0.20 to 0.52)	32.13 (14.84 to 69.60)	4	0
12	0.57 (0.48 to 0.65)	0.97 (0.96 to 0.98)	21.08 (12.73 to 34.92)	0.44 (0.36 to 0.54)	47.62 (26.70 to 84.94)	5	0
13	0.66 (0.47 to 0.80)	0.96 (0.91 to 0.98)	17.35 (8.25 to 36.51)	0.36 (0.22 to 0.57)	48.52 (25.15 to 93.63)	7	0
14	0.38 (0.28 to 0.48)	0.99 (0.97 to 1.00)	33.38 (11.87 to 93.88)	0.63 (0.54 to 0.74)	52.86 (17.17 to 162.77)	4	25

FIGURE 22.

Graphical summary of pooled sensitivity and specificity of the EPDS postnatally at varying cut points to diagnose any psychiatric disorder.

FIGURE 23.

EPDS sROC plot for diagnosis of any psychiatric disorder at cut point 9.

Other classifications

Other classifications incorporated studies that considered depression alongside a mixed group of related psychiatric disorders, such as anxiety, and did not report psychometric properties for depression alone. Five studies were identified that used identification strategies to identify other types of disorders. 34,35,115,121,130 Pooling was not undertaken as each study was classifying different disorders (Table 19); however, a summary of the sensitivities and specificities are shown in Figure 24. The earliest study115 focused on identifying all RDC diagnoses using the GHQ. Women were examined twice during the postnatal period, at 5 days and 1 month postnatally. Sensitivity and specificity values were reported for a range of cut points for both time points. Sensitivity ranged from 0.11 (specificity 0.96) to 1.00 (specificity 0.08) at 5 days postnatally and from 0.25 (specificity 0.94) to 1.00 (specificity 0.05) at 1 month postnatally. The authors set an optimal cut point of 8, giving a sensitivity of 0.28 (specificity 0.79) at 5 days postnatally and a sensitivity of 0.50 (specificity 0.84) at 1 month postnatally.

TABLE 19 - Other classifications

NOS, not otherwise stated; RDC, Research Diagnostic Criteria.

Study	Types of disorders
Kitamura et al., 1994115	All RDC diagnoses
Leverton and Elliott, 200034	Depressed mood alone, neurotic or psychotic depression and other diagnoses
Matthey et al., 200135	Depressive and anxiety disorders
Milgrom et al., 2005121	Major depressive disorder, depressive disorder NOS, adjustment disorder with depression, mixed anxiety depressive disorder, bipolar disorder, psychotic disorder and dysthymic disorder
Navarro et al., 2007130	Mood disorders, anxiety disorders, adjustment disorders and other diagnoses

FIGURE 24.

Summary of identification strategies used postnatally at varying cut points to diagnose other classifications.

The second study34 focused on identifying women with depressed mood alone, neurotic or psychotic depression and other diagnoses using the EPDS at 3 months postnatally. Sensitivity and specificity values were reported for two cut points using four versions of the gold standard (Catego any time in the month, Bedford College any time in the month, Catego last 2 weeks of the month, Bedford College last 2 weeks of the month). Sensitivity ranged from 0.64 (specificity 0.92) to 1.00 (specificity 0.84).

The third study35 focused on depressive and anxiety disorders, aiming to validate the EPDS for use among women and their partners; the results summarised here relate only to the performance of the EPDS in women. Sensitivity and specificity values were reported for various cut points. Sensitivity ranged from 0.38 (specificity 0.95) to 0.97 (specificity 0.23). The authors found an optimal cut point of 8, giving a sensitivity of 0.70 and a specificity of 0.73.

The next study121 focused on identifying women with major depressive disorder, depressive disorder not otherwise specified, adjustment disorder with depression, mixed anxiety depressive disorder, bipolar disorder, psychotic disorder and dysthymic disorder using the EPDS and BDI at 4 months postnatally. Sensitivity and specificity values were reported at three cut points for both instruments. Sensitivity ranged from 0.92 (specificity 0.80) to 0.96 (specificity 0.64) for the BDI and from 0.79 (specificity 0.86) to 0.91 (specificity 0.60) for the EPDS. In this population the BDI had higher diagnostic efficiency than the EPDS.

In the final study130 the EPDS and GHQ-12 were used to identify women with mood disorders, anxiety disorders, adjustment disorders and other diagnoses at 6 weeks postnatally. Sensitivity and specificity were reported at a range of cut points for both instruments. Sensitivity ranged from 0.64 (specificity 0.94) to 0.99 (specificity 0.30) for the GHQ-12 and from 0.57 (specificity 0.97) to 0.98 (specificity 0.58) for the EPDS. The authors found an optimal cut point of 5 for the GHQ-12, giving a sensitivity of 0.81 and a specificity of 0.80, and a cut point of 10 for the EPDS, giving a sensitivity of 0.86 and a specificity of 0.85.

Discussion

In total, 14 identification strategies have been validated among women during pregnancy or the postnatal period (up to 1 year). Identification strategies included PND-specific measures and also generic depression identification strategies (specific: EPDS, PDSS, PRQ, PI; generic: BDI, GHQ, HADS, HSCL, HAMD, Zung’s SDS, SCL-90-R, Raskin, MADRS; other: EPDS–GHQ double test). By far the most frequently used identification strategy was the EPDS. Quality assessment was undertaken using QUADAS and there was variability in the results across the individual items. It is interesting to note that none of the studies fulfilled all of the quality criteria. Studies demonstrated high quality in five out of the eight questions focusing on bias (questions 3, 5, 6, 7, and 14) and the three questions relating to reporting quality (questions 8, 9 and 13). The poorest quality rating was associated with question 10, regarding whether the index test results were interpreted without knowledge of the reference standard results. This item is important as interpretation of the results of the index test may be influenced by knowledge of the results of the reference standard leading to inflated estimates of diagnostic accuracy. Two other poorly reported items were selection criteria and disease progression bias.

There were sufficient data for postnatal studies across a large number of cut points of the EPDS to be able to combine results and produce pooled summary estimates of sensitivity and specificity. However, there were insufficient data at each cut point for most other identification strategies to be able to pool data. For major depression only, the pooled estimates of sensitivity and specificity for the EPDS ranged from 0.60 to 0.96 and from 0.45 to 0.97 respectively. For any psychiatric disorder, the pooled estimates of sensitivity and specificity for the EPDS ranged from 0.38 to 0.86 and from 0.85 to 0.99 respectively. Finally, for major or minor depression, the pooled estimates of sensitivity and specificity for the EPDS ranged from 0.31 to 0.91 and from 0.67 to 0.99 respectively. In addition, for major or minor depression there were sufficient data to pool the BDI and HAMD at a single cut point. Results from this analysis highlighted that generic identification strategies may be less sensitive than the EPDS, but more specific. Caution should be taken when interpreting these results as there were only four studies included for the generic identification strategies and also the cut points used may not be the best cut points to use to identify women with PND.

When the psychometric attributes were pooled across the studies, high levels of between-study heterogeneity were identified in most analyses (major depression: 0–93%; major or minor depression: 41–98%). Unfortunately, none of the a priori sources of heterogeneity were predictive in a meta-regression analysis, and high levels of between-study heterogeneity remained in the model. There was some suggestive evidence that the timing of administration of the EPDS (within 6 weeks postnatally or not) may be an important factor in influencing diagnostic performance. Two other variables, verification bias and blinding, also demonstrated some potential effects on diagnostic performance, hence any future validation studies should be undertaken using methods to avoid such biases. Further research would be informative to identify key sources of heterogeneity and specifically whether different items need to be considered when pooling psychometric instruments in diagnostic accuracy studies.

There are limitations to the review. In many cases multiple data points were presented for the studies evaluated. Multiple data points arose for many reasons: more than one identification strategy was used; more than one cut point was presented; the identification strategy was repeatedly administered; two versions of the identification strategy were used; different classifications of depression were recorded; more than one reference standard was used. Studies were pooled at individual cut points in an attempt to overcome the fact that multiple data points were presented. The drawbacks of pooling at individual cut points were multiple testing and fewer studies included, thus reductions in power. Furthermore, because of the low number of studies included at some cut points, meta-regression could not be undertaken and potential sources of heterogeneity could not be explored.

A unique issue that arises when undertaking meta-analyses of diagnostic accuracy studies is variations in the cut point chosen to indicate a positive test. The higher the cut point value chosen, the higher the specificity and the lower the sensitivity will be. Threshold effects create a potential source of heterogeneity and to reduce this we pooled studies within individual cut points. However, despite pooling within cut points, threshold effects were still apparent. Methods to simultaneously model thresholds within thresholds would be useful. Finally, difficulties arose in defining the appropriate cut points to be used in practice. The relevant cut point depends on the viewpoint taken: statistical, clinical or economic. From a statistical viewpoint it would seem sensible to find the optimal values based on the trade-off between sensitivity and specificity (e.g. Youden’s index). From a clinical perspective it may be more important, in this situation of identifying PND, for the identification strategy to be more sensitive than specific. Nevertheless, maximising sensitivity estimates will lead to subsequent reductions in the specificity estimates resulting in more false positives being identified. Increasing the number of false positives will increase costs and resource use within the NHS.

The identification strategies reviewed here appear to be able to identify PND in women during pregnancy and the postnatal period with a degree of accuracy that is similar, if not slightly better, to that for depression in the general population. In an evaluation of case-finding instruments for identifying patients with major depression or dysthymia in primary care,83 16 instruments were assessed in 38 studies and the overall sensitivity was 0.79 (95% CI 0.74 to 0.83) and overall specificity was 0.75 (95% CI 0.70 to 0.81). Equivalent estimates from this review for the EPDS resulted in overall estimates of sensitivity of 0.86 (95% CI 0.81 to 0.89) and of specificity of 0.87 (95% CI 0.80 to 0.92). In summary, the EPDS is the most frequently reported identification strategy and its diagnostic performance seems reasonably good.

Reflection on current policy and practice within the UK

As outlined in Chapter 1 current NICE guidance recommends the use of two questions to identify possible depression and a third question if the women answers ‘yes’ to either of the initial questions [(1) ‘During the past month, have you often been bothered by feeling down, depressed or hopeless?’; (2) ‘During the past month, have you often been bothered by little interest or pleasure in doing things?’; and (3) ‘Is this something you feel you need or want help with?’). No studies were identified in this review that validated this recommended method to identify women with PND. Studies undertaken to validate the Whooley questions plus the help question in a general depressed population (including both men and women) found that the sensitivity was 0.96 (95% CI 0.86 to 0.99) and the specificity was 0.89 (95% CI 0.87 to 0.91). 39 The diagnostic performance of the three questions in a depressed population was better than the performance of the EPDS in a postnatal population; however, until further research is undertaken we cannot be confident that these results will be maintained when the three questions are used in a postnatal population. Within the NICE guidance, the psychometric properties of the EPDS postnatally were based on eight validation studies identified from a literature search and a single systematic review (which included eight validation studies). Within this systematic review and meta-analysis we identified 48 validation studies of the EPDS post-natally.

Methods

Results

In total, 16 studies were eligible and are presented in Tables 20–25, which include information on study characteristics and results. Three studies assessed both women’s and health professionals’ views and so are repeated in the respective tables. Studies are reported according to the timing of the administration of the strategy, either postnatally or prenatally. Studies that administered the identification strategy both before and after the women gave birth were called perinatal studies. None of the included studies assessed non-response rates for the completion of a questionnaire as an indication of the acceptability of a strategy.

Topics that were identified from the research literature as likely to affect views on identification strategies were used to define a number of subgroups. These were:

• method of administration of the strategy (including expectation/awareness that the strategy was to be implemented; timing, setting and mode of administration; feedback of results)

• difficulties in answering questions, such as being a sensitive topic, or fear of disclosure/being honest

• interpersonal relationships between women and health professionals

• cultural or ethnic differences

• training issues (particularly when exploring health professionals’ views)

• overall acceptability of the identification strategy.

Studies belonging to each subgroup were identified and synthesised in a narrative fashion according to the timing of the identification strategy (postnatal, prenatal, or perinatal) and whether the studies reflected the views of women or health professionals.

Discussion

Women liked to be informed in advance that they would be asked to complete a questionnaire to identify PND. 167,168 If they were not prepared they might have felt anxious and reluctant to answer questions honestly. Health professionals also felt that informing mothers about the completion of questionnaires at earlier visits helped to normalise the process of identification of PND and make it acceptable to them. 172,173 They suggested that this subject should be raised early on at an antenatal visit. 173,175 Having raised the subject of PND identification there was variation in when women and health professionals thought was the appropriate time to undertake this. Women said that it should be at around 8 weeks postnatally, as a period of adjustment to being a mother was required. 168 Some health professionals agreed with the need for mothers to adapt to having a baby and thought that PND identification should be undertaken at around 6–8 weeks postnatally,172,173 although others felt that it should be introduced during pregnancy. 175,180 There was consensus, however, between women167,168,176 and health professionals172,173,175 that questionnaires should be administered at the woman’s home. Administering questionnaires in clinics was found to be too distracting and uncomfortable for women, whereas at home there was more privacy and time to discuss issues concerning emotional health, which made it easier to respond to questions more honestly. In general, when administering the instrument women preferred to talk rather than complete a standardised questionnaire, and they were critical of the lack of dialogue that could result from a pen and paper assessment. 167–169 Similarly, in general, women appreciated being given feedback of the results of completing the questionnaire as they then felt listened to and found it helpful to talk freely with the health visitor. 167,168 Health professionals also found that following the completion of a questionnaire it was useful to discuss the results and engage in a two-way conversation. 172

Several studies made criticisms about specific questions used in the instruments to identify PND. For the EPDS the last question about the thought of self-harm was identified as inappropriate for reasons including the extent of shame that this would bring on an individual in some cultures. 170,174 English women also reported feeling vulnerable and not wanting to admit to self-harm168 and black Caribbean women found the item difficult to answer because of the uncertainty about whether it referred to accidental or deliberate self-harm. 178 Similarly health professionals expressed difficulties with this question because of the ambiguity about ‘cutting’ or attempting suicide. 172 English women and health professionals also found difficulties with the question about sleeping as mothers might not have been able to sleep because of being woken by the baby. 172,174 More generally, women expressed concerns about offering truthful answers to questions about depression for fear of disclosure and losing their baby or the shame of being seen as an incompetent mother. 167–170 Moreover, women in four studies167,169,178,180 reported that they deliberately lied on the questionnaire for fear of answering questions honestly. In contrast, some health professionals thought that the EPDS was easy to understand and complete173 or did not identify specific questions as sensitive or difficult to ask. 180

As mothers found it helpful to talk about their feelings and difficult to answer questions honestly, the interpersonal relationship with the health professional was important. Indeed, two studies found that the pre-existing relationship with the person administering the questionnaire affected how it was completed. 168,175 Most mothers wanted the health professional to be supportive and caring and show an interest,167,168,174 which helped increase the likelihood of honesty and disclosure. 167 Furthermore, the relationship between mothers and health professionals improved over a number of meetings, when it became more likely that a mother would disclose additional information,172 and so several contacts antenatally helped to establish a good relationship for the early detection of PND. 173 Although an instrument such as the EPDS was seen as a route into dialogue with a health professional,172,180 what mattered was the warmth and intimacy of the relationship. 180 Women and health professionals also found it important to be clear about the role of the health professional so that mothers would not perceive them as agents of social control. 169,173 Adequate training in identifying PND would therefore appear necessary to facilitate the relationship between mother and health professional. Most health professionals received in-house training in PND identification using the EPDS, but had no problems using the EPDS before training. In contrast, some staff sought training in counselling to acquire the skills for dealing with the disclosures that their work uncovered. 172 Health visitors also had very little training in PND, which was mostly self-directed study using the internet or reading journals, and thought that they would benefit further from more consistent training. 173 Therefore the perceived need for training ranges from not requiring any training to specialised training in counselling. Training health visitors in the appropriate use of the EPDS and non-directive counselling skills has also been shown to reduce EPDS scores. 181

Some concerns raised about the acceptability of the EPDS were culturally specific. 178 Compared with Anglo-Celtic women, who did not consider questions culturally inappropriate, Vietnamese women felt that questionnaires used to identify PND would not elicit true feelings for fear of shame. 170,174 Black and minority ethnic women were also reported as having problems with the absence of any culturally contextualised understanding or awareness of PND,180 and black Caribbean women found some items in the EPDS difficult to answer, although other items were good indicators of depression. 178 In contrast, indigenous First Nations and Métis women of Canada found questions in instruments to be culturally sensitive and appropriate and were mostly not offended. 124 English and Punjabi women also found completing the EPDS to be acceptable. 117

Both qualitative and quantitative methods were used to address the overall question of whether or not postnatal identification strategies were acceptable to women and health professionals. Most qualitative studies117,174,180 showed that English-speaking women thought that the EPDS was acceptable, although one study167 that used in-depth interviews found that 21 of 39 English-speaking women felt that the EPDS was unacceptable. This study has limitations, though, in that the interviews were undertaken several months after the completion of the EPDS and thus the study may have selected women who preferred to talk rather than complete forms. Nearly all First Nations and Métis women found questionnaires to be appropriate, and black Caribbean women overall found the EPDS to be acceptable. 178 From the perspective of the health professionals, qualitative studies showed that staff were positive about using the EPDS,172 although while it could be a useful prompt in face-to-face discussion173,180 it could be overused and open to manipulation. 173 Two surveys including several hundred Australian women found that the majority indicated that PND identification using the EPDS was ‘comfortable’ to ‘very comfortable’171 or that the EPDS was fairly easy to complete. 179 A survey177 of health visitors also found that the majority would use the 17-item pregnancy questionnaire and the EPDS in practice and most did not feel uncomfortable with the content of the questionnaire. Another survey179 of 801 health professionals (GPs, nurses and midwives) found that the majority thought that the EPDS was easy or fairly easy to complete by their patients and found no discomfort in explaining the EPDS.

For our study a systematic approach was used to underpin the synthesis of the evidence base concerning the acceptability of postnatal identification strategies to women and health professionals. We identified several studies by undertaking a comprehensive literature search, independently selected eligible studies and extracted data, and synthesised data using the textual narrative approach. We also collected data from qualitative studies, which examined in detail the views of a small sample of women or health professionals on the acceptability of identification strategies, and from surveys, which quantified the acceptability of these strategies for samples of several hundred. There are limitations, however, to the validity of the findings and the generalisability of the review.

First, it was difficult to integrate the evidence from qualitative and quantitative research because the surveys only asked broad questions about, for example, how comfortable women found the process of PND identification. It was not possible to assess how different themes discussed in qualitative research studies were included in the surveys and how subsequently this affected the estimates of acceptability of identification strategies to women and health professionals. Therefore, from surveys alone it was not possible to understand what made an identification strategy acceptable or not. Furthermore, the surveys could be open to selection bias in that only one179 of the four surveys selected a random sample. The surveys also had poor response rates further indicating the possibility of bias in those women or health professionals who chose to respond and possibly limiting the validity and generalisability of the findings. The response rates were 52% for a survey of Australian women,171 29% for a survey of Scottish health visitors,177 and 59%, 20%, 50% and 29%, respectively, for a survey of Australian women and GPs, nurses and midwives. 179 No surveys were conducted on women other than those in Australia.

Second, although qualitative research methods are a more useful design to help understand the acceptability of identification strategies from women’s and health professionals’ perspectives, it is important to critically appraise the study designs. Most studies recruited a convenience sample of participants and collected data using semistructured interviews. Subsequently, the sampling strategy did not promote the generalisability of the individuals included in the sample. Moreover, the two studies167,178 that used in-depth interviews to explore this subject were the most critical about the acceptability of postnatal identification strategies using the EPDS. Most studies collected data around the same time as the process of PND identification, but a criticism of both studies which used in-depth interviews is that the data were collected several months after the process of PND identification. This suggests that there might be recall bias in the response of the participants or that a particular type of participant wanted to take part in the study when collecting data several months later. However, it might be that women’s views about the acceptability of PND identification soon after the administration of, for example, the EPDS might reflect the relief that labour is over and that it takes more negative aspects of this process longer to integrate. This has been found with women after childbirth, whose assessments of their experiences can change from positive to negative over time. 182

Third, most studies that explored women’s views about PND identification strategies did report their EPDS scores. This helps with understanding the prevalence and severity of PND in the sample. The findings, however, were often not presented and subsequently discussed in the context of whether the women had PND or not, which might influence how acceptable they found the identification strategy. For example, Buist et al. 179 found from their survey of women who completed a questionnaire about the acceptability of the EPDS that 87% of women who had an EPDS score < 13 experienced no discomfort in completing the EPDS compared with 64% of women with an EPDS score > 13, which was statistically significant (p < 0.0001). Furthermore, the EPDS and other self-completed questionnaires are not incontrovertible as women will be misclassified as having PND or not (i.e. false positives and false negatives). None of the studies explored women’s views about the acceptability of an identification strategy when this misclassification occurred.

Fourth, there are a number of identification strategies for identifying PND, such as the use of standardised or generic questionnaires postnatally or prenatally, the identification of known risk factors through prenatal identification of PND, or the use of training packages to enhance the awareness of health professionals about the clinical signs of PND. Our review only identified studies of pre- or postnatal identification strategies using standardised or generic questionnaires, and every study but one used the EPDS as the instrument of choice. Although the preponderance for the use of the EPDS reflects clinical practice, this has highlighted an evidence gap in the assessment of the acceptability of alternative identification strategies. In particular, it is possible that strategies such as prenatal identification using known risk factors or training packages targeted at health professionals could avoid the need for a paper and pen assessment and issues surrounding women answering these questions openly and honestly.

Reflection on current policy and practice within the UK

As outlined in Chapter 1, recent NICE guidance recommends the use of case-finding questions to help identify women with PND [(1) ‘During the past month, have you often been bothered by feeling down, depressed or hopeless?’, (2) ‘During the past month, have you often been bothered by little interest or pleasure in doing things?’ and (3) ‘Is this something you feel you need or want help with?’]. Whilst these questions appear to offer a relatively quick and convenient way for healthcare professionals to identify post-natal depression, the acceptability of these questions remains unexplored. Some of the findings from this review would indicate that the case finding questions may potentially be acceptable to women and health professionals. The finding that women wanted to talk rather than complete a paper and pencil questionnaire, would potentially support the use of the case findings questions over using the EPDS. Furthermore, the case finding questions may overcome some of the difficulties surrounding specific items on the EPDS (e.g. question 10) and may be less culturally sensitive. The findings that women and health professionals should be forewarned of the use of formal methods to identify PND, that they should be administered in the woman’s’ home and the importance of the interpersonal relationship between the woman and the health professional may still be important factors in the acceptability of the case finding questions. Further research into the acceptability of these difference identification strategies is desirable.

Methods of review

Inclusion criteria

Two reviewers screened titles and abstracts generated by the searches to identify potentially eligible studies. Any disagreements were resolved by consensus or deferred to a third party if necessary. Full papers of potentially eligible studies were obtained and assessed for inclusion independently by two reviewers. Articles were eligible for inclusion if they fulfilled the criteria outlined below.

Study design

The review included RCTs and controlled trials. Trials had to include an identification strategy component that was incorporated in some way into clinical decision-making. Hence, trials that included an identification strategy to gain a baseline measure of depression and did not use the results from the identification strategy for any other purpose were excluded from the review. A hierarchy of evidence was established that incorporated different models of assessment of the effectiveness of PND identification strategies on maternal and infant outcomes. For the purposes of this review we considered the highest level of evidence to come from trials comparing methods to identify PND with no formal methods or delayed methods to identify PND (Figure 25).

FIGURE 25.

Level I evidence.

There is some suggestive evidence from a review by the US Preventative Services Task Force189 that using identification strategies for depression can become effective when they are accompanied by organisational enhancements of care, involving clinician education, support from case managers and a collaborative care approach between specialists and primary care physicians. 190 Hence, this systematic review examined the impact of using identification strategies by themselves and also identification strategies that had been incorporated into some from of educational or organisational enhancement of care.

The first model of level II evidence (top of Figure 26) compares using an identification strategy plus some form of enhancement of care with usual care. Hence, participants in the control group would not be formally assessed using the PND identification strategy. The second model (bottom of Figure 26) involves the whole sample of participants receiving the identification strategy, but randomising them so that the intervention group receive feedback of their results from a health-care professional and the control group do not receive feedback of their results. Some form of enhancement of care could also be incorporated into this model.

FIGURE 26.

Level II evidence.

Finally, it was anticipated that there would be relatively few, if any, trials that would fit the above two models; hence, broader criteria were developed and level III evidence incorporated trials that used the PND identification strategy as an eligibility criterion in a trial (Figure 27). For example, women scoring 12 or above on the EPDS would be included in the trial and randomised to the intervention group or usual care group. Women scoring 11 or below on the EPDS would not be eligible for inclusion in the trial. Trials comparing more than one intervention were also included as long as there was a usual care group to which women could be allocated.

FIGURE 27.

Level III evidence.

Trials that could be categorised into one of the three levels of evidence described above were eligible for inclusion in phase 3 of the review.

Interventions

Trials comparing the implementation of any PND identification strategy, with or without enhancement of postnatal care, in any setting were included. Any type of intervention was included as long as it was compared with usual primary care for expectant and postnatal mothers. Head-to-head trials comparing interventions without the option of participants being randomised to a usual care group (e.g. a trial comparing antidepressants with placebo) were excluded from the review. Trials assessing interventions aimed at health-care professionals and/or at women with PND were included in the review. Studies providing level III evidence were divided into prevention and treatment studies, and were categorised into four types: educational, psychosocial, psychological and more complex interventions that included educational and/or psychosocial and/or psychological components. Prevention studies were those that recruited women and delivered the intervention under study while women were pregnant, with the aim of preventing the onset of PND. Treatment studies recruited women and delivered the intervention under study during the postnatal period (any time immediately following birth until 1 year postnatally).

Outcomes

Data on maternal and infant outcomes were included, however defined. Short-term (< 6 months), medium-term (6–12 months) and longer-term (> 12 months) outcomes were all considered. Trials reporting no maternal or infant outcomes were excluded from the review.

Results

Overall, 30 studies (from 32 publications) were included in phase 3 of the review. Only five of these studies provided level I or level II evidence, with the remaining 25 studies providing level III evidence. Among the studies categorised as providing level III evidence there were four main types of intervention under study: educational, psychosocial, psychological and more complex interventions that included educational and/or psychosocial and/or psychological components. Fuller details of the included studies are provided in Appendix 5.

Level I and level II evidence

Five studies191–195 were identified that provided level I or level II evidence. One of the studies compared the rate of identification of PND using the EPDS with the rate of spontaneous detection from routine clinical evaluations by physicians and mid-level health-care providers in a residency training programme. 191 Women were recruited from a single hospital in the USA and allocated to groups based on the date of delivery; those women who delivered on even days were assigned to the EPDS group. Women assigned to the EPDS group were posted the EPDS to complete at 6 weeks postnatally. Data for the control group were gathered by review of the postnatal record. Only 37% (n = 79) of women in the intervention group returned the EPDS and only 54% (n = 96) of the control group women attended the 6-week postnatal visit. The rate of detection of PND was 35.4% (n = 28) in the EPDS group and 6.3% (n = 6) in the routine clinical evaluation group (p < 0.0001). A cut point of 10 was used to indicate a high risk for PND.

The remaining studies focused on detecting women experiencing PND and subsequently reducing the number experiencing PND or the severity of PND (indicated by a reduction in identification scores), rather than detecting PND alone as in the previous study. All of the studies used the EPDS (alone or in combination with other strategies) as an identification strategy, although the EPDS was administered at various time points: 25 weeks’ gestation;192 4 weeks postnatally;193 1 day before discharge, 1 week postnatally and 6 weeks postnatally;194 or unclear. 195 The EPDS was used as an outcome measure across all of the studies, either as a continuous variable comparing mean EPDS scores between the intervention and control groups or as a dichotomous variable comparing the number of women scoring above or below a cut point on the EPDS. Outcomes were assessed at 36 weeks’ gestation;192 6 weeks postnatally194 or 4 months postnatally. 193,195 Additional outcomes to be considered were number of visits to midwife and GP (n = 2), physical and mental components of the SF-36 (n = 1), number of referrals (n = 1), women’s views about care (n = 1), overall satisfaction with care (n = 1) and the use of a support contact number (n = 1). The results of each individual study for the threshold EPDS scores and the mean EPDS scores are displayed in Figures 28 and 29 respectively. For the studies providing dichotomous outcomes the I² value was 0%.

FIGURE 28.

Forest plot showing odds ratios for EPDS threshold scores.

FIGURE 29.

Summary of mean EPDS scores in the intervention and control groups.

The aim of the study by Wickberg et al. 192 was to compare the antenatal management of depression when midwives were aware of women’s EPDS scores with the management of depression when midwives were not aware of women’s EPDS scores. Women were recruited from antenatal care centres in six sectors of primary health-care districts in Sweden. Midwives were randomised to the intervention and control groups, rather than the individual women themselves. Additional training about depression was given to midwives in the intervention group. In total, 669 women took part in the study, 318 in the intervention group and 351 in the control group. Women completed the EPDS twice, first at gestational week 25 and second at gestational week 36. Those women scoring > 11 on the EPDS in the intervention group were phoned and asked for permission to inform their midwife about the EPDS score. Analysis showed that in gestational week 25, 48 (15%) of the 318 women in the intervention group and 45 (13%) of the 351 women in the control group scored above 11 on the EPDS, whereas in gestational week 36, 26 (10%) of the 273 women in the intervention group and 40 (12%) of the 345 women in the control group scored above the threshold. This difference was statistically significant (p < 0.0001). Women in the intervention group also displayed a reduction in the mean EPDS score from week 25 (6.41, range 0–25) to week 36 (5.39, range 0–19) (p < 0.001); however, the mean EPDS score in the control group remained almost the same (6.07, range 0–21, versus 6.11, range 0–22). None of the analyses accounted for the clustering by midwife.

A second study by Webster et al. 195 evaluated the effectiveness of a prenatal intervention in reducing the incidence of PND. Women attending their first prenatal appointment at a single hospital in Australia were screened for risk factors of PND [Maternity Social Support Scale (MSSS) < 25; past personal or family history of mental illness or PND]. Those women identified with elevated risk were randomised to the intervention (n = 299) or control group (n = 301). The intervention consisted of a booklet about PND with contact numbers, use of the EPDS, a discussion with the woman about her risk of developing PND and a letter to the woman’s GP and local child health nurse alerting them to the woman’s risk of PND (i.e. feedback of the risk). Women in the control group received standard care, which included case management and referral to a hospital social worker or psychiatrist if required. Women were asked to complete a second EPDS at 16 weeks postnatally. Of the 509 women sent the second EPDS, 73% (n = 371) responded. The number of women with an EPDS score of 12 or above was 24% (n = 46) in the intervention group and 28% (n = 50) in the control group (OR 0.80, 95% CI 0.50 to 1.28).

Another study194 investigated whether the provision of an early intervention by midwives decreased the incidence of depression at 6 weeks postnatally. The sample of women was recruited from a single teaching hospital in Hong Kong. Women in the intervention group were assessed with the EPDS three times during the postnatal period (1 day before discharge, 1 week postnatally and 6 weeks postnatally). The intervention included an in-depth assessment by midwives (women were asked to describe any sleep disturbance, poor appetite or self-blame or if they had ideas of hurting the baby), with a subsequent phone follow-up service by midwives and/or volunteers to the women with high EPDS scores (cut point 10). Training on basic counselling and interviewing skills and emotional problems of postnatal women was provided to midwives, social workers and volunteers before the study commenced. Women in the control group were not assessed with the EPDS until 6 weeks postnatally. In the intervention group 12 (5.9%) women had EPDS scores above 9 at 6 weeks postnatally compared with 24 (11.8%) women in the control group (p = 0.03). The mean EPDS score of the intervention group was 4.32 whereas that of the control group was 4.97; an exact p-value was not reported in the article – it was reported only that the p-value was not statistically significant.

The final study193 explored the effects of redesigning community postnatal care on women’s health 4 months postnatally. A total of 36 general practices from the West Midlands were cluster randomised using minimisation to the intervention or control group. General practices were randomised rather than women to prevent contamination. In the intervention group, care was led by midwives and included extended care to 28 days, use of a symptoms checklist and the EPDS at day 28, referral to a GP as necessary, and a 10- to 12-week discharge visit. The control group received routine care that included seven midwifery home visits until 10–14 days postnatally (although this could be extended to 28 days) and care by health visitors thereafter. GPs completed routine home visits and a final check-up at 6–8 weeks postnatally. Additional training in postnatal care and health and trial design was provided to all midwives. In total, 17 practices (1087 women) were randomised to the intervention group and 19 practices (977 women) were randomised to the control group. The mental health score was significantly better in the intervention group than in the control group (OR 3.03, 95% CI 1.53 to 4.52; OR_adj 4.31, 95% CI 2.50 to 6.12), as were the mean EPDS score (OR –1.92, 95% CI –2.55 to –1.29; OR_adj –2.68, 95% C –3.46 to –1.89) and the proportion of women with an EPDS score of 13 or above (OR 0.57, 95% CI 0.43 to 0.76; OR_adj 0.47, 95% CI 0.31 to 0.76). The physical health score did not differ between study groups (OR –0.79, 95% C –1.91 to 0.34; OR_adj –0.80, 95% C –2.32 to 0.72). Overall satisfaction with care from the community midwives did not differ statistically between groups (OR 1.09, 95% CI 0.72 to 1.63).

Level III evidence

In total, 25 studies were identified that provided level III evidence, nine of which were prevention studies (Table 26) and 16 of which were treatment studies (Table 27). There were a number of clinical and methodological differences between the studies. Most importantly for the focus of this review there were differences between the studies in terms of the identification strategy used during recruitment, the cut point chosen and the timing of administration of the identification strategy. The studies also differed with respect to the intervention used, the outcome assessed, the timing of the outcome measure and any subsequent follow-up measures. Because of the heterogeneity between studies included in the review it was not possible to combine the data to produce an overall pooled estimate; however, individual study estimates are still presented in forest plots to allow visual inspection of the data.

TABLE 26 - Prevention studies

BDI, Beck Depression Inventory; CCEI, Crown–Crisp Experimental Index; DAS, Dyadic Adjustment Scale; DSM, Diagnostic and Statistical Manual of Mental Disorders; EPDS, Edinburgh Postnatal Depression Scale; GHQ, General Health Questionnaire; ICD, International Classificaton of Diseases, Injuries and Causes of Death; MASQ, Modified Antenatal Screening Questionnaire; SCL-90-R, Symptom Checklist 90-Revised.

Study	Intervention	Identification strategy (cut point), timing	Threshold depression	Continuous depression
Brugha et al., 2000196	Preparing for Parenthood – structured antenatal risk factor-reducing intervention designed to increase social support and problem-solving skills	Pregnancy and You plus any one of six depression items on GHQ-D 12–20 weeks’ gestation	EPDS, GHQ, ICD-10 (3 months)
Buist et al., 1999197	Intervention group classes	Eight or more risk factors 12–24 weeks’ gestation		EPDS, BDI (6 and 24 weeks)
Elliot et al., 1988198	Psychoeducation and social support in a programme with continuity of care	Leverton questionnaire: poor marital relationship or previous psychological problems or score of more than 10 on the CCEI anxiety items Pregnancy (within 18 weeks’ gestation)	Bedford College (3 months)	EPDS, CCEI-depression, CCEI-anxiety (3 months)
Elliot et al., 2000199	Social support, education, continuity of care	Scored 2 on any one of the vulnerability questions in Leverton questionnaire or scored 1 on more than one question or scored ≥ 10 on the CCEI anxiety subscale During pregnancy	Bedford College (3 months)	EPDS (3 months)
Gorman, 2001200	Interpersonal therapy	Definite personal history of major depressive disorder with or without treatment; BDI (13); first-degree relative with a history of psychiatric treatment; DAS < 100; experiencing two or more life events predicted to have moderate or severe negative impact for the woman in the postnatal period Third trimester	DSM-III-R (1 and 6 months)	BDI, SCL-90-R, EPDS (1 and 6 months)
McKee et al., 2006;201 Zayas et al., 2004202	Cognitive behavioural therapy + child development psychoeducational modules + social support building sessions	BDI (14) Within 32 weeks’ gestation		BDI-II (2 and 12 weeks)
Stamp et al., 1995203	Support groups	MASQ (2) 24 weeks’ gestation	EPDS (6, 12 and 24 weeks)
Zlotnick et al., 2001204	Interpersonal therapy	At least one risk factor: previous history of depression; poor social support; BDI > 10; recent stressful event 20–32 weeks’ gestation	DSM-IV (3 months)	BDI (3 months)
Zlotnick et al., 2006205	Interpersonal therapy	Predictive Index (27) 23–32 weeks’ gestation	Longitudinal interval follow-up interview (3 months)	BDI (3 months)

TABLE 27 - Treatment studies

BDI, Beck Depression Inventory; DASS, Depression Anxiety Stress Scale; DSM, Diagnostic and Statistical Manual of Mental Disorders; EPDS, Edinburgh Postnatal Depression Scale; GHQ, General Health Questionnaire; HAMA, Hamilton Rating Scale for Anxiety; ; HAMD, Hamilton Rating Scale for Depression; HRSD, Hamilton Rating Scale for Depression; MADRS, Montgomery–Asberg Depression Rating Scale; PND, postnatal depression; POMS, Profile of Mood States; RDC, Research Diagnostic Criteria.

Study	Intervention	Identification strategy (cut point), timing	Threshold depression	Continuous depression
Armstrong and Edwards, 2003206	Exercise and social support	EPDS (12) 6 weeks to 12 months postnatally	EPDS (6 and 12 weeks)	EPDS, DASS, GHQ (6 and 12 weeks)
Chabrol et al., 2002215	Cognitive behavioural therapy	EPDS (9) 2–5 days postnatally	HAMD, BDI, EPDS (10–12 weeks)	HAMD, BDI, EPDS (10–12 weeks)
Chen et al., 2000214	Support groups	BDI (10) 3 weeks postnatally	BDI (10–14 weeks)	BDI (10–14 weeks)
Cooper et al., 2003;216 Murray et al., 2003217	Cognitive behavioural therapy, psychodynamic therapy, non-directive counselling	EPDS (12) Early postnatally	DSM-III-R (4.5 and 9 months)	EPDS (4.5 and 9 months)
Dennis, 2003207	Peer support	EPDS (10) 8 weeks postnatally	EPDS (4 and 8 weeks)	EPDS (4 and 8 weeks)
Holden et al., 1989218	Counselling	EPDS (12) 6 weeks postnatally	RDC (3 months)	EPDS, psychiatric interview (3 months)
Heh and Fu, 2003208	Informational support – posted PND booklet	EPDS (10) 4 weeks postnatally	EPDS (3 months)	EPDS (3 months)
Honey et al., 2002209	Psychoeducational group intervention: cognitive behavioural therapy + education + relaxation	EPDS (13) Postnatally	EPDS (8 weeks and 6 months)	EPDS (8 weeks and 6 months)
Horowitz et al., 2001221	Coached behavioural intervention to promote maternal–infant responsiveness	EPDS (10) 2–4 weeks postnatally		BDI-II (10–14 weeks and 14–18 weeks)
Ingadottir and Thome, 2006210	Web-based course for health professionals	EPDS (12) 9 weeks postnatally	EPDS (15 and 24 weeks)	EPDS (15 and 24 weeks)
Meager and Milgrom, 1996220	Education, social support and cognitive behavioural therapy	EPDS (13), BDI (16) 6 months postnatally		BDI, EPDS, POMS (10 weeks)
Milgrom et al., 2005222	Cognitive behavioural therapy or counselling	DSM-IV diagnosis 6–18 weeks postnatally		BDI (3 months)
Misri et al., 2004211	Cognitive behavioural therapy	HAMD (18), HAMA (20), EPDS (12) Postnatally	HAMD, HAMA, EPDS (3 months)	HAMD, HAMA, EPDS (3 months)
O’Hara et al., 2000213	Interpersonal therapy	Inventory to diagnose depression, DSM-IV, HAMD (12) Postnatally	HRSD, BDI (4, 8 and 12 weeks)	HAMD, BDI (4, 8 and 12 weeks)
Prendergast and Austin, 2001212	Cognitive behavioural therapy	EPDS (13) Mean = 15 weeks postnatally	EPDS (6 weeks and 6 months)	EPDS (6 weeks and 6 months), MADRS (6 weeks)
Wickberg and Hwang, 1996219	Counselling	EPDS (12) 2 months postnatally	DSM-III-R (6 weeks)	MADRS (6 weeks)

For the prevention studies a variety of identification strategies were used: BDI (n = 3), Crown–Crisp Experimental Index (CCEI) (n = 2), EPDS (n = 1), GHQ (n = 1), Modified Antenatal Screening Questionnaire (MASQ) (n = 1), PI (n = 1) and risk factors (n = 1). The identification strategies were administered at various time points during pregnancy, ranging from within 18 weeks’ gestation to 32 weeks’ gestation. Forest plots for dichotomous and continuous outcomes are displayed in Figures 30 and 31 respectively. In the forest plots the studies are grouped by the identification strategy used during recruitment and ordered within these groups by the cut point used, which is reported in parentheses after the publication details. All but one of the prevention studies reporting dichotomous outcomes reported the number of women with depression as defined by a diagnostic interview;196,198–200,204,205 the final study203 used the EPDS at a cut point of 13 to classify women as being depressed or not. Interestingly, this last study is the study located on the opposite side of the forest plot to all of the other studies. We can clearly see that there is wide variability across the results in these studies.

FIGURE 30.

Forest plot of prevention studies reporting various dichotomous outcomes.

FIGURE 31.

Forest plot of prevention studies reporting various continuous outcomes.

For the continuous outcomes four197–200 of the studies reported mean EPDS scores, with the remaining three201,204,205 studies reporting mean BDI scores. Two197,198 of the studies reporting mean EPDS scores did not report measures of dispersion and hence could not be displayed in the forest plot. We can clearly see that there is wide variability across the results in these studies.

In total, 13 of the treatment studies used the EPDS (alone or in combination with other strategies) during the recruitment stage of the study. The studies used different cut points on the EPDS to distinguish between those women who were at risk of having PND and those who were not. Cut points of 9, 10, 12 and 13 were used. Using different cut points leads to different types of women being included in the studies. The remaining treatment studies used generic depression identification strategies (BDI or HRSD) or diagnostic interviews. Identification strategies were administered at various time points during the first year postnatally, ranging from 2–5 days postnatally to 12 months postnatally; however, the majority of women in the studies were administered the identification strategies within 6 months postnatally. Forest plots for dichotomous and continuous outcomes are displayed in Figures 32 and 33 respectively. Seven206–212 of the treatment studies reported dichotomous outcomes as the number of women with depression as defined by the number of women scoring about a cut point on the EPDS, three213–215 studies used the BDI and the final three216–219 studies used diagnostic interviews. For the continuous outcomes, 11 of the studies reported mean EPDS scores,206–212,215–218,220 with the remaining four studies reporting mean BDI scores. 213,214,221,222 Three of the studies reporting mean EPDS scores210,218,220 and two reporting mean BDI scores214,222 did not report measures of dispersion and hence could not be displayed on the forest plot. We can clearly see that there is wide variability across the results for both dichotomous and continuous outcomes

FIGURE 32.

Forest plot of treatment studies reporting various dichotomous outcomes.

FIGURE 33.

Forest plot of treatment studies reporting various continuous outcomes.

Discussion

Five studies were identified that compared the use of an identification strategy with or without enhancement of care or feedback of scores with not using an identification strategy or usual care (i.e. providing level I or II evidence). As identified in previous chapters, the EPDS was the most frequently used identification strategy. All of the studies indicated beneficial effects of using the EPDS in reducing EPDS scores, although some of the individual studies did not show statistically significant differences. Studies reporting dichotomous outcomes were combined and the pooled estimate gave an OR of 0.64 (95% CI 0.52 to 0.78). Thus, the odds of scoring above the threshold for depression in a population in which a formal method was used to identify PND (with an intervention for those identified) were 0.64 times the odds of scoring above the threshold for depression in a population in which there was no formal method to identify PND (with an intervention for those identified). Problems occurred when attempts were made to combine studies reporting continuous outcomes as measures of dispersion were often not reported. As three out of the five studies also included some form of enhancement of care it is impossible to disentangle the effect of the identification strategy component alone. For example, it could be that the enhancement of care is the important factor and use of the identification strategy has little impact.

A total of 25 studies were identified that reported using identification strategies at the recruitment stage to identify women at risk of PND with the aim of examining the impact of various interventions compared with usual care (i.e. level III evidence). Despite a large number of studies being identified there were a number of clinical and methodological differences between the studies, which did not permit statistical pooling of results. Furthermore, it was hard to distinguish between the benefit of using identification strategies and the effects of the intervention under study. Thus, it was difficult to draw conclusions about the impact of using identification strategies on maternal and infant outcomes. Further research would be informative in this area.

There were a number of methodological weaknesses of the studies included in this review. We included controlled trials as well as randomised trials and, for example, in some of the studies odd or even expected dates of delivery were used to randomise participants to treatment groups. Obviously, such methods of randomisation are not truly random and thus, in practice, this often results in a selection bias being introduced. In addition, in some of the studies described as RCTs it was often unclear how the randomisation sequence was generated and what methods were used to conceal the sequence. Hence, it was difficult to judge whether the methods used to randomise participants to treatment groups were subject to bias. In some studies randomisation was undertaken on a cluster rather than on an individual basis. When data are ordered in clusters, individuals may not be independent of one another and subsequently this needs to be taken into account in the analysis. In two out of the three studies using cluster randomisation the analyses did not account for the clustering. When clustering is not accounted for in the analysis it results in p-values and standard errors (SEs) that are too low and confidence intervals that are too narrow. Another frequently occurring problem was attrition, with most studies reporting that some women were lost to follow-up. Non-compliance or non-attendance was also reported across studies. It was often unclear whether analyses were undertaken using an intention to treat approach or not. An intention to treat approach should be used as the primary analysis when non-compliance occurs as this ensures that the initial randomisation sequence is preserved.

Reflection on current policy and practice within the UK

As outlined in Chapter 1, current NICE guidance recommends the use of two questions to identify possible depression and a third question if the women answers ‘yes’ to either of the initial questions [(1) ‘During the past month, have you often been bothered by feeling down, depressed or hopeless?’, (2) ‘During the past month, have you often been bothered by little interest or pleasure in doing things?’ and (3) ‘Is this something you feel you need or want help with?’]. No studies were identified in this review that assessed the clinical effectiveness of these questions in improving maternal and infant outcomes in a postnatal population.

Methods

Results

The additional searches for full economic evaluations yielded 3667 articles. On the basis of the title and abstract two studies from the original searches and two studies from the economic searches appeared to be potentially eligible for inclusion. On closer inspection none of the studies were a full economic evaluation of a PND identification strategy. The reasons for exclusion from the review are outlined in Table 28.

Discussion

Despite an extensive systematic search of the literature none of the studies identified presented a full economic evaluation of a PND identification strategy. Future research needs to address this gap in the literature. We are aware of one large, but yet unpublished, RCT with an economic evaluation (PoNDER trial). The PoNDER trial aimed to assess the costs and effectiveness and broad impact of two health visitor psychological interventions for PND.

Reflection on current policy and practice within the UK

As outlined in Chapter 1, recent NICE guidance recommends the use of case-finding questions to help identify women with PND [(1) ‘During the past month, have you often been bothered by feeling down, depressed or hopeless?’, (2) ‘During the past month, have you often been bothered by little interest or pleasure in doing things?’ and (3) ‘Is this something you feel you need or want help with?’]. No studies assessing the cost-effectiveness of these questions (or any other method to identify PND) were identified.

Key concepts in cost-effectiveness analyses of identification strategies

To inform the development of a decision-analytic model it is important to establish the key features of cost-effectiveness analyses for the purpose of informing resource allocation in the NHS:

1. The specification of the decision problem should ideally include the comparison of all identification strategies that could feasibly be used in the NHS. It is recognised, however, that in practice these options may be constrained by the availability of evidence and the structural complexity of any model.

2. The analysis should make a clear link between the diagnostic accuracy of a given identification strategy, the impact on subsequent treatment decisions and the ultimate effect on health outcomes and costs. Hence, the costs and outcomes of each of the four diagnosis groups – true positive, false negative, true negative and false positive – need to be assessed.

3. The ultimate health effects of the alternative identification strategies should be expressed in terms of a generic measure of health such as a quality-adjusted life-year (QALY). This is because it is necessary to assess the value of improved outcomes from more accurate identification strategies in units that can be compared with those of programmes and interventions in other specialties and disease areas that are competing for finite health-care resources.

4. The evidence used to estimate cost-effectiveness should be relevant to patients and clinical practice in the UK health service.

5. The uncertainty in the evidence base needs to be reflected in the model. To simultaneously assess the implications of uncertainty in all elements of evidence, probabilistic analysis should be used to establish the decision uncertainty associated with each diagnostic strategy being compared. 228,229 This informs decision-makers about the probability of each strategy being the most cost-effective conditional on the value that the decision-maker places on a unit of health gain. Such methods can also be used to provide an opportunity to apply value of information (VOI) methods to inform priority setting in research. 43,230,231

Methods

The objective of the model was to estimate, based on best available data, the costs and health outcomes for a range of feasible identification strategies. The analysis was conducted from an NHS/personal social services (PSS) perspective, with costs expressed in 2006/7 prices and health outcomes expressed in terms of QALYs. The time horizon of the analysis was 1 year; as such, no discounting of costs and health outcomes was applied.

The model was made up of two parts including: (1) an identification model reflecting the diagnostic performance and administration costs of the alternative identification strategies; and (2) a treatment model that evaluated the subsequent costs and outcomes (expressed in QALYs) of each of the four diagnosis groups – true positive, false negative, true negative and false positive.

The model was probabilistic in that input parameters were entered into the model as probability distributions to reflect second order uncertainty – that is, uncertainty surrounding the mean estimates. 229 Monte Carlo simulation was used to propagate uncertainty in input parameters through the model in such a way that the results of the analysis could also be presented with their uncertainty. The probabilistic analysis also provided a formal approach to quantifying the consequences associated with the uncertainty surrounding the model results and could be used to identify priorities for future research. Scenario analysis was used to explore the impact of alternative assumptions in the model.

The following sections outline the decision problem and the structure of the model and also provide an overview of the key assumptions and data sources used to populate the model in more detail. A prerequisite of the treatment model was ensuring that subsequent treatments for PND (i.e. for women with a true positive diagnosis) were, in themselves, cost-effective. Given the importance of this aspect and as this issue was, in part at least, separable from the broader question of the cost-effectiveness of identification strategies, the cost-effectiveness of alternative treatment strategies for the management of women with PND was considered first. This analysis was used to identify the optimal treatment (according to cost-effectiveness considerations) that was assigned to women with PND as part of the broader modelling work.

Establishing the cost-effectiveness of treatment for women with postnatal depression

Overview

For any PND identification strategy to be considered cost-effective, a cost-effective treatment strategy must exist for those women with PND correctly diagnosed as depressed. The recent NICE guidance on antenatal and postnatal mental health identified structured psychological therapy and listening home visits as the most suitable first-line treatments for depression in the postnatal period. 30 A deterministic economic model conducted as part of the NICE guidance found listening home visits to be both more effective and more costly than structured psychological therapy. The incremental cost-effectiveness ratio (ICER) of listening home visits versus structured psychological therapy was reported to be £9435 per QALY. However, neither treatment strategy was compared with usual care (i.e. the management of PND without a formal policy of listening home visits or psychological therapy); hence, it was not clear from the NICE guidance whether either treatment strategy was cost-effective compared with usual care. To clarify this the NICE treatment model was reconstructed with a usual care arm; the opportunity was also taken to revise some of the parameter values used and to reflect uncertainty in these values using probabilistic methods.

Model structure and key structural assumptions

The reconstructed treatment model took the form of a decision tree and a schematic is given in Figure 34. The schematic was adapted from the one presented in the NICE guidance and included a ‘usual care’ alternative. For a detailed description of the model, reference is made to the NICE guidance;30 the following provides a very brief summary of the model structure and key structural assumptions.

FIGURE 34.

Schematic of the treatment model (adapted from NICE guideline30).

At the model start point a woman with PND was provided with one of the two treatments (with corresponding ‘additional care’, as described in the NICE guidance) or with usual care. If a treatment was provided the woman could continue or discontinue treatment, whereas it was assumed that the woman could not discontinue usual care. The woman then either responded to the treatment (or improved under usual care) or did not respond. If the woman responded or improved then it was assumed that she underwent a linear improvement in health-related quality of life (HRQoL), evaluated in terms of QALYs, from the model start point to week 8. If the woman then relapsed within the time horizon of the model it was assumed that she underwent a linear deterioration in HRQoL from week 8 to 1 year (the model end point), otherwise she remained non-depressed until the end point. If the woman did not respond to treatment or improve under usual care it was assumed that she remained depressed until the model end point.

Parameter inputs for the treatment model

The parameter values incorporated into the reconstructed NICE treatment model are reported in Appendix 6. A brief summary of the approaches used to inform the relevant parameters are reported in the following sections.

Risk of discontinuing treatment

The NICE estimate for the absolute risk of discontinuation for usual care was calculated by summing over the control arms of all psychological treatment studies given in the relevant guideline meta-analysis (01.04). A revised value for the absolute risk of discontinuation for usual care was calculated by carrying out a meta-analysis over the same control arms. As in the NICE guidance,30 the absolute risk of discontinuation for each treatment was calculated by multiplying the absolute risk of discontinuation for usual care by the relative risk of discontinuation for the respective treatment and then subtracting the absolute risk of discontinuation for usual care. The justification for this was given in the NICE guidance (p. 179), and the calculation remained unchanged here.

Risk of no response/improvement

The NICE estimate for the absolute risk of no improvement for usual care was calculated by summing over the control arms of all psychological treatment studies given in the relevant guideline meta-analysis. A revised estimate for the absolute risk of no improvement for usual care was calculated by undertaking a random-effects meta-analysis over the same control arms. As in the NICE guidance,30 the absolute risk of no response from each treatment was calculated by multiplying the absolute risk of no improvement from usual care by the relative risk of no response from the respective treatment (see NICE guidance, p. 179).

Risk of relapse

The NICE estimate for the risk of relapse was assumed to be common for women who responded to treatment and those who improved under usual care and was calculated by summing over the treatment arms of the studies included in the relevant meta-analysis (08.04.06.03) given in Appendix 19b of the NICE guideline on depression. 89 The estimate of the risk of relapse for women who improved under usual care was revised by carrying out a random-effects meta-analysis over the control arms of these studies, whereas a revised risk of relapse for women who responded to treatment was calculated by multiplying the revised risk of relapse for those improving under usual care by the estimate of the relative risk of relapse given in the original meta-analysis.

Resource utilisation and cost inputs

The estimates of unit costs adopted by NICE were taken from Curtis and Netten;232 these were updated by estimates taken from Curtis. 233 These were combined with the assumptions employed in the NICE guideline to estimate the total cost of each strategy reported in Appendix 6.

Utility weights

The utility weights used in the reconstructed NICE treatment model remained unchanged from the NICE guidance, with the exception that a beta-distribution was fitted to each parameter; the SE in each case was calculated from the respective standard distribution and sample size given in Revicki and Wood. 234

Results of the treatment model

The probabilistic treatment model was run over 1000 simulations; the results are given in Table 29. In common with the NICE guideline model, listening home visits were found to be both more effective and more costly than structured psychological therapy; however, the respective ICER of £66,275 per additional QALY suggested that listening home visits were not a cost-effective alternative to structured psychological therapy based on conventional thresholds, representing a decision-maker’s willingness to pay (WTP) for additional health outcome, applied to establish cost-effectiveness (typically in the region of £20,000–30,000 per QALY). 235 The ICER of £17,481 per QALY for structured psychological therapy versus usual care suggested that structured psychological therapy was a cost-effective treatment based on similar thresholds.

TABLE 29 - Results of the reconstructed NICE treatment model

CE, cost-effective; ICER, incremental cost-effectiveness ratio; N/A, not applicable; QALYs, quality-adjusted life-years, WTP, willingness to pay.

All stated costs are incremental to those of usual care.

The probability that a strategy was more CE than all others given a particular maximum WTP for an additional QALY.

Identification option	QALYs	Costa	ICER	Prob. CE for max. WTPb
				£20,000	£30,000	£40,000
Usual care	0.7036	£0.00	N/A	0.2200	0.0370	0.0060
Structured psychological therapy	0.7489	£792.10	£17,481	0.5040	0.5490	0.5360
Listening home visits	0.7513	£946.48	£66,275	0.2760	0.4140	0.4580

For each of the three WTP thresholds given, structured psychological therapy had a greater than 50% probability of being the cost-effective strategy; for a WTP threshold of £20,000 per QALY, listening home visits were only slightly more likely to be cost-effective than the strategy of usual care.

Implications of the treatment model

Structured psychological therapy was found to be a cost-effective treatment for women with PND. Although listening home visits were slightly more effective, the higher cost of this strategy resulted in an ICER of listening home visits versus structured psychological therapy that was well above conventional cost-effectiveness thresholds considered to represent value for money in the NHS. As such, in constructing the identification model it was assumed that structured psychological therapy would be provided as the first-line treatment to all postnatal women identified with depression.

The identification model

Overview

The identification model took the form of a decision tree and comprised five components – a ‘diagnostic’ component and four mutually exclusive ‘treatment’ components – representing the pathways of care, costs and outcomes for (1) true-positive, (2) false-negative, (3) true-negative and (4) false-positive cases. The identification model was used to evaluate the costs and outcomes of a range of alternative formal identification strategies compared with current practice (i.e. opportunistic case finding identified as part of routine consultations with a GP and/or health visitor). It was assumed that each postnatal woman entered the diagnostic component of the model at 6 weeks postnatally, at which time a formal identification strategy may be employed to complement any diagnosis of PND by the woman’s GP or health visitor. Depending upon the woman’s actual state of depression and her diagnosis she was then assigned to one of the four treatment components and followed up for 52 weeks until the model end point at 58 weeks postnatally.

To simplify the detailed description of the model each of the five model components is discussed separately. Before this an overview of the strategies considered in the model is provided.

Strategies evaluated

As previously stated the decision problem should ideally include the comparison of all identification strategies that could feasibly be used in the NHS. However, in practice, these options may be constrained by the availability of evidence and the structural complexity of any model. The review of the validity of methods to identify PND, reported in Chapter 5, was used to inform the identification strategies considered in the economic analysis and provided the basis for selecting strategies.

Strategies were only considered in the economic analysis if there were sufficient data reported to be able to combine results and produce pooled summary estimates of sensitivity and specificity based on evaluations undertaken postnatally. In Chapter 5, studies were separately examined according to disease classification (major depression only, major or minor depression, any psychiatric disorder or other). The economic model focused on those classifications for which the data appeared most widely reported – major depression alone (DSM or equivalent) and major/minor depression. Given the wider range of instruments and cut points reported for major/minor depression, this classification (and associated diagnostic data) was used to inform the base-case economic analysis. This classification was also consistent with the population considered in the treatment model reported previously. Major depression alone was considered as part of the sensitivity analysis.

For the comparison of major or minor depression, results from the bivariate meta-analysis of sensitivity and specificity were reported for a range of instruments and cut points. Specifically, the bivariate analysis considered the EPDS (cut points 7–16), BDI (cut point 10) and HAMD (cut point 11). However, as a single study provided the data to pool for the HAMD (with an associated I² value of 0), the evidence for this strategy was not considered sufficiently robust for the purposes of the economic analysis. Therefore, the main strategies considered in the base-case analysis were the EPDS (cut points 7–16) and BDI (cut point 10) compared with current practice (i.e. routine case identification without the formal use of a diagnostic instrument). The different EPDS cut points were assessed as separate strategies (10 in total), which enabled an evaluation of the trade-off between the different sensitivity and specificity values to be considered in terms of cost-effectiveness. All case identification strategies in the base-case model were modelled as one-time screens, such that the readministration of the same instrument was not modelled.

In addition to the base-case analysis, separate scenarios were considered that explored a range of alternative approaches. These are discussed more fully in later sections. The alternative approaches considered: (1) alternative classifications (i.e. considering major depression only); (2) alternative identification strategies (including a separate scenario including the Whooley questions and a scenario in which a separate confirmatory strategy was used employing a gold standard instrument; and (3) alternative costs associated with the management of false positives.

The diagnostic component

Model structure and key assumptions

The diagnostic component is shown in Figure 35 in the form of a decision tree. All postnatal women (whether depressed or non-depressed) entered the diagnostic component of the model at 6 weeks postnatally. It was assumed that at this time those women who were depressed may have been identified via routine care as being depressed, whereas those women who were not depressed were not incorrectly identified as being depressed as part of routine care.

FIGURE 35.

The diagnostic component.

If no formal identification strategy was employed then depressed women positively identified via routine care entered the ‘true-positive’ treatment component of the model; depressed women not identified by routine care entered the ‘false-negative’ treatment component, whereas women who were non-depressed (and so not diagnosed) entered the ‘true-negative’ treatment component.

If a formal identification strategy such as the EPDS was employed then it was assumed that the EPDS was administered to all postnatal women at 6 weeks postnatally. Women were assumed to be diagnosed as depressed if they were positively identified by either the formal identification strategy or routine care. Hence, the value of identification strategies was determined not by the overall probability of identifying women with PND, but rather by the difference between this probability and the probability of identifying women with PND by routine care alone. Depressed women correctly diagnosed as depressed entered the ‘true-positive’ treatment component of the model; depressed women not diagnosed as depressed entered the ‘false-negative’ treatment component; non-depressed women not diagnosed as depressed entered the ‘true-negative’ treatment component; and non-depressed women wrongly diagnosed as depressed entered the ‘false-positive’ treatment component. Note that the terms ‘true positive’, ‘false negative’, etc. were used to relate a woman’s overall diagnosis (including any identification via routine care) to her actual state of depression; the terms should not be conflated with the diagnostic accuracy results of the formal identification strategies.

An important structural assumption was that all postnatal women entered the relevant treatment component at 6 weeks postnatally (immediately following the identification strategy when given). It was therefore assumed that, in cases in which depression was diagnosed by routine care earlier in the postnatal period, no treatment was administered until 6 weeks postnatally and only then if the depression persisted at this time.

Parameter inputs

The parameter inputs incorporated into the diagnostic component are given in Tables 30–32. The estimate of the prevalence of major and minor depression among postnatal women at 6 weeks postnatally (11.3%) was taken from Gaynes et al. 27 for the end date ‘2 months PP’. It was assumed that the parameter was normally distributed, with the SE calculated from the given 95% CI (7.7% to 16.2%).

TABLE 30 - Input parameters – prevalence and routine case identification

SE, standard error.

Input parameter	Mean	SE	Distribution	Source
Prevalence of major or minor depression	0.1130	0.0221	Normal	Gaynes et al., 200527
Probability of identification of depression at 6 weeks postnatally via routine care	0.3864	0.0516	Beta	Kessler et al., 2002236

TABLE 31 - Performance of diagnostic tests

BDI, Beck Depression Inventory; EPDS, Edinburgh Postnatal Depression.

Diagnostic test	Sensitivity (major/minor depression)				Specificity (major/minor depression)				Co-variance	Source
	Log-odds	Variance	Distribution	Mean	Log-odds	Variance	Distribution	Mean
EPDS (cut point 7)	2.3346	1.7439	Normal	0.9117	0.7075	0.4125	Normal	0.6699	–0.1164	Bivariate meta-analysis (Chapter 5)
EPDS (cut point 8)	2.3379	0.9962	Normal	0.9120	1.0744	0.7713	Normal	0.7454	0.1702
EPDS (cut point 9)	1.7565	1.0652	Normal	0.8528	1.5274	0.8888	Normal	0.8216	0.0457
EPDS (cut point 10)	1.4962	0.3514	Normal	0.8170	1.8373	1.3740	Normal	0.8626	–0.0524
EPDS (cut point 11)	0.9548	0.5324	Normal	0.7221	2.3261	1.6619	Normal	0.9110	–0.3808
EPDS (cut point 12)	0.7563	0.2095	Normal	0.6805	2.5959	1.4008	Normal	0.9306	–0.2222
EPDS (cut point 13)	0.6716	0.6313	Normal	0.6619	2.5431	0.7443	Normal	0.9271	–0.3553
EPDS (cut point 14)	0.1326	0.0008	Normal	0.5331	3.1069	1.1104	Normal	0.9572	–0.0282
EPDS (cut point 15)	–0.4431	0.0301	Normal	0.3910	3.7755	0.2873	Normal	0.9776	0.0930
EPDS (cut point 16)	–0.7875	0.3989	Normal	0.3127	4.5233	0.1812	Normal	0.9893	0.2689
BDI (cut point 10)	0.9408	0.4659	Normal	0.7193	2.2675	0.4583	Normal	0.9061	0.4621
Diagnostic test	Sensitivity (major/minor depression)				Specificity (major/minor depression)				Co-variance	Source
	Log-odds	Variance	Distribution	Mean	Log-odds	Variance	Distribution	Mean
EPDS (cut point 7)	3.1781	0.0000	Normal	0.9600	–0.1879	0.7037	Normal	0.4532	–0.0001	Bivariate meta-analysis (Chapter 5)
EPDS (cut point 8)	2.7487	0.0263	Normal	0.9398	0.3168	0.4941	Normal	0.5786	0.1139
EPDS (cut point 9)	2.4998	0.0574	Normal	0.9241	0.6582	0.4766	Normal	0.6589	0.1653
EPDS (cut point 10)	2.4026	0.1657	Normal	0.9170	1.2129	0.4529	Normal	0.7708	0.2740
EPDS (cut point 11)	1.9174	0.1906	Normal	0.8719	1.6353	0.6600	Normal	0.8369	0.1095
EPDS (cut point 12)	1.7886	0.0580	Normal	0.8568	1.9138	0.9112	Normal	0.8714	–0.2299
EPDS (cut point 13)	1.3385	0.0040	Normal	0.7922	2.1361	0.5333	Normal	0.8944	–0.0395
EPDS (cut point 14)	0.9766	0.0301	Normal	0.7264	2.4369	0.6733	Normal	0.9196	0.0692
EPDS (cut point 15)	0.6283	0.0350	Normal	0.6521	3.0953	0.6562	Normal	0.9567	–0.0963
EPDS (cut point 16)	0.3863	0.0110	Normal	0.5954	3.6035	1.1603	Normal	0.9735	–0.1130

TABLE 32 - Cost inputs

BDI, Beck Depression Inventory; EPDS, Edinburgh Postnatal Depression.

Cost element	Value	Distribution	Source
BDI license fee (per test)	US$2.00	Fixed	www.harcourtassessment.com (accessed 1 April 2008)
Health visitor (per hour)	£91.00	Fixed	Curtis, 2007233
Exchange rate	Value	Distribution	Source
US$ to UK£	£0.51	Fixed	www.FT.com (accessed 1 April 2008)
Identification strategy	Resources used (per test)		Source
EPDS	Health visitor (5 minutes)		Cox et al., 198716
BDI	Health visitor (5 minutes)		www.harcourtassessment.com (accessed 1 April 2008)
	License fee
Total cost (per test)	Value	Distribution	Source
EPDS	£7.57	Fixed	Estimated from above inputs
BDI	£8.59	Fixed

In the absence of estimates of the probability that PND was identified via routine care from a study reporting results in a PND population, the estimate of the probability that PND was detected via routine care at 6 weeks postnatally was derived from a study of subjects with depression or anxiety by Kessler et al. 236 This study found that 34 of 88 patients with depression or anxiety attending a general practice in Bristol in 1997 were detected by the GP at baseline, with 22 of the remaining 54 detected by the GP during 3 years of follow-up (this latter finding was returned to in the sensitivity analysis). Taking the baseline to be 6 weeks postnatally, the probability that PND was detected by a GP at this time was assumed to take a beta (34,54) distribution with a mean of 34/88.

The sensitivity and specificity of the alternative identification strategies were derived from the bivariate meta-analysis reported in Chapter 5. For the purposes of the probabilistic model the sensitivity and specificity were modelled on the log-odds scale using normal distributions. The correlation between sensitivity and specificity, which was integral to the bivariate approach, was reflected in the probabilistic analysis using the Cholesky decomposition of the covariance matrix. The input distributions are reported in Table 31.

It was assumed that all postnatal women, whether depressed or non-depressed, made the same number of visits to a GP and received the same number of visits by a health visitor during the first 6 weeks postnatally. As such, any costs associated with these visits ‘net out’ of the analysis and so were not considered. The only costs relevant to the diagnostic component were those of administering the identification strategy (when applicable).

It was also assumed that all identification strategies were administered via a health visitor as part of a routine visit. As such, only the marginal cost of administration was considered (i.e. the additional time taken to administer an instrument over and above that required for a routine visit). The EPDS and BDI were assumed to require 5 additional minutes of the health visitor’s time. In addition, the cost of the license fee of US$2 per test (£1.02 at current exchange rates in 2008) for the use of the BDI was included. The total costs of administrating each test are reported in Table 32.

The ‘true-positive’ treatment component

Model structure and key assumptions

The ‘true-positive’ treatment component is shown in Figure 36 in the form of a decision tree. All women who entered this treatment component were depressed and were correctly diagnosed as such by routine care and/or a formal identification strategy (when applicable).

FIGURE 36.

The ‘true-positive’ treatment component.

The structure of this component was adapted from that of the reconstructed NICE treatment model discussed earlier. As it was assumed that all postnatal women diagnosed with depression were offered structured psychological therapy, the usual care and listening home visits arms of the reconstructed NICE treatment model were irrelevant and so were not considered. All other structural assumptions given in the reconstructed NICE treatment model remained.

Parameter inputs

The parameter inputs for this component were those used in the reconstructed NICE treatment model reported in Appendix 6.

The ‘false-positive’ treatment component

Model structure and key assumptions

The ‘false-positive’ treatment component is given in Figure 37. All women who entered this treatment component were non-depressed, but were wrongly diagnosed as having depression by the identification strategy. It was assumed in the diagnostic component that non-depressed women were not incorrectly diagnosed as depressed as part of routine care. Hence, the false-positive costs were only considered for the formal identification strategies. This could be considered a conservative approach towards evaluating the use of formal identification approaches as it is possible that some of the women identified as false positives via these approaches would also have been incorrectly diagnosed as part of routine care and, as such, some of the costs attributed to the identification strategies would have also been incurred as part of routine care. However, equally it could be argued that the process of routine case detection is potentially independent from the process of formal identification strategies and that the issue of false positives associated with routine care could affect all strategies in the same way.

FIGURE 37.

The ‘false-positive’ treatment component.

In common with the ‘true-positive’ component it was assumed that all women were initially offered structured psychological therapy with the preceding ‘additional care’. However, it was assumed that the true non-depressed state of each woman would become apparent during the ‘additional care’ phase and as a result structured psychological therapy itself would not be provided. Those women wrongly diagnosed as depressed by the screen therefore incurred only the cost of ‘additional care’ (this assumption was returned to in the sensitivity analysis) and not the costs of psychological therapy. Furthermore, it was assumed that incorrectly identifying a non-depressed woman as depressed carried a loss in quality of life (i.e. no decrement in utility was assigned in the model) and that all women non-depressed at 6 weeks postnatally remained non-depressed until the model end point; as such, all women in this component experienced the HRQoL associated with remission throughout the follow-up period. Although in reality women who were not suffering PND at 6 weeks postnatally may become depressed at some point in the following year, the evaluation of identification strategies was based on the 6-week time point and hence has no impact on the subsequent management of women who were non-depressed at 6 weeks (aside from the costs of false positives) during the remaining course of the model. Consequently, this assumption had no impact on the incremental cost-effectiveness estimates reported here.

Parameter inputs

The only parameter inputs for this component of the model were the cost of additional care and the utility value for the state of remission (see Appendix 6).

The ‘false-negative’ treatment component

Model structure and key assumptions

The ‘false-negative’ treatment component is given in Figure 38. All women who entered this treatment component were depressed, but had not been identified either by routine care or by the formal identification strategies.

FIGURE 38.

The ‘false-negative’ treatment component.

As the depression remained unidentified, each woman initially received usual care for non-depressed postnatal women. As in the reconstructed NICE treatment model there was a possibility that the woman’s depression might improve under usual care by 14 weeks postnatally (8 weeks after the model start point), in which case the woman either remained in a state of remission or relapsed over the remaining 44-week follow-up period.

If the woman did not improve under usual care then it was assumed that she would make an additional visit to her GP – this was assumed to take place half-way through the follow-up period, at 32 weeks postnatally. At this visit there was a possibility that the woman’s depression would be identified by routine care and structured psychological therapy immediately offered as treatment. The structure of the model following this node was adapted from the ‘true-positive’ treatment component: if the woman responded to treatment she was assumed to have experienced a linear improvement in HRQoL from 32 weeks postnatally to 40 weeks postnatally; if she did not respond to treatment she remained depressed until the model end point at 58 weeks postnatally. If the woman responded but then relapsed it was assumed that she relapsed at the same rate as those women who relapsed following treatment initiated at 6 weeks postnatally (i.e. a linear deterioration in HRQoL over 44 weeks); as the model end point was 18 weeks into this 44-week linear deterioration, those who relapsed after responding to treatment initiated half-way through the follow-up period were assumed to be in a state of ‘partial relapse’ at the model end point. The state of ‘partial relapse’ was used simply to refer to the calculations made in relation to the HRQoL inputs as opposed to representing a separate health state of the model. Meanwhile, if the woman responded to treatment, but did not relapse then she was assumed to remain in a state of remission until the model end point, while if the woman’s depression was not identified by routine care at the additional GP visit it was assumed that she continued receiving usual care and remained depressed until the model end point.

Parameter inputs

The parameter inputs for this component were those used in the reconstructed NICE treatment model (Appendix 6) and the probability of GP identification of depression at 32 weeks postnatally. The estimate of the probability that PND was detected by a woman’s GP at 32 weeks postnatally was taken from Kessler et al. 236 This study found that 22 of the remaining 54 patients with depression who were not identified at baseline were eventually detected by the GP during 3 years of follow-up. Because of the lack of more suitable data, this finding was used to approximate the probability of GP identification of depression at the model midpoint (32 weeks postnatally); the probability that PND was detected by a GP at this time was assumed to take a beta (22,32) distribution with a mean of 22/54 (Table 32).

TABLE 33 - Probability of GP identification of depression at 32 weeks postnatally

SE, standard error.

	Mean	SE	Distribution	Source
Probability of GP identification of depression at 32 weeks postnatally	0.4074	0.0663	Beta	Kessler et al., 2002236

The ‘true-negative’ treatment component

Model structure and key assumptions

The ‘true-negative’ treatment component is given in Figure 39. All women who entered this treatment component were non-depressed and had either not been diagnosed at all or been correctly identified as non-depressed by routine care and/or a formal identification approach (when applicable).

FIGURE 39.

The ‘true-negative’ treatment component.

Each woman received usual care for non-depressed postnatal women. As the model only considered the additional costs related to the management of PND, the costs of usual care incurred by women who were not depressed were not included in the model. It was assumed that all women non-depressed at 6 weeks postnatally would remain non-depressed until the model end point; as such, all women in this component experienced the HRQoL associated with remission throughout the follow-up period.

Parameter inputs

The only parameter input for this component of the model was the utility value for the state of remission (Appendix 6).

Analytic methods of the identification model

The model was developed in Microsoft excel. 237 The Monte Carlo simulation was run for 10,000 iterations. The model was run several times representing the base-case analysis and alternative scenarios considered as part of the sensitivity analysis.

The results of the model are presented in two ways. First, mean costs and QALYs of the alternative identification strategies are presented and their cost-effectiveness compared, estimating ICERs as appropriate, using standard decision rules. 238 The ICERs examined the additional costs that one strategy incurred over another and compared this with the additional benefits. When more than two strategies were being compared the ICERs were calculated using the following process:

• The identification strategies were ranked in terms of cost (from the least expensive to the most costly).

• If a strategy was more expensive and less effective than the previous strategy then that strategy was said to be dominated and was excluded from the calculation of the ICERs.

• The ICERs were calculated for each successive alternative, from the least expensive cost to the most costly. If the ICER for a given strategy was higher than that of the next most effective strategy then that strategy was ruled out on the basis of extended dominance.

• Finally, the ICERs were recalculated excluding any strategies that were ruled out using the notions of dominance and extended dominance.

The advantage of entering input parameters as uncertain variables in the probabilistic analysis was that the uncertainty could be propagated through the model and reflected in model outputs representing uncertainty surrounding the decision itself. To present the uncertainty in the cost-effectiveness of the alternative strategies, cost-effectiveness acceptability curves (CEACs) were calculated. 239,240 These figures show the probability that each strategy was more cost-effective than the other three using alternative values for the maximum value that the health service was willing to pay for an additional QALY in these patients.

Although the CEACs provide a useful graphical representation of the uncertainty associated with the probability that individual strategies were cost-effective over a range of threshold values, the results of the CEACs could only be used to identify the optimal implementation decision under a restrictive set of assumptions. This was because the strategy with the highest probability of being cost-effective did not necessarily have the highest expected pay-off (i.e. net benefit), and would only do so when the distribution of these pay-offs was symmetrical. This limitation could be overcome by using a cost-effectiveness frontier to indicate which strategy was optimal (and the associated probability that this strategy was the most cost-effective) across the range of values representing the maximum amount that the NHS was WTP for an additional QALY. 239

Results

The results are presented in two stages: those of the base-case analysis, in which the assumptions set out in previous sections were employed; and those of the sensitivity analysis, in which the impact of employing alternative assumptions to those of the base case was explored.

Base-case results

A summary of the results of the base-case analysis is given in Table 34. Routine care was the least costly and least effective strategy. When the identification strategies were ranked in terms of cost (from the least expensive to the most expensive) the EPDS cut points 7 and 13 and the BDI cut point 10 were more expensive and less effective than the previous strategies in the list and thus were ruled out on the grounds of dominance. After calculating the ICERs for the non-dominated strategies, the ICER for the EPDS cut point 15 was found to be higher than that of the next more effective strategy on the ranked list and thus was ruled out on the grounds of extended dominance. Of the remaining non-dominated identification strategies, the EPDS at a cut point of 16 was the next most costly and effective strategy compared to routine care, with an associated ICER of £41,103 per QALY. The ICER of the EPDS at a cut point of 14 was £49,928 per QALY compared to the EPDS cut point 16. The ICER of the EPDS at lower cut points (e.g. 8, 9–11) exceeded £100,000 per QALY.

TABLE 34 - Base-case results

BDI, Beck Depression Inventory; CE, cost-effective; EPDS, Edinburgh Postnatal Depression Scale; ICER, incremental cost-effectiveness ratio; N/A, not applicable; QALY, quality-adjusted life-year; WTP, willingness to pay.

D represents a strategy ruled out through dominance; ED represents a strategy ruled out through extended dominance.

The probability that a strategy is more CE than all others given a particular maximum WTP for an additional QALY.

Identification option	QALY	Cost	ICERa	Prob. CE for max. WTPb
				£20,000	£30,000	£40,000
Routine care	0.846	£49.29	N/A	0.8765	0.5869	0.393
EPDS 16	0.846	£73.49	£41,103	0.0221	0.0614	0.0684
EPDS 15	0.846	£80.95	ED	0.007	0.0182	0.0198
EPDS 14	0.847	£94.21	£49,928	0.0158	0.0439	0.0527
EPDS 13	0.847	£110.47	D	0.0052	0.0253	0.0425
EPDS 12	0.847	£109.95	£56,697	0.0177	0.0611	0.0877
BDI 10	0.847	£121.51	D	0.0115	0.0507	0.0895
EPDS 11	0.847	£118.82	£113,411	0.0186	0.0587	0.0853
EPDS 10	0.847	£140.44	£120,968	0.0172	0.0564	0.089
EPDS 9	0.847	£156.95	£245,210	0.0065	0.026	0.0464
EPDS 7	0.847	£215.07	D	0.0001	0.0004	0.0012
EPDS 8	0.847	£187.32	£272,463	0.0018	0.011	0.0245

In general, the ranking of identification strategies appeared to be driven by their specificity, such that strategies with a high specificity (e.g. EPDS cut point 16) were associated with more favourable ICERs than strategies with a lower specificity (but correspondingly a higher sensitivity), suggesting that the costs of managing false positives represents a key driver of these results.

At each of the three WTP thresholds considered, the strategy with the highest individual probability of being cost-effective was routine case detection. However, at a threshold of £30,000 per QALY there was a probability of 0.4131 (i.e. 41%) that routine care was less cost-effective than a policy of using formal identification approaches. However, despite the formal identification strategies having a combined probability of being cost-effective which exceeded that of routine care, the probabilities associated with each of the individual strategies were low. This suggested that, although formal identification approaches had a higher chance of being cost-effective than routine care, there was significant uncertainty between the separate strategies about which should be the optimal strategy based on cost-effectiveness considerations.

The base-case cost-effectiveness frontier is given in Figure 40 for WTP threshold values up to £100,000 per QALY, demonstrating which strategy was cost-effective (and the probability that this was the most cost-effective) across this range. The high uncertainty surrounding the decision between individual identification strategies was clearly evident. For threshold values above £41,103 per QALY, the optimal identification strategy across their respective ranges had a very low probability of being cost-effective.

FIGURE 40.

Base-case cost-effectiveness frontier.

Discussion

The results of the base-case analysis suggested that the use of formal identification strategies did not appear to represent value for money based on conventional thresholds of cost-effectiveness used in the NHS. However, the scenarios considered demonstrated that this conclusion was primarily driven by the costs of false positives assumed in the base-case model. Alternative assumptions employed in separate scenarios resulted in more favourable estimates of cost-effectiveness, such that the use of the EPDS as an identification approach for women with PND fell within these conventional thresholds.

It should be recognised that the costs of additional care assumed for the management of false positives (£414) represented a significant additional cost associated with the identification strategies that in reality remains highly uncertain. In the absence of reliable data on this aspect, a conservative approach was employed as part of the base-case assumptions. However, alternative and plausible estimates (e.g. assuming a single GP appointment) or approaches (e.g. the use of a confirmatory SCID assessment) to managing false positives resulted in markedly more favourable cost-effectiveness estimates for formal identification approaches.

In addition to being a key driver in relation to the overall cost-effectiveness estimates, the costs of false-positive diagnoses were also central in terms of the relative cost-effectiveness of the different identification strategies. When the cost of a false-positive diagnosis was relatively high, as in the base case, the specificity of an identification strategy was an important contributor to its cost-effectiveness – in the base-case analysis the cost-effectiveness of the EPDS strategies with low cut points was less favourable because of their associated lower specificity. As the cost of a false-positive diagnosis fell, however, specificity became less important relative to sensitivity and, in the extreme case that such false diagnoses carried no additional cost, the EPDS cut point 8 (the strategy with the highest sensitivity) emerged as the optimal strategy. When the cost of a false-positive diagnosis was assumed to be that of a single GP attendance, the EPDS cut point 10 emerged as the optimal strategy in terms of cost-effectiveness, corresponding closely with the results presented in Chapter 5 based on the sROC curves for the alternative approaches in which the trade-off between sensitivity and specificity was considered (but not in terms of their related costs and outcomes).

Although there was an absence of reliable published data on the costs of false positives, it would appear reasonable to conclude that the actual estimate lies somewhere between the full cost of the additional care considered in the base-case analysis and that assumed for a single GP appointment. Hence, a definitive answer to the question as to whether formal identification strategies were cost-effective, and, if they were, which individual strategy was optimal in cost-effectiveness terms, clearly requires further more reliable evidence in relation to the costs of managing false positives. However, the results presented here suggested that in these scenarios the most cost-effective identification approaches are the EPDS at a cut point of 10 or higher. It should also be recognised that the cost of false positives associated with routine care were not considered as part of the base-case analysis and that some of the costs attributed to the formal identification strategies would have also been incurred during routine practice. As such, the base-case results should be considered as representing conservative estimates as to the potential value for money of formal identification approaches.

There were limited published data available for estimating other parameters in the model, namely: the probability that PND was identified via routine care at six weeks, the risk of relapse and the utility weights. As a result, the estimates used in the model were derived from studies of general depressed populations (i.e. non post-natal populations), which represents a serious limitation of the model. Furthermore, the EPDS was the only identification strategy where there was sufficient data at more than one cut point to be able to combine results and produce pooled summary estimates of sensitivity and specificity; as such, the performance of other identification strategies could not be assessed.

A further issue is the degree to which the QALY is an appropriate measure of health outcome.  While the QALY is ubiquitous throughout the health economic evaluation literature, it has been argued that (as currently constructed) it is an insensitive measure of outcomes in mental health care.241 In the absence of a suitable alternative, the QALY was adopted to ensure comparability between the interventions considered here and those outside the field of mental health; however, the potential insensitivity of the QALY in this context should be considered when interpreting the results.

Interestingly, the use of a combination identification strategy with confirmatory SCID dominated (i.e. were less costly and equally effective) the counterpart single identification strategy. However, using the base-case assumption related to the cost of a false-positive diagnosis (£414), the combination strategies still did not reach the conventional thresholds of cost-effectiveness. The finding that the confirmatory use of the SCID dominated the same individual strategy without a confirmatory screen was an interesting finding that was also closely related to the general issue of the cost of managing false positives. The use of the confirmatory SCID provided an alternative approach to reducing the costs associated with detecting false-positive diagnoses, obviating the costs of ‘additional care’ assumed elsewhere. Clearly the confirmatory SCID strategies would be dominated by the existing single identification strategies if the cost of a false positive was assumed to be the cost of a single GP attendance (as the administration cost of the SCID exceeded the cost of a GP attendance). As such, these estimates presented as part of this scenario analysis should be seen as the most optimistic estimates of the potential cost-effectiveness of using more definitive diagnostic instruments as a confirmatory approach to managing patients positively identified as depressed from an initial identification strategy such as the EPDS. There also remains an important issue of whether such a strategy would be feasible to implement in practice and whether health visitors could be trained to deliver the instrument and to interpret the subsequent data without the additional input of a more specialist practitioner. However, it was evident that alternative approaches that might be considered more appropriate, for example use of the SCID by a trained practitioner in a hospital setting, would be markedly more expensive than that considered here and hence it would appear unlikely that such an approach would be more cost-effective than the single identification approaches.

The results presented for major depression alone suggested that the use of formal identification strategies may be cost-effective as part of an approach to the management of major depression even in scenarios in which the cost of managing false positives was high. In the scenario considered, use of the EPDS as an identification strategy fell within conventional cost-effectiveness thresholds. This was primarily because of the improved test performance associated with the diagnosis of major depression alone (i.e. the probability of false positives was reduced compared with the base-case analysis evaluating major and minor depression together) and the relatively higher consequences associated with not detecting true positives as part of routine care (i.e. in terms of loss of quality of life through not receiving appropriate treatment). However, it should also be recognised that a higher proportion of women with major depression may be identified via routine practice than the estimate applied in the model, which could offset (partially or wholly) the improved cost-effectiveness estimates identified as part of this scenario.

Finally, it should be noted that the model focused on the costs and outcomes associated with the mother herself. Because of a lack of reliable evidence, no account was taken of the potential impact of successful identification and subsequent management of PND on other family members or the infant. Clearly if identification strategies and subsequent management have an important effect on these aspects then the results presented here will represent highly conservative estimates of the potential value of identification approaches.

Despite the limitations and uncertainties noted, the results presented here represent the first attempt to formally evaluate the potential cost-effectiveness of alternative identification strategies for the management of women with PND. Equally important is that this evaluation provided a systematic, integrated and transparent approach to synthesising the available evidence on the diagnostic performance of identification strategies (reported in Chapter 5) with the best available evidence relating to the subsequent management of and outcomes for women with PND reflecting current policy and practice in the NHS. Clearly there remain a number of uncertainties in relation to the assumptions that underpin the model and also in terms of the uncertainty characterised by the probability distributions assigned to the inputs. As new evidence emerges, the assumptions and inputs of this model could be updated and the results re-estimated on an ongoing basis. However, the current uncertainties related to parameter inputs could also be considered in terms of their impact on existing decision uncertainty and could be used as the basis for identifying those aspects for which further research appears to be most valuable. This was considered and is presented as part of Chapter 10.

Methods

The expected costs of decision uncertainty could also be interpreted as the EVPI, as perfect information would eliminate the possibility of making the wrong decision. Furthermore, the EVPI also represents the maximum amount that a decision-maker should be willing to pay for additional evidence to inform this decision in the future. The EVPI was used to provide an upper bound on the value of additional research to that provided by the model. This valuation was then used as a necessary requirement for determining the potential efficiency of further primary research. Applying this decision rule, additional research was only considered if the EVPI exceeded the expected cost of the research. In addition to providing a global estimate of the total cost of uncertainty related to all inputs in the model, the EVPI was also estimated for individual parameters (and groups of parameters) contained in the model. The objective of this analysis (termed partial EVPI or EVPPI) was to identify the model parameters for which it would be most worthwhile obtaining more precise estimates.

The use of Monte Carlo simulation allowed the expected costs of uncertainty associated with the initial adoption decision to be expressed as the proportion of iterations that results in an adoption decision other than that arising from maximising expected cost-effectiveness. The benefits forgone were simply the difference in the costs and outcomes (net benefit) between the optimal strategy for a given iteration and the strategy that was identified as optimal in the adoption decision (i.e. based on the expected cost-effectiveness estimates). The expectation of benefits forgone over all iterations represented the EVPI per individual.

As information can be of value to more than one individual, the EVPI was also expressed for the total population who stand to benefit over the expected lifetime of the programme/technology. If the EVPI for the population of current and future patients exceeded the expected costs of additional research then it was considered potentially cost-effective to conduct further research. The overall VOI for a population was determined by applying the individual EVPI estimate to the number of people who would be affected by the information over the anticipated lifetime of the technology:

where I = incidence in period, t = period, T = total number of periods for which information from research would be useful and r = discount rate.

Results

Base case

Population EVPI

Table 41 reports the population EVPI at the separate willingness to pay thresholds (£20,000, £30,000 and £40,000 per QALY) for the different population sizes (representing time horizons of 10 years, 15 years and 20 years) considered. The EVPI estimates ranged between £5.79 million and £170.05 million across the separate scenarios. Assuming a 10-year horizon, the corresponding estimates were in the region of £5.79 million to £99.51 million, demonstrating that there appears to be considerable value surrounding further research which could reduce the current decision uncertainty represented by the model.

Each population was calculated using the formula given previously, with I_t representing the number of new mothers per annum and assuming a discount rate, r, of 3.5%. 235 An estimate of the number of new mothers per annum was calculated using data from the Office for National Statistics243 by taking the total number of live births in 2006 (669,601) and subtracting the total number of multiple births in 2006 (10,137). The discounted 10-year, 15-year and 20-year populations were estimated to be 5,676,459, 7,861,154 and 9,700,608 respectively.

As Figure 41 shows, the population EVPI was negligible for WTP thresholds below approximately £10,000; at these thresholds the strategy of routine case detection was almost certain to be cost-effective and so there appeared to be very little value in obtaining further information about each parameter evaluated as part of the decision model. For WTP thresholds from approximately £10,000 to £41,103 the EVPI for each population increased seemingly exponentially as it became more likely that a strategy other than routine case detection was cost-effective. The growth rate of each population EVPI reached a local maximum at a WTP threshold of £41,103, where the cost-effective strategy switched to the EPDS with a cut point of 16. The growth rate of each population EVPI then slowly increased with WTP threshold and reached a second local maximum at a WTP threshold of around £72,000. At higher WTP thresholds each population EVPI continued to increase, reflecting the greater value placed on health outcomes and hence the greater value placed on perfect parameter information, but the growth rate of each remained relatively flat – at these thresholds it was highly probable that the cost-effective strategy was an identification strategy with a lower cut point (12 or below), all of which had very similar expected QALY outcomes.

FIGURE 41.

Base-case population EVPI.

Table 40 reports the population EVPI at three separate thresholds for the different population sizes considered. The EVPI estimates ranged between £5.79 million and £170.05 million across the separate scenarios. Assuming a 10-year horizon the corresponding estimates ranged from £5.79 million to £99.51 million demonstrating that there appears to be considerable value surrounding further research, which could reduce the current decision uncertainty represented by the model.

TABLE 40 - Base-case population EVPI

EVPI, expected value of perfect information; WTP, willingness to pay.

Population	EVPI for maximum WTP
	£20,000	£30,000	£40,000
10 years	£5,789,989	£40,075,803	£99,508,333
15 years	£8,018,377	£55,499,747	£137,806,029
20 years	£9,894,620	£68,486,294	£170,051,662

Partial EVPI

Although the estimates of the total population EVPI provided a useful global measure of the uncertainty surrounding the choice of identification strategy and the maximum value of future research aimed at reducing such uncertainty, they did not provide any indication of which particular aspects future research would be of most value targeting. Indeed, the population EVPI was only relevant in terms of informing further research that could address each of the separate aspects (e.g. the separate elements associated with the diagnostic and treatment models) simultaneously (e.g. by undertaking a prospective study addressing both the effectiveness and cost-effectiveness of identification and treatment strategies). However, the EVPPI could be used to estimate the maximum value of reducing uncertainty around particular parameters (or groups of policy-related parameters), allowing future research to be more specifically targeted at those parameters for which more precise estimates would be most valuable and for which a range of different study designs and approaches may be possible.

Table 41 presents the EVPPI for a number of groups of parameters which were considered to represent groupings that were relevant to both broader policy questions and the actual design of further research. For example, the diagnostic test performance of the identification strategies (as a whole and then separately according to the instrument) was separated from other aspects related to the diagnostic model (i.e. epidemiological parameters such as the prevalence of depression and the probability of routine case detection) and to the subsequent treatment model and the estimates of quality of life assigned in the model. This separation was undertaken both to reflect the different uncertainties that exist in these separate elements (and their individual contributions to the overall costs of current decision uncertainty) and because these separate aspects suggest different research studies and designs (i.e. the study designs and associated costs are likely to be markedly different for the parameters of the treatment model, which would ideally be informed by a randomised trial to minimise potential bias, compared with those represented by the utility parameters, for which issues of bias may be less important in terms of designing further research, i.e. an observational study may be considered appropriate). The estimates of the EVPPI for these groups of parameters were reported for three WTP thresholds assuming a 10-year population. For comparative purposes the total population EVPI is also given.

TABLE 41 - Base-case EVPPI

BDI, Beck Depression Inventory; EPDS, Edinburgh Postnatal Depression Scale; EVPI, expected value of perfect information; EVPPI, expected value of partial perfect information; WTP, willingness to pay.

Parameter group	EVPI/EVPPI for max. WTP (10-year population)
	£20,000	£30,000	£40,000
Total population EVPI (all parameters)	£5,789,989	£40,075,803	£99,508,333
EPDS and BDI (all cut points) sensitivity/specificity	£0	£3,178,817	£55,742,831
EPDS (all cut points) sensitivity/specificity	£0	£2,894,994	£52,393,720
BDI (all cut points) sensitivity/specificity	£0	£227,058	£12,147,623
Utility weights	£0	£170,294	£7,209,103
Other diagnostic parameters	£0	£0	£5,279,107
Treatment parameters	£227,058	£9,025,570	£40,075,803

For a WTP threshold of £20,000 per QALY, over a 10-year population, the EVPPI around the parameters associated with the treatment model alone was £227,058. There appeared no economic value in obtaining additional information solely around any of the other groups of parameters (although in relative terms there appeared significant value in obtaining perfect information around all parameters simultaneously). This finding was not unexpected as in the base-case model the probability that formal identification strategies were cost-effective was very low (and hence there was low uncertainty that routine care was optimal). However, the ICER associated with the treatment option considered (£17,481 per QALY for structured psychological therapy compared with usual care) was close to the £20,000 threshold and hence there appeared to be a high cost of decision uncertainty in relation to the question of whether treatment was itself cost-effective regardless of the identification approach employed.

For higher WTP thresholds (£30,000–40,000) there was positive expected value associated with obtaining perfect information around each of the groups of parameters separately, most notably the sensitivity and specificity associated with the diagnostic performance of the alternative identification approaches (from £3.18 million to £55.74 million) and the treatment model parameters (from £9.03 million to £40.08 million). In terms of the diagnostic performance parameters, the value associated with the EPDS (from £2.89 million to £52.39 million) represented the majority of the value associated with the alternative identification approaches considered in the model. This was because one of the key determinants of a parameter’s EVPPI was the likelihood that more precise estimates of its true value would result in a change in the cost-effective strategy (this was also true for groups of parameters).

Discussion

The results from Chapter 9 indicated that there was a considerable decision uncertainty surrounding the role of formal identification strategies for the management of women with PND. This finding was reflected in the high cost of decision uncertainty represented by the EVPI estimates presented here suggesting that further research is potentially worthwhile. At low thresholds of cost-effectiveness, future research appeared most worthwhile targeted at evidence related to the effectiveness and cost-effectiveness of treatment strategies for the management of women with confirmed PND. At higher thresholds of cost-effectiveness there appeared markedly higher potential value associated with further research more generally and also specifically around: (1) diagnostic test performance (primarily related to the use of the EPDS); (2) treatment strategies for confirmed PND; (3) the impact of PND on HRQoL; and (4) other epidemiological data considered in the diagnostic model (e.g. prevalence rates, routine case detection).

Although these findings indicate that further research is potentially worthwhile, several factors need to be taken into consideration. First, these values represent an upper bound to the value of further research as a whole and in relation to the individual parameter groups as they represent the value of perfect information (i.e. assuming that further research will completely resolve any remaining uncertainties). Clearly further research would only partially resolve these uncertainties; the costs reported here are thus only indicative as to the potential value of further research. In effect, they represent a necessary, but not sufficient condition for further research to be considered efficient (assuming that the costs of research were actually lower than that represented by the EVPI estimates). Second, these estimates are based on the assumptions and strategies considered in the associated decision model. The model itself focused on strategies for which there were considered sufficiently robust data for them to be included in the evaluation. As such the full range of potentially feasible strategies was not considered (e.g. although a number of separate cut points was considered for the EPDS, only a single cut point was considered for the BDI). Hence, the finding that the majority of the value associated with the diagnostic performance of the alternative identification strategies was attributed to the use of the EPDS (and not the BDI/Whooley questions) needs to be weighed against the more restrictive range of cut points considered in the case of the BDI and the validity of the diagnostic performance data based on the Whooley questions. Hence, there may still be considerable value that could be associated with obtaining more reliable data from a range of alternative diagnostic approaches not considered as part of the decision modelling work.

Item 5: There should be a simple, safe, precise and validated screening test

Numerous potentially suitable identification strategies for PND were identified. The systematic review in Chapter 5 identified that the EPDS was the most frequently used instrument and was the only instrument for which sufficient data were available to combine studies at multiple cut points. In terms of test performance, the EPDS performed reasonably well: sensitivity ranged from 0.60 (specificity 0.97) to 0.96 (specificity 0.45) for major depression only; from 0.31 (specificity 0.99) to 0.91 (specificity 0.67) for major or minor depression; and from 0.38 (specificity 0.99) to 0.86 (specificity 0.87) for any psychiatric disorder. Although the EPDS has reasonable sensitivity and specificity, some women with PND will be unidentified and some women without PND will be wrongly identified as having PND.

Criterion met? Yes, for the EPDS. There was a lack of evidence for the other potential identification strategies identified.

Item 6: The distribution of test values in the target population should be known and a suitable cut-off level defined and agreed

A wide variety of cut points for the EPDS have been reported. There were sufficient data at 10 cut points to be able to combine studies for certain types of disorders. In the original validation study of the EPDS16 a cut point of 10 for ‘possible depression’ and a cut point of 13 for ‘probable depression’ were suggested. The lower the cut point used to distinguish between cases and non-cases the higher the sensitivity becomes. Increasing the sensitivity of an identification strategy will lead to fewer women with PND being unidentified. Unfortunately, increasing sensitivity results in lower specificity values leading to an increase in the number of women wrongly diagnosed with PND and increasing demands on NHS resources. The systematic review in Chapter 5 demonstrated that the optimal cut point, in terms of the trade-off between sensitivity and specificity, was 12 for major depression only, 10 for major or minor depression and 9 for any psychiatric disorder. If the cut point was chosen to maximise sensitivity (from a clinical perspective) then from this analysis the optimal cut point was 7 for major depression only, 8 for major or minor depression and 9 for any psychiatric disorder. From an economic perspective the results suggested that in the scenarios considered the most cost-effective identification approach would be the EPDS at a cut point of 10 or higher.

Criterion met? Yes, in principle for the EPDS. There was a lack of evidence for other potential identification strategies identified.

Item 7: The test should be acceptable to the population

The systematic review in Chapter 6 identified 16 studies that explored the acceptability of methods to identify PND. The most frequently explored views of women and health professionals were those regarding the EPDS. Overall, the majority of studies indicated that the EPDS was acceptable when undertaken in the home, with due attention to training, with empathetic skills of the health visitor and with due consideration of positive responses to question 10 (‘the thought of harming myself has occurred to me’).

Criterion met? Yes for the EPDS. There was a lack of evidence for other potential identification strategies identified.

Item 13: There should be evidence from high-quality randomised controlled trials that the screening programme is effective in reducing mortality or morbidity

The systematic review in Chapter 7 highlighted that insufficient evidence is available to conclude that identification strategies are effective in improving maternal and infant outcomes. Some suggestive evidence indicated that the EPDS, maybe with some enhancement of care, may lead to reductions in the number of women with EPDS scores above a certain threshold or reductions in EPDS scores. Despite additional outcomes being considered, only EPDS outcomes were presented across all of the studies included in the review.

Criterion met? No.

Item 16: The opportunity cost of the screening programme (including testing, diagnosis and treatment, administration, training and quality assurance) should be economically balanced in relation to expenditure on medical care as a whole (i.e. value for money)

No full economic evaluations of PND identification strategies were identified in the systematic review in Chapter 8. In the absence of existing cost-effectiveness studies of PND identification strategies, a decision-analytic model was developed. The results of the base-case analysis suggested that the use of formal identification strategies do not appear to represent value for money based on conventional thresholds of cost-effectiveness used in the NHS. However, the scenarios considered demonstrated that this conclusion was primarily driven by the costs of false positives assumed in the base-case model. Alternative assumptions employed in separate scenarios result in more favourable estimates of cost-effectiveness, such that the use of the EPDS as an identification strategy to identify women with PND falls within these conventional thresholds. For example, when the cost of a false-positive diagnosis was assumed to be the cost of a single GP attendance, the EPDS using a cut point of 10 emerged as the optimal strategy in terms of cost-effectiveness. A definitive answer to the question as to whether formal identification strategies are cost-effective, and, if they are, which individual strategy is optimal in cost-effectiveness terms, clearly requires further more reliable evidence.

Criterion met? No.

Reflection on current policy and practice within the UK

We found that the accepted criteria for a PND screening programme were not currently met by any of the identification strategies identified. The evidence suggested that the EPDS is a simple, safe, precise and validated screening test, that in principle a suitable cut-off level could be defined and that the test is acceptable to the population. Evidence surrounding the clinical and cost-effectiveness of PND screening with the EPDS is lacking. There was insufficient evidence for all other identification strategies identified to assess them against the NSC criteria. As outlined in Chapter 1 current NICE guidance recommends the use of two questions to identify possible depression and a third question if the women answers ‘yes’ to either of the initial questions [(1) ‘During the past month, have you often been bothered by feeling down, depressed or hopeless?’, (2) ‘During the past month, have you often been bothered by little interest or pleasure in doing things?’ and (3) ‘Is this something you feel you need or want help with?’]. It is worth noting that no evidence was identified across the four systematic reviews, in terms of validity, acceptability, clinical effectiveness and cost-effectiveness, for these three questions in a postnatal population. Thus, we would also conclude that the NSC criteria for a PND screening programme using the three questions would not currently be met.

Objective 1: To provide an overview of all available methods to identify postnatal depression in primary care and to assess their validity

The survey of methods to identify PND highlighted that there were numerous measures that could be used. Diagnostic measures such as the DSM, ICD and RDC were considered the gold standard reference case for the classification of mood disorders within the postnatal period. However, the current classification of PND as a mood disorder within the postnatal period reflects underlying uncertainty regarding the entity of PND as a distinct diagnosis. In addition, disparities in diagnosis may have arisen because of differences in the time frame specified for onset and differences in the criteria for a major depressive disorder between the classification systems, for example more symptoms must be established using the RDC than using the DSM. In contrast to the use of diagnostic interview schedules, four other approaches to identify PND were identified: clinician-rated scales, generic depression identification strategies, specific PND identification strategies and case-finding questions. Clinician-rated scales were defined as measures of depression used to standardise clinical judgements and provide ratings of the duration and severity of symptoms. Generic depression (and sometimes anxiety) instruments were those designed and validated for the identification of depression in non-postnatal populations, whereas postnatal-specific measures were those designed and validated for the identification of depression in postnatal populations. Both generic depression measures and postnatal-specific measures assessed self-reported depressive symptoms and subjects rated their symptoms in terms of their frequency and severity. Six postnatal-specific measures were identified. Two of these measures (PRQ and PI) were developed for use prenatally to identify those women with depression during pregnancy and to identify those women at risk of development of significant depression in the postnatal period. In some areas of research there has been shift away from using self-report strategies to using case-finding questions (e.g. for depression). Recent NICE guidance issued on antenatal and postnatal mental health recommended using case-finding questions developed by Whooley and colleagues with an additional help question if women responded ‘yes’ to one of the two questions.

All potential PND identification strategies identified from the survey were then subjected to a systematic review of their validity (in terms of key psychometric properties). In total, 14 identification strategies were validated among women during pregnancy or the postnatal period (up to 1 year). Identification strategies included PND-specific identification strategies (EPDS, PDSS, PRQ, PI), generic depression identification strategies (BDI, GHQ, HADS, HSCL, HAMD, Zung’s SDS, SCL-90-R, Raskin, MADRS) and others (EPDS–GHQ double test). No validation studies were identified that validated the case-finding questions recommended in recent NICE guidance in a postnatal population. By far the most frequently used identification strategy was the EPDS. Studies that reported the results of applying the same identification strategy using the same cut point to diagnose the same type of disorder were pooled using a bivariate meta-analysis. There were sufficient data from postnatal studies across a large number of cut points of the EPDS to be able to combine results and produce pooled summary estimates of sensitivity and specificity. However, there were insufficient data at each cut point for most other identification strategies to be able to pool data. For major depression only the pooled estimates of sensitivity and specificity for the EPDS ranged from 0.60 to 0.96 and from 0.45 to 0.97 respectively; for any psychiatric disorder the pooled estimates of sensitivity and specificity for the EPDS ranged from 0.38 to 0.86 and from 0.85 to 0.99 respectively; and for major or minor depression the pooled estimates of sensitivity and specificity for the EPDS ranged from 0.31 to 0.91 and from 0.67 to 0.99 respectively. In addition, for major or minor depression there were sufficient data to pool the BDI and HAMD results at a single cut point. Results from this analysis highlighted that generic identification strategies may be less sensitive than the EPDS but more specific. When the psychometric attributes were pooled across the studies, high levels of between-study heterogeneity were identified in most analyses (major depression: 0–93%; major or minor depression: 41–98%). Unfortunately, none of the a priori sources of heterogeneity was predictive in a meta-regression analysis, and high levels of between-study heterogeneity remained in the model. There was some suggestive evidence that the timing of administration of the EPDS (within 6 weeks postnatally or not) may be an important factor in influencing diagnostic performance. Two other variables, verification bias and blinding, also demonstrated some potential effects on diagnostic performance. The identification strategies reviewed here appear to be able to identify PND in women during pregnancy and the postnatal period with a degree of accuracy that is similar, if not slightly better, to that for identifying depression in general populations. In an evaluation of case-finding instruments for identifying patients with major depression or dysthymia in primary care, 16 instruments were assessed in 38 studies and the overall sensitivity was 0.79 (95% CI 0.74 to 0.83) and overall specificity was 0.75 (95% CI 0.70 to 0.81). Equivalent estimates from this review for the EPDS resulted in an overall sensitivity of 0.86 (95% CI 0.81 to 0.89) and overall specificity of 0.87 (95% CI 0.80 to 0.92). In summary, the EPDS is the most frequently reported identification strategy and its diagnostic performance seems reasonably good.

Objective 2: To assess the acceptability of methods to identify postnatal depression in primary care

Both qualitative and quantitative methods were used to address the overall question of whether postnatal identification strategies were acceptable to women and health professionals. A textual narrative approach was used to synthesis the evidence, which involved grouping studies together into subgroups, writing a commentary on key aspects of studies in relation to the subgroup and then conducting subgroup synthesis. Most qualitative studies highlighted that English-speaking women thought that the EPDS was acceptable, although one study that used in-depth interviews found that 21 of 39 English-speaking women felt that the EPDS was unacceptable. With regards to the quantitative research, two surveys, comprising several hundred Australian women, reported that the majority of women found that using the EPDS was ‘comfortable’ to ‘very comfortable’ or that it was ‘fairly easy to complete’.

An important theme about the acceptability of postnatal identification strategies to women and health professionals was to ensure that the women felt comfortable and relaxed about the process so that subsequently they would answer the questions honestly. To achieve this there was evidence to support informing women well in advance of administering the identification strategy, at antenatal visits, that this was going to happen. The evidence also supported the view that identification strategies should be administered in women’s homes where there would be more privacy to discuss their emotional well-being. When administering the questionnaire it was suggested that this should be undertaken at around 6–8 weeks postnatally to allow women a period of adjustment after becoming a mother. Evidence also highlighted that rather than the completion of a questionnaire being only a pen and paper assessment it should be an opportunity to open up a dialogue and discuss the results with women as an adjunct to clinical practice. It was important that the health professional was caring and showed an interest and that it was clear to the woman that they were there to support her and not as a threat to taking the baby away. Training in counselling may therefore be beneficial so that the health professional has the skills to deal with the disclosures that the work uncovers. Similarly, health professionals should be sensitive to different cultural attitudes towards being a mother, and the ambiguity of the question about self-harm in the EPDS should be altered. An identification strategy that applies these elements to the routine assessment of women postnatally should make the experience of identification more ‘normal’ and less anxiety-provoking so that women are more likely to be honest about how they truly feel and therefore more likely to receive the support that they need.

Objective 3: To assess the clinical effectiveness of methods to identify postnatal depression in improving maternal and infant outcomes in primary care

To meet objective 3 we reviewed studies that focused on whether the routine use of case identification strategies for PND or the integration of case identification strategies with enhancements of care resulted in improvements in maternal and infant outcomes. Five studies were identified that compared use of an identification strategy with or without enhancement of care or feedback of scores with either not using an identification strategy or usual care (i.e. providing level I or level II evidence). As identified in the other reviews presented in this report, the EPDS was the most frequently used identification strategy. All of the studies indicated beneficial effects of using the EPDS in reducing EPDS scores, although some of the individual studies did not show statistically significant differences. Studies reporting dichotomous outcomes were combined in a fixed-effects meta-analysis and the pooled estimate gave an OR of 0.64 (95% CI 0.52 to 0.78). Thus, the odds of scoring above the threshold for depression in a population in which a formal method was used to identify PND (with an intervention for those identified) was 0.64 times the odds of scoring above the threshold for depression in a population in which there was no formal method to identify PND (with an intervention for those identified). In total, 25 studies were identified that reported using identification strategies at the recruitment stage to identify women at risk of PND with the aim of examining the impact of various interventions compared with usual care (i.e. level III evidence). Despite a large number of studies being identified there were a number of clinical and methodological differences between the studies, which did not permit statistical pooling of results. Furthermore, it was hard to distinguish between the benefits of using identification strategies and the effects of the intervention under study. Thus, it was difficult to draw conclusions regarding the impact of using identification strategies on maternal and infant outcomes.

Objective 4: To assess the cost-effectiveness of methods to identify postnatal depression in improving maternal and infant outcomes in primary care

Despite an extensive systematic search of the literature none of the studies identified presented full economic evaluations of PND identification strategies. In the absence of existing cost-effectiveness studies of PND identification strategies, a decision-analytic model was developed to evaluate the cost-effectiveness of a range of alternative identification strategies. The model provided a framework for the synthesis of diagnostic accuracy data reported in the validation review with a range of other relevant parameters required to establish the cost-effectiveness of using formal identification strategies for women with PND. In the base-case analysis it was assumed that a single identification strategy was used at 6 weeks postnatally to identify women with major or minor depression. The choice of identification strategies was limited to the EPDS (cut points 7–16) and the BDI (cut point 10). Separate scenarios were considered subsequent to the base-case analysis including alternative classifications (i.e. considering major depression only) and alternative identification strategies (considering a separate scenario including the Whooley questions and a scenario in which a separate confirmatory strategy was used, employing a gold-standard instrument such as SCID, in women identified as positive cases from the results of the initial diagnostic instrument). The results of the base-case analysis suggested that the use of formal identification strategies did not appear to represent value for money based on conventional thresholds of cost-effectiveness used in the NHS. However, the scenarios considered demonstrated that this conclusion was primarily driven by the costs of false positives assumed in the base-case model. Alternative assumptions employed in separate scenarios resulted in more favourable estimates of cost-effectiveness, such that the use of the EPDS as an identification strategy to identify women with PND considered in some of these scenarios fell within these conventional thresholds. For example, when the cost of a false-positive diagnosis was assumed to be that of a single GP attendance, the EPDS using a cut point of 10 emerged as the optimal strategy in terms of cost-effectiveness. Interestingly, this corresponded closely with the results presented in the validation review in which the trade-off between sensitivity and specificity was considered (but not in terms of their related cost and outcomes). A definitive answer to the question of whether formal identification strategies are cost-effective, and, if they are, which individual strategy is optimal in cost-effectiveness terms, clearly requires further more reliable evidence in relation to the costs of managing false positives. However, the results presented here suggest that, in the scenarios considered, the most cost-effective identification approach would be the EPDS at a cut point of 10 or higher.

Objective 5: To identify research priorities and the value of further research into methods to identify postnatal depression from the perspective of the UK NHS

The decision-analytic model developed to meet the previous objective was also used to provide a vehicle for identifying potential future research priorities by undertaking a VOI analysis. The VOI analysis produced an upper limit to the value of future research that could be undertaken to reduce the uncertainty associated with a decision related to the use of identification strategies for PND in the NHS. The results from the decision model indicated that there was a considerable decision uncertainty surrounding the role of formal identification strategies for the management of women with PND. This finding was reflected in the high cost of decision uncertainty represented by the EVPI estimates suggesting that further research is potentially worthwhile. At low thresholds of cost-effectiveness, future research appeared most worthwhile targeted at evidence relating to the effectiveness and cost-effectiveness of treatment strategies for the management of women with confirmed PND. At higher thresholds of cost-effectiveness there appeared markedly higher potential value associated with further research more generally and also specifically around (1) diagnostic test performance (primarily related to the use of the EPDS); (2) treatment strategies for confirmed PND; (3) the impact of PND on HRQoL; and (4) other epidemiological data considered in the diagnostic model (e.g. prevalence rates, routine case detection).

Although these findings indicated that further research was potentially worthwhile, several factors need to be taken into consideration. First, these values represent an upper bound to the value of further research as a whole and in relation to the individual parameter groups as they represent the value of perfect information (i.e. assuming that further research will completely resolve any remaining uncertainties). Second, these estimates were based on the assumptions and strategies considered in the associated decision model. The model itself focused on strategies for which there was considered sufficiently robust data for them to be included in the evaluation. As such, the full range of potentially feasible strategies was not considered (e.g. although a number of separate cut points was considered for the EPDS, only a single cut point was considered for the BDI). Hence, the finding that the majority of the value associated with the diagnostic performance of the alternative identification strategies was attributed to the use of the EPDS (and not the BDI/Whooley questions) needs to be weighed against the more restrictive range of cut points considered in the case of the BDI and the validity of the diagnostic performance data based on the Whooley questions. Hence, there may still be considerable value that could be associated with obtaining more reliable data from a range of alternative diagnostic approaches not considered as part of the decision modelling work.

Objective 6: To assess whether methods to identify postnatal depression meet minimum criteria outlined by the NSC in the light of the evidence synthesis

In the light of the results of our evidence synthesis and decision modelling we revisited the examination of PND screening against five of the NSC criteria. We found that the accepted criteria for a PND screening programme were not currently met. The evidence suggested that there is a simple, safe, precise and validated identification strategy, that in principle a suitable cut-off level could be defined and that the strategy is acceptable to the population. Evidence surrounding the clinical effectiveness and cost-effectiveness of methods to identify PND is lacking.

Limitations, assumptions and uncertainties

A series of systematic reviews was undertaken using innovative methodological approaches to summarise the available evidence. These are the largest and most comprehensive reviews to have been undertaken within the area of PND to date. Systematic searches were undertaken using 20 electronic databases, forward citation searching of key literature, personal communication with authors and searching of reference lists. All databases were searched from their inception until February 2007 with no language or other restrictions being applied. A large number of potentially relevant articles were identified from the searches (n = 11,945), of which 125 were selected for full assessment. Because of the large number of studies identified it is acknowledged that some articles of relevance may have been excluded unintentionally; however, two reviewers independently assessed study titles and abstracts so the likelihood of this was minimised. The comprehensive nature of the review was highlighted by the fact that 64 validation studies, 16 acceptability studies and 30 clinical effectiveness studies were included in reviews 1, 2 and 3 respectively. The number of studies identified, given previous research published in this area, exceeded the authors’ expectations. Despite an extensive search of the literature, which included an additional focused search of economic databases, no full economic evaluations of methods to identify PND were identified.

As the searches were undertaken in February 2007 it is possible that new literature will have emerged while preparing the HTA report for final publication. To assess the magnitude of literature published and to assess how any new literature might impact on the results of our review we undertook a scoping search of MEDLINE. The MEDLINE search outlined in Appendix 1 was re-executed in January 2009. After deduplicating and excluding studies included in the previous searches 844 studies were retrieved. After screening titles and abstracts 37 appeared to be potentially eligible for inclusion. On closer inspection 21 of the 37 studies, including three studies that were already included in the review from correspondence with authors or in other formats, would not have met the inclusion criteria. Hence, 16 studies would have been eligible for inclusion in one of the systematic reviews: validation (n = 11), acceptability (n = 2), clinical effectiveness (n = 3) and cost-effectiveness (n = 0).

Eleven studies244–254 assessed the validity of identification strategies, seven of which assessed the validity of the EPDS compared with a diagnostic interview conducted according to internationally recognised criteria. Given the large number of studies assessing the EPDS already included in the validation review the addition of these new studies is unlikely to alter the conclusions drawn from Chapter 5. The validity of nine other instruments was assessed: PDSS (n = 2); Postpartum Depression Risk Scale (PDRS; n = 1); BDI (n = 1); K6 (n = 1); K10 (n = 1); PHQ-9 (n = 1); Self-Reporting Questionnaire (SRQ; n = 1); How I feel (n = 1); Aga Khan University Anxiety and Depression Scale (AKUADS; n = 1). Although the large majority of the literature focused on the EPDS, it is interesting to note that some new instruments have been validated in women during pregnancy or the postnatal period. Given the small number of studies exploring these new instruments it would not have been possible to combine the sensitivity and specificity data in a meta-analysis.

Two studies255,256 assessed the acceptability of the EPDS when used as a screening tool and they both found that the EPDS was acceptable. One study focused on the use of the EPDS prenatally and the views of women were explored and the second study focused on the use of the EPDS postnatally and the views of health professionals were sought.

Three studies257–259 provided level III evidence for the clinical effectiveness review. One of the studies provided additional outcome data for a study already included in the review. 257 The remaining two studies were both treatment studies and used the EPDS during the recruitment stage to identify women at risk of PND with the aim of examining an exercise support programme and a multicomponent intervention compared with usual care. As all of the studies identified provided level III evidence using the EPDS as an identification strategy the conclusions drawn from Chapter 7 are unlikely to change in light of this new literature.

In summary, the scoping search of MEDLINE identified 16 studies that would potentially be eligible for inclusion in one or more of the systematic reviews; however, none of the studies would substantially alter the conclusions drawn from any of the reviews.

Future research recommendations

Evidence was lacking regarding the effectiveness of methods to identify PND in improving maternal and infant outcomes. Before undertaking a trial, further research would be desirable to determine which, out of the set of potential identification strategies highlighted in this project, is the optimal method to use:

• Evidence to underpin the validity of the two case-finding questions plus the help question in a postnatal population. This could be achieved by undertaking a validation study comparing the diagnostic performances of the EPDS, a generic depression measure (e.g. the BDI) and the two case-finding questions plus the help question with a standardised diagnostic interview conducted according to internationally recognised criteria.

• Evidence to underpin the acceptability of the EPDS, a generic depression measure and the two case-finding questions plus the help question. This could be achieved by conducting a survey on a large sample of women and health-care professionals that asks broad questions about how comfortable respondents feel about using the identification strategy, and exploring this issue in more detail using qualitative methods such as in-depth interviews. In particular, emphasis should be placed on collating acceptability data by whether women were correctly classified (i.e. true positives or true negatives) or not (i.e. false positives or false negatives). A 2 × 2 table of acceptability responses could be created.

• Evidence to underpin the natural history of PND over time. This could be achieved by undertaking a longitudinal study with particular emphasis on exploring the population of women who are formally assessed with identification strategies and the population of women in whom formal identification strategies are not utilised.

• Findings from the decision model and the VOI highlight that further evidence is desirable to underpin the costs associated with false positives, the diagnostic performance of identification strategies (primarily related to the use of the EPDS), treatment strategies for confirmed PND, the impact of PND on HRQoL and epidemiological data considered in the model (e.g. prevalence rates).

• Evidence to underpin the clinical effectiveness of the most valid and acceptable instrument to identify PND, identified from the previous two studies,. This could be achieved by undertaking an RCT of an identification strategy with additional training for health-care professionals on the procedures of using the identification strategy versus routine care. Maternal and infant outcomes could be assessed at various time points during the first postnatal year.

Contribution of authors

All authors were involved in the conception and design of the study, or the acquisition of data or analysis and interpretation of the data; drafting and revising the report; and final approval of the version to be published.

Individual contributions were as follows: CE Hewitt (Research Fellow) was responsible for the day-to-day activity of the project, study selection, data extraction, conducting the review of validation studies, conducting the review of clinical effectiveness studies, conducting the review of cost-effectiveness studies, quality assessment, data analysis, developing the stakeholder consultation, drafting the full report and making revisions to the report; SM Gilbody (Professor of Psychological Medicine and Health Services Research) was the principal investigator and was involved in the conception and design of the study, day-to-day supervision of the project, developing and implementing the stakeholder consultation and revising the manuscript critically for important intellectual content; S Brealey (Research Fellow) was involved in the day-to-day activity of the project, study selection, data extraction, developing and implementing the stakeholder consultation, drafting and making revisions to the acceptability chapter and commenting on the full report; M Paulden (Research Fellow) was responsible for the economic analysis, the interpretation and presentation of the economic data, drafting and making revisions to the economic chapter and commenting on the report. S Palmer (Senior Research Fellow) was involved in the conception and design of the study, with particular emphasis on the economic component, providing day-to-day supervision of the economic analysis, drafting and revising the economic chapter and commenting on the full report. R Mann (Research Fellow) was involved with quality assessment of all studies in review 1, carrying out the survey of methods to identify PND, drafting of the survey chapter and commenting on the report. J Green (Professor of Psychosocial Reproductive Health and Deputy Director of the Mother and Infant Research Unit) was involved in the conception and design of the study, developing and implementing the stakeholder consultation, interpretation and commenting on the report. J Morrell (Senior Research Fellow) was involved in the conception and design of the study, developing and implementing the stakeholder consultation, interpretation and commenting on the report. M Barkham (Professor of Clinical and Counselling Psychology) was involved in the conception and design of the study, interpretation and commenting on the report. K Light (Information Officer) was involved in devising the search strategy, carrying out the literature searches and writing the search methodology sections of the report. D Richards (Professor of Mental Health) was involved in the conception and design of the study, interpretation and commenting on the report.

Disclaimers

The views expressed in this publication are those of the authors and not necessarily those of the HTA programme or the Department of Health.