Three biomarker tests to help diagnose preterm labour: a systematic review and economic evaluation

Aetiology, pathology and prognosis

The WHO1 subcategorises preterm birth based on gestational age as:

• extremely preterm – < 28 weeks’ gestational age

• very preterm – ≥ 28 weeks’ and < 32 weeks’ gestational age

• moderate to late preterm – ≥ 32 weeks’ and < 37 weeks’ gestational age.

Iatrogenic preterm births are medically instigated deliveries, such as early labour induction or caesarean section. 4 These elective deliveries aim to reduce health risks to the mother or fetus owing to complications such as hypertension, intrauterine growth restriction or pre-eclampsia. 4

Spontaneous preterm labour is a multifactorial condition with various underlying pathologies including infection, breakdown of fetal–maternal tolerance, stress, decidual senescence and uterine distension (commonly associated with multifetal pregnancies). 5 Spontaneous preterm deliveries can be broadly categorised as either spontaneous labour with intact membranes or those following preterm premature rupture of membranes (PPROMs). 4 Factors associated with an increased risk of preterm delivery include stress, tobacco use, drug abuse, trauma, multifetal gestations, in vitro fertilisation, low body mass index (BMI) before pregnancy, extremes of maternal age, diabetes mellitus, high blood pressure and infection. 6,7 However, previous preterm delivery is the greatest risk factor for preterm birth. 8

Symptoms of suspected preterm labour include painful contractions or cramps, abdominal and low-back pain and an increase or change in vaginal discharge. 9 Symptoms do not always result in progression to established labour and birth; they may occur but then settle, allowing the pregnancy to continue towards term. It is understood that > 90% of women presenting with symptoms of preterm labour do not go on to deliver in the next 2 weeks and, of these, 50% will continue with pregnancy until full term. 10,11 It is important to determine whether or not preterm labour is the cause of the symptoms and to assess the risk of preterm delivery to allow appropriate management to begin as soon as possible. 12

The focus population for this report is women presenting with signs and symptoms of spontaneous preterm labour with intact membranes.

Epidemiology

Data from the England and Wales 2016 birth cohort13 report 54,143 live, preterm deliveries in accordance with the WHO definition of preterm birth (< 37 weeks’ gestational age), corresponding to 7.8% of all live births. Of these deliveries, 5.9% were categorised as extremely preterm (< 28 weeks’ gestation), 10.4% were very preterm (gestational age of ≥ 28 to < 32 weeks) and 83.7% were moderate to late preterm (≥ 32 to < 37 weeks’ gestation). 13

The 2016 UK birth cohort data collected by the Office for National Statistics (ONS)13 show that the rate of preterm births varies between ethnic populations, with the highest proportion of preterm births occurring in black Caribbean and Indian populations (10.4% and 8.03% of pregnancies in these populations, respectively) and the lowest rate of preterm births occurring in women of ‘white other’ ethnicity (6.6%). The rate of preterm delivery in the population in which ethnicity was ‘not stated’ was 8.3%. 13 In the UK, preterm labour, particularly extreme preterm labour, disproportionately affects women from low socioeconomic backgrounds. 14,15

Incidence and/or prevalence

Improvements in perinatal health-care services have resulted in vastly improved outcomes for babies born preterm, yet the prevalence of preterm birth continues to rise. 1,16

Preterm birth rates vary between countries, with higher prevalence and poorer outcomes in lower-income countries. 16 However, preterm birth is a global issue that also affects developed countries.

Impact of the health problem

Globally, preterm birth complications are directly responsible for 35% of all neonatal deaths and are the second leading cause of death in children aged < 5 years. 16,17

Morbidities associated with preterm birth are both acute and chronic and can affect all organ systems. Respiratory distress can progress to bronchopulmonary dysplasia18 and cerebral pathology (e.g. IVHs and ischaemia can lead to neurodevelopmental disorders including learning and behavioural difficulties). 19,20 In addition, gastrointestinal disorders and immunodeficiencies are also associated with preterm birth. 21,22

Although mortality and morbidity rates are higher for infants delivered at lower gestational ages and lower birthweights, near-term premature infants remain at a considerably higher risk of complications than their full-term counterparts. 20

Preterm deliveries place a significant cost burden on the NHS. In addition to initial hospitalisation, rehospitalisation and rehabilitation, other direct medical costs include medication, aids and devices such as wheelchairs, visits to physicians and home care. 23 Direct non-medical costs such as special education, adaptations to homes or cars, special meal requirements, higher insurance premiums and other disease-associated costs are an expensive burden on both families and the state. 23

Summary of the technologies

Following the NICE guidance, the technologies under assessment in this review would appear in the treatment pathway where the fFN test (at the threshold of 50 ng/ml) is currently being used. A summary of information relating to the tests is given in Table 1.

Population

For the purpose of this report, the population of interest is women with signs and symptoms of preterm labour with intact amniotic membranes, who are not in established labour and for whom a transvaginal ultrasound scan is not available or acceptable.

Current usage in the NHS

Current NICE guidelines are described in Current guidelines.

Advising clinicians report that the guidelines are not always followed in typical clinical practice. Symptomatic women presenting irrespective of gestational age will usually have the fFN test administered. In addition, clinicians advise that transvaginal ultrasound scan is rarely used in routine practice owing to a lack of trained staff available, experience or equipment availability.

Anticipated costs associated with the intervention

The cost of the Rapid fFN 10Q System is usually £35 per test, not including value-added tax (VAT). The cost per control is £40 and this is usually incurred twice per year for each site. Additional costs associated with equipment maintenance and test consumables are negligible (request for information from NICE to Hologic, Inc., 2017, personal communication from NICE).

The cost per Actim Partus test is £15, not including VAT. No other costs are associated with this test (request for information from NICE to Alere Inc., 2017, personal communication from Alere Inc.).

The cost per PartoSure test (PAMG-1) is £32, not including VAT. No other costs are associated with this test (request for information from NICE to Parsagen Diagnostics Inc., 2017, personal communication from Parsagen Diagnostics Inc.).

Fetal fibronectin used with a threshold of 50 ng/ml

Point-of-care, qualitative fFN tests currently in use in the UK include QuikCheck fFN™ (Hologic, Inc., Marlborough, MA, USA) and Rapid fFN for the TLi_IQ^® System (Hologic, Inc., Marlborough, MA, USA). 29

Clinical assessment of symptoms alone

Clinical assessment consists of taking a clinical history, observations of the woman and unborn baby and a speculum examination. See Current guidelines for more details.

Tocolytic therapy

Tocolytic therapies increase the latency period for up to 48 hours. The aim of this therapy is to allow time for neonatal transfers and to complete the course of antenatal corticosteroids (ACSs). 30 There are many classes of tocolytic drugs with different mechanisms of action. 30

The NICE guidelines24 recommend nifedipine for women between 24^+ 0 and 33^+ 6 weeks’ gestation in suspected or diagnosed labour with intact membranes. If nifedipine is contraindicated, NICE recommends oxytocin receptor antagonists [e.g. Atosiban (Sun Pharmaceuticals UK Ltd)] for tocolytic therapy.

Our clinical advisors suggest that tocolytic therapy is not commonly used in routine clinical practice. This may be attributable to recent evidence on the potential harms to the fetus and infant. 30,31

Antenatal corticosteroids

Antenatal corticosteroids {e.g. dexamethasone [Alliance Healthcare (Distribution) Ltd] or betamethasone (AAH Pharmaceuticals Ltd)} are prescribed in cases of threatened preterm labour to stimulate fetal lung development to reduce infant mortality and morbidity. 32 Following administration of steroids, there is a window within which the steroids appear to be most beneficial for the infant (Figure 1C, Norman et al.). 33 The primary documented negative effect of giving steroids is a negative reduction in birthweight of approximately 100 g. 34

The National Institute for Health and Care Excellence currently recommends ACSs for women between 26^+ 0 and 33^+ 6 weeks’ gestation in suspected, diagnosed or established preterm birth, PPROM or undergoing planned preterm delivery. ACSs should be considered for extremely preterm (between 24^+ 0 and 25^+ 6 weeks’ gestation) and near-term (34^+ 0 and 35^+ 6 weeks’ gestation) women in suspected, diagnosed and established preterm labour, PPROM or undergoing iatrogenic deliveries. 24 However, evidence regarding the effectiveness of corticosteroid treatment at very low gestational ages remains uncertain. 24 NICE recommends that clinicians should discuss the benefits and risks associated with ACSs with the patient and their family. Repeat doses are contentious owing to possible risks, although this needs to be weighed against potential benefits of reduced RDS and serious adverse infant outcomes. 32,35 For this reason, some cases of repeat dosing may be acceptable depending on the interval since last dose, gestational age and likelihood of delivery within 48 hours. 24

Magnesium sulphate

Magnesium sulphate is a neuroprotective agent that significantly reduces neurological morbidities such as cerebral palsy in preterm infants. 36

The National Institute for Health and Care Excellence recommends magnesium sulphate for women at 24^+ 0 to 29^+ 6 weeks’ gestation in established labour or with iatrogenic delivery planned within 24 hours. Magnesium sulphate should also be considered for women between 30^+ 0 and 33^+ 6 weeks’ gestation. A 4-g intravenous bolus dose of magnesium sulphate should be administered over 15 to 20 minutes, followed by an intravenous infusion of 1 g per hour for 24 hours or until delivery. Patients should be routinely monitored for signs of magnesium toxicity.

Methods of the systematic review

The aim of this systematic review was to identify and summarise the diagnostic test accuracy (DTA) data for PartoSure, Actim Partus and qfFN (at thresholds other than 50 ng/ml) from test accuracy studies that provide data for one or more of these index tests.

Overview of the quantity and quality of research available

The searches retrieved a total of 2619 unique titles and abstracts. A total of 2177 articles were excluded, based on screening titles and abstracts. The remaining 442 articles were requested as full texts for more in-depth screening.

Of the 442 articles retrieved as full texts, 415 were excluded. The primary reasons for exclusion were use of an irrelevant test, typically qualitative fFN (n = 129), the study design (n = 59), that outcomes did not match the review inclusion criteria (n = 31) or that the article was an abstract that both had insufficient information to be included in the review and was unconnected to any of the included studies (n = 108). Abstracts were included if they were connected (by reporting data from the same study) to a full-text included study. The bibliographic details of studies retrieved as full papers and subsequently excluded, along with the reasons for their exclusion, are detailed in Appendix 3. Additional tables (see Tables 28–30) are provided in Appendix 3, listing all the citations provided by the industry to NICE along with whether or not the citation was included, and, if not, the reason for exclusion. After screening relevant systematic reviews (n = 11; see Appendix 3, Included systematic reviews), and forward and backward citations of the included studies, no further new included studies were identified. Twenty studies from 31 citations met the review inclusion criteria. The process of study selection is shown in Figure 2.

FIGURE 2.

Summary of the selection process.

Description of the included studies

Characteristics of the included studies are summarised in Table 3. Two studies, Assessment of Perinatal Outcome after Sustained Tocolysis in Early Labour (APOSTEL-1)42,43 and Hadzi-Lega et al. ,44 assessed the DTA of two different index tests in the same population: APOSTEL-142,43 assessed both Actim Partus and qfFN whereas the study by Hadzi-Lega et al. 44 assessed Actim Partus and PartoSure. APOSTEL-142,43 was one of the larger studies (n = 350), conducted in 10 centres around the Netherlands, whereas the study by Hadzi-Lega et al. 44 was smaller (n = 57), from one centre in Macedonia.

A further 14 studies45–58 assessed the DTA for Actim Partus only. Three studies41,59–61 assessed PartoSure only and one62 assessed qfFN only.

For Actim Partus, study sizes ranged from 30 in Vishwekar et al. 58 to 468 in Tripathi et al. 57 and covered the following countries: Kuwait,45 Turkey,46,51,53,55 Malaysia,47 Spain,48 Canada,49 Italy,50,54 India52,57,58 and Singapore. 56 The three studies assessing PartoSure were conducted in Russia,59 Macedonia60,61 and France41 and the study size ranged from 41 in Werlen et al. 41 to 203 in Nikolova et al. 61 Finally, the Bruijn study62 assessed qfFN only and recruited 455 participants from 10 centres across Austria, Belgium, Germany, the Netherlands and Switzerland.

Description of included participants

Summary of any treatments given

The index tests of interest are designed to be used before the reference standard (the occurrence of preterm delivery within 48 hours and/or 7 days). For this reason, it is important to consider the use of any treatments that might have an impact on the reference standard and, thus, the test accuracy results. Whether or not a woman received treatment for symptoms of preterm labour varied substantially between studies. This variability was based on (1) the standard treatment protocols used within each study and (2) the test(s) used to initiate treatment within the study (e.g. Actim Partus, PartoSure or qfFN or another test that is not being assessed as part of this review, such as transvaginal cervical length). Subsequently, when treatment was given, the number of women receiving treatment was not always reported, particularly with reference to the results of the diagnostic tests of interest. This means that in several cases it is difficult to ascertain the extent to which treatments may have had an impact on the test accuracy results.

Quality appraisal of included studies

Quality appraisal was conducted, using phase 3 of the QUADAS-2 tool,39 for all 20 studies. Phase 3 of the QUADAS-2 tool contains four domains: patient selection, index tests, reference standard, and flow and timing. The APOSTEL-1 study42,43 and the study by Hadzi-Lega et al. 44 each assess two index tests (Actim Partus and qfFN, and PartoSure and Actim Partus, respectively). For these two studies, the index test domain was conducted separately for each test.

It is important to note that the QUADAS-239 tool assesses the likely risk of bias and not the presence or magnitude of bias. Therefore, any rating of ‘high’ means that the risk of bias is high but does not mean that there is a high degree of bias, or even that bias has been detected. The quality of the included studies is discussed in the following sections, and a summary of the QUADAS-2 ratings is provided in Table 5. Further details relating to patient selection, index tests, reference standard, and flow and timing can also be found in Table 5.

Results of quantitative data synthesis (test accuracy data)

The 7-day delivery reference standard

Studies evaluating more than one index test

Two studies (APOSTEL-142,43 and Hadzi-Lega et al. 44) reported test accuracy data on two index tests. Both studies used only the 7-day (and not the 48-hour) delivery reference standard. The prevalence of preterm birth within 7 days was 19.7% (95% CI 15.7% to 24.3%) in the APOSTEL-1 study42,43 and 10.5% (95% CI 4.0% to 21.5%) in the study by Hadzi-Lega et al. 44

The APOSTEL-1 study reports test accuracy from 350 women for both Actim Partus and qfFN (Table 6). 42,43 The sensitivity and specificity for Actim Partus, against delivery within 7 days, were 78.3% (95% CI 66.7% to 87.3%) and 89.3% (95% CI 85.1% to 92.7%), respectively. As would be expected, in the qfFN results from the APOSTEL-1 study, lowering the threshold for a positive test result increased sensitivity and decreased specificity whereas elevating the threshold for a positive test result increased specificity and decreased sensitivity (see Table 6). The qfFN sensitivity and specificity values that were most similar to Actim Partus values for the APOSTEL-1 study were those provided at a threshold of 200 ng/ml, at which sensitivity was 71.0% (95% CI 58.8% to 83.1%) and specificity was 83.6% (95% CI 78.8% to 87.8%). With regard to the qfFN data from APOSTEL-1, the threshold with the highest PPV was 500 ng/ml (70.7%, 95% CI 54.5% to 83.9%) and the threshold with the lowest PPV was 10 ng/ml (28.9%, 95% CI 23.2% to 35.3%). For Actim Partus the PPV of delivery within 7 days was 64.3% (95% CI 53.1% to 74.4%). For qfFN, the highest NPV was at a threshold of 10 ng/ml (97.5%, 95% CI 93.0% to 99.5%) and the lowest at a threshold of 500 ng/ml (87.1%, 95% CI 82.8% to 90.6%). For Actim Partus, NPV was 94.4% (95% CI 90.9% to 96.8%). LRs, concordance and yield were also calculated for this study, and these values are provided in Table 6.

TABLE 6 - Calculated diagnostic accuracy parameters against the 7-day reference standard

N/A, not applicable.

Study (first author/study name and year)	Participants (n)	Diagnostic accuracy (95% CI)
		Sensitivity (%)	Specificity (%)	LR+	LR–	PPV (%)	NPV (%)	Prevalence (%)	Concordance	Yield
Study assessing qfFN and Actim Partus
APOSTEL-1 (2016)42,43
fFN at 10 ng/ml	350	95.7 (87.8 to 99.1)	42.3 (36.5 to 48.4)	1.66 (1.48 to 1.86)	0.10 (0.03 to 0.31)	28.9 (23.2 to 35.3)	97.5 (93.0 to 99.5)	19.7 (15.7 to 24.3)	0.53 (0.48 to 0.58)	0.65 (0.60 to 0.70)
fFN at 200 ng/ml	350	71.0 (58.8 to 81.3)	83.6 (78.8 to 87.8)	4.34 (3.20 to 5.88)	0.35 (0.24 to 0.50)	51.6 (41.1 to 62.0)	92.2 (88.1 to 95.1)	19.7 (15.7 to 24.3)	0.81 (0.77 to 0.85)	0.27 (0.23 to 0.32)
fFN at 500 ng/ml	350	42.0 (30.2 to 54.5)	95.7 (92.7 to 97.8)	9.84 (5.30 to 18.28)	0.61 (0.49 to 0.74)	70.7 (54.5 to 83.9)	87.1 (82.8 to 90.6)	19.7 (15.7 to 24.3)	0.85 (0.81 to 0.89)	0.12 (0.09 to 0.16)
Actim Partus	350	78.3 (66.7 to 87.3)	89.3 (85.1 to 92.7)	7.33 (5.11 to 10.51)	0.24 (0.16 to 0.38)	64.3 (53.1 to 74.4)	94.4 (90.9 to 96.8)	19.7 (15.7 to 24.3)	0.87 (0.83 to 0.91)	0.24 (0.20 to 0.29)
Study assessing PartoSure and Actim Partus
Hadzi-Lega (2017)44
PartoSure	57	83.3 (35.9 to 99.6)	90.2 (78.6 to 96.7)	8.50 (3.43 to 21.03)	0.18 (0.03 to 1.11)	50.0 (18.7 to 81.3)	97.9 (88.7 to 99.9)	10.5 (4.0 to 21.5)	0.90 (0.79 to 0.96)	0.18 (0.09 to 0.30)
Actim Partus	57	83.3 (35.9 to 99.6)	76.5 (62.5 to 87.2)	3.54 (1.92 to 6.52)	0.22 (0.04 to 1.31)	29.3 (10.3 to 56.0)	97.5 (86.8 to 99.9)	10.5 (4.0 to 21.5)	0.77 (0.64 to 0.87)	0.30 (0.18 to 0.43)
Actim Partus
Abo El-Ezz (2014)45
Actim Partus	57	66.7 (47.2 to 82.7)	66.7 (46.0 to 83.5)	2.00 (1.11 to 3.61)	0.50 (0.28 to 0.89)	69.0 (49.2 to 84.7)	64.3 (44.1 to 81.4)	52.6 (39.0 to 66.0)	0.67 (0.53 to 0.79)	0.51 (0.37 to 0.64)
Altinkaya (2009)46
Actim Partus	105	64.3 (35.1 to 87.2)	82.4 (73.0 to 89.6)	3.66 (2.02 to 6.61)	0.43 (0.21 to 0.88)	36.0 (18.0 to 57.5)	93.8 (86.0 to 97.9)	13.3 (7.5 to 21.4)	0.80 (0.71 to 0.87)	0.24 (0.16 to 0.33)
Azlin (2010)47
Actim Partus	51	80.0 (28.4 to 99.5)	93.5 (82.1 to 98.6)	12.27 (3.77 to 39.86)	0.21 (0.04 to 1.24)	57.1 (18.4 to 90.1)	97.7 (88.0 to 99.9)	9.8 (3.3 to 21.4)	0.92 (0.81 to 0.98)	0.14 (0.06 to 0.26)
Brik (2010)48
Actim Partus	276	74.2 (55.4 to 88.1)	66.1 (59.8 to 72.0)	2.19 (1.67 to 2.87)	0.39 (0.21 to 0.71)	21.7 (14.3 to 30.8)	95.3 (90.9 to 97.9)	11.2 (7.8 to 15.6)	0.67 (0.61 to 0.73)	0.38 (0.33 to 0.44)
Cooper (2012)49
Actim Partus	349	33.3 (4.3 to 77.7)	74.1 (69.1 to 78.6)	1.28 (0.41 to 4.04)	0.90 (0.51 to 1.59)	2.2 (0.3 to 7.7)	98.4 (96.1 to 99.6)	1.7 (0.6 to 3.7)	0.73 (0.68 to 0.78)	0.26 (0.22 to 0.31)
Danti (2011)50
Actim Partus	60	50.0 (6.8 to 93.2)	69.6 (55.9 to 81.2)	1.65 (0.57 to 4.74)	0.72 (0.27 to 1.94)	10.5 (1.3 to 33.1)	95.1 (83.5 to 99.4)	6.7 (1.8 to 16.2)	0.68 (0.55 to 0.80)	0.32 (0.20 to 0.45)
Eroglu (2007)51
Actim Partus	51	83.3 (35.9 to 99.6)	84.4 (70.5 to 93.5)	5.36 (2.48 to 11.56)	0.20 (0.03 to 1.19)	41.7 (15.2 to 72.3)	97.4 (86.5 to 99.9)	11.8 (4.4 to 23.9)	0.84 (0.71 to 0.93)	0.24 (0.13 to 0.38)
Goyal (2016)52
Actim Partus	60	59.1 (43.2 to 73.7)	50.0 (24.7 to 75.3)	1.18 (0.68 to 2.04)	0.82 (0.45 to 1.50)	76.5 (58.8 to 89.3)	30.8 (14.3 to 51.8)	73.3 (60.3 to 83.9)	0.57 (0.43 to 0.69)	0.57 (0.43 to 0.69)
Lembet (2002)53
Actim Partus	36	93.8 (69.8 to 99.8)	85.0 (62.1 to 96.8)	6.25 (2.19 to 17.88)	0.07 (0.01 to 0.49)	83.3 (58.6 to 96.4)	94.4 (72.7 to 99.9)	44.4 (27.9 to 61.9)	0.89 (0.74 to 0.97)	0.50 (0.33 to 0.67)
Riboni (2011)54
Actim Partus	210	50.0 (15.7 to 84.3)	83.7 (77.8 to 88.5)	3.06 (1.43 to 6.54)	0.60 (0.30 to 1.20)	10.8 (3.0 to 25.4)	97.7 (94.2 to 99.4)	3.8 (1.7 to 7.4)	0.82 (0.77 to 0.87)	0.18 (0.13 to 0.24)
Tanir (2009)55
Actim Partus	68	93.3 (68.1 to 99.8)	79.2 (65.9 to 89.2)	4.50 (2.61 to 7.74)	0.08 (0.01 to 0.56)	56.0 (34.9 to 75.6)	97.7 (87.7 to 99.9)	22.1 (12.9 to 33.8)	0.82 (0.71 to 0.91)	0.37 (0.25 to 0.49)
Ting (2007)56
Actim Partus	94	70.6 (44.0 to 89.7)	77.9 (67.0 to 87.6)	3.20 (1.90 to 5.38)	0.38 (0.18 to 0.80)	41.4 (23.5 to 61.1)	92.3 (83.0 to 97.5)	18.1 (10.9 to 27.4)	0.77 (0.67 to 0.85)	0.31 (0.22 to 0.41)
Tripathi (2016)57
Actim Partus	468	94.7 (89.9 to 97.7)	92.4 (88.9 to 95.1)	12.43 (8.45 to 18.30)	0.06 (0.03 to 0.11)	85.7 (79.5 to 90.6)	97.3 (94.8 to 98.8)	32.5 (28.3 to 37.0)	0.93 (0.91 to 0.95)	0.36 (0.32 to 0.41)
Vishwekar (2017)58
Actim Partus	30	72.2 (46.5 to 90.3)	90.9 (58.7 to 99.8)	7.94 (1.20 to 52.62)	0.31 (0.14 to 0.66)	92.9 (66.1 to 99.8)	66.7 (38.4 to 88.2)	62.1 (42.3 to 79.3)	0.79 (0.60 to 0.92)	0.48 (0.29 to 0.67)
PartoSure
Bolotskikh (2017)59
PartoSure	99	100.0 (73.5 to 100.0)	95.4 (88.6 to 98.7)	21.75 (8.35 to 56.64)	0.00 (N/A)	75.0 (47.6 to 92.7)	100.0 (95.7 to 100.0)	12.1 (6.4 to 20.2)	0.96 (0.90 to 0.99)	0.16 (0.10 to 0.25)
Nikolova (2014 and 2015)60,61
PartoSure	203	80.0 (63.1 to 91.6)	94.6 (90.1 to 97.5)	14.93 (7.74 to 28.80)	0.21 (0.11 to 0.41)	75.7 (58.8 to 88.2)	95.8 (91.5 to 98.3)	17.2 (12.3 to 23.2)	0.92 (0.88 to 0.95)	0.18 (0.13 to 0.24)
Werlen (2015)41
PartoSure	41	0.0 (0.0 to 97.5)	97.5 (96.8 to 99.9)	0.00 (N/A)	1.03 (0.98 to 1.08)	0.0 (0.0 to 97.5)	97.5 (96.8 to 99.9)	2.4 (0.1 to 12.9)	0.95 (0.84 to 0.99)	0.02 (0.00 to 0.13)
qfFN
Bruijn (2016)62
fFN at 10 ng/ml	455	93.8 (82.8 to 98.7)	32.2 (27.7 to 37.0)	1.38 (1.25 to 1.53)	0.19 (0.06 to 0.59)	14.0 (10.4 to 18.3)	97.8 (93.6 to 99.5)	10.5 (7.9 to 13.7)	0.39 (0.34 to 0.43)	0.71 (0.66 to 0.75)
fFN at 200 ng/ml	455	70.8 (55.9 to 83.0)	78.6 (74.3 to 82.5)	3.31 (2.55 to 4.30)	0.37 (0.24 to 0.58)	28.1 (20.3 to 37.0)	95.8 (93.1 to 97.7)	10.5 (7.9 to 13.7)	0.78 (0.74 to 0.82)	0.27 (0.23 to 0.31)
fFN at 500 ng/ml	455	29.2 (17.0 to 44.1)	94.3 (91.6 to 96.4)	5.16 (2.85 to 9.34)	0.75 (0.63 to 0.90)	37.8 (22.5 to 55.2)	91.9 (88.8 to 94.3)	10.5 (7.9 to 13.7)	0.99 (0.84 to 0.90)	0.08 (0.06 to 0.11)

The other study providing data on more than one index test in the same sample was the study by Hadzi-Lega et al. ,44 in which test accuracy data from 57 women were reported for both Actim Partus and PartoSure (see Table 6). The sensitivity of both Actim Partus and PartoSure for delivery within 7 days was 83.3% (95% CI 35.9% to 99.6%), and specificity was higher for PartoSure (90.2%, 95% CI 78.6% to 96.7%) than for Actim Partus (76.5%, 95% CI 62.5% to 87.2%). In addition, PPV, LR+ and concordance were higher for PartoSure than for Actim Partus, although the wide CIs, particularly for PPV, are notable. LR– and NPV were similar for both tests and diagnostic yield was higher for Actim Partus than for PartoSure. Specific values are given in Table 6.

Actim Partus

Results for Actim Partus against the 7-day delivery reference standard were provided by 16 studies (Figure 3, and see Table 6). Across these studies, sensitivity ranged from 33.3% (95% CI 4.3% to 77.7%) in the study by Cooper et al. 49 to 94.7% (95% CI 89.9% to 97.7%) in the study by Tripathi et al. 57 Specificity of Actim Partus ranged from 50.0% (95% CI 24.7% to 75.3%) in the study by Goyal et al. 52 to 93.5% (95% CI 82.1% to 98.6%) in the study by Azlin et al. 47 The three studies with the lowest sensitivity were those by Cooper et al. 49 (33.3%, 95% CI 4.3% to 77.7%), Danti et al. 50 (50%, 95% CI 6.8% to 93.2%) and Riboni et al. 54 (50%, 95% CI 15.7% to 84.3%). The prevalence reported in these studies was much lower [prevalence ranging from 1.7% (95% CI 0.6% to 3.7%) to 6.7% (95% CI 1.8% to 16.2%)] than in all other studies [prevalence ranging from 9.8% (95% CI 3.3% to 21.4%) to 73.3% (95% CI 60.3% to 83.9%)]. Indeed, the large range of prevalence estimates across these studies is particularly noteworthy (see Differences between studies in prevalence of preterm birth). Meanwhile, the three studies with the lowest specificities were those by Goyal et al. 54 (50%, 95% CI 24.7% to 75.3%), Brik et al. 48 (66%, 95% CI 59.8% to 72.0%) and Abo El-Ezz et al. 45 (67%, 95% CI 46.0% to 83.5%). There were no obvious methodological or participant characteristics in these studies to explain the differences, and, although two of these studies45,54 had high prevalence, the other48 did not.

FIGURE 3.

The ROC plot for Actim Partus against the 7-day reference standard. HSROC, hierarchical summary receiver-operating characteristic.

A summary ROC plot for all 16 studies assessing Actim Partus against the 7-day delivery reference standard is provided in Figure 3. Pooled analyses were undertaken for these data and provided a pooled sensitivity of 77% (95% CI 68% to 83%) and a pooled specificity of 81% (95% CI 76% to 85%).

Data from these 16 Actim Partus studies were also used to calculate LR+, LR–, PPV, NPV, concordance and yield. These values are provided in Table 6.

PartoSure

Results for PartoSure against the 7-day delivery reference standard were reported in four studies (Figure 4, and see Table 6). Prevalence of preterm delivery within 7 days ranged from 2.4% (95% CI 0.1% to 12.9%) in the study by Werlen et al. 41 to 17.2% (95% CI 12.3% to 23.2%) in the study by Nikolova et al. 61 These four studies had wide-ranging sensitivity, from 0% (95% CI 0.0% to 97.5%) in the study by Werlen et al. 41 to 100% (95% CI 73.5% to 100.0%) in the study by Bolotskikh et al. ,59 whereas specificity was more similar across studies, ranging from 90.2% (95% CI 78.6% to 96.7%) in the study by Hadzi-Lega et al. 44 to 97.5% (95% CI 96.8% to 99.9%) in the study by Werlen et al. 41 The low sensitivity in the study by Werlen et al. 41 is because, in the sample of 41 participants, only one tested (falsely) positive using the PartoSure test. Discounting this study, the sensitivity range would be 80% (95% CI 63.1% to 91.6%) in the study by Nikolova et al. 61 to 100% (95% CI 73.5% to 100.0%) in the study by Bolotskikh et al. 59

FIGURE 4.

The ROC plot for PartoSure against the 7-day reference standard.

A summary ROC plot for the four studies assessing PartoSure against the 7-day delivery reference standard is provided in Figure 3. Pooled analyses were conducted for these data and provided a pooled sensitivity of 83% (95% CI 61% to 94%) and a pooled specificity of 95% (95% CI 89% to 98%).

Data from these four PartoSure studies were also used to calculate LR+, LR–, PPV, NPV, concordance and yield. These values are provided in Table 6.

Quantitative fetal fibronectin

Results against the 7-day delivery reference standard for the two qfFN studies (Bruijn62 and APOSTEL-142,43), at the three thresholds (10 ng/ml, 200 ng/ml and 500 ng/ml), are presented in Table 6. Prevalence of preterm birth within the sample was lower in Bruijn62 (10.5%, 95% CI 7.9% to 13.7%) than in APOSTEL-142,43 (19.7%, 95% CI 15.7% to 24.3%). Bruijn62 presented with slightly lower (within 2%) sensitivity values than APOSTEL-142,43 at both the 10-ng/ml and 200-ng/ml thresholds (see Table 6). At the 500-ng/ml threshold, the sensitivity was much lower in Bruijn62 (29.2%, 95% CI 17.0% to 44.1%) than in APOSTEL-142,43 (42%, 95% CI 30.2% to 54.4%). Similarly, specificity values were slightly lower (within 5%) in Bruijn62 than in APOSTEL-142,43 at both the 200-ng/ml and 500-ng/ml thresholds, whereas at the 10-ng/ml threshold the specificity was much lower (32.2%, 95% CI 27.7% to 37.0%) in Bruijn62 than in APOSTEL-142,43 (42.3%, 95% CI 36.5% to 48.4%).

Data from the two qfFN studies were also used to calculate LR+, LR–, PPV, NPV, concordance and yield. These values are provided in Table 6.

The 48-hour delivery reference standard

Seven of the included studies also provided test accuracy data for the index tests against a 48-hour preterm delivery reference standard. Six of these studies evaluated Actim Partus48,52,53,56–58 and one evaluated PartoSure. 41

Actim Partus

Across the six studies evaluating Actim Partus against the prevalence of preterm birth within 48 hours,48,52,53,56–58 the prevalence of preterm birth within 48 hours ranged from 5.3% (95% CI 1.7% to 12.0%) in the study by Ting et al. 56 to 58.3% (95% CI 44.9% to 70.9%) in the study by Goyal et al. 52 Sensitivity ranged from 65.7% (95% CI 47.8% to 80.9%) in the study by Goyal et al. 52 to 100.0% (95% CI 47.8% to 100.0%) in the study by Ting et al. 56 Specificity ranged from 56.0% (95% CI 34.9% to 75.6%) in the study by Goyal et al. 52 to 82.4% (95% CI 56.6% to 96.2%) in the study by Vishwekar et al. 58

Specific sensitivity and specificity values for all six studies are given in Table 7, in which it can be seen that the sensitivity and specificity of Actim Partus for the 48-hour reference standard were lowest in the studies by Goyal et al. 52 and Brik et al. 48 compared with the other four studies,53,56–58 which seem more in line with each other. There were no obvious methodological or participant characteristic differences in these studies (other than that the women in the studies by Goyal et al. 52 and Brik et al. 48 were, on average, 1 year older than those in the other studies). Brik et al. 48 did present data for the number of women who received tocolytics (77.2%), but the other five studies did not provide this information, so we cannot assess whether this was particularly high or low in comparison.

TABLE 7 - Calculated diagnostic accuracy parameters against the 48-hour delivery reference standard

N/A, not applicable.

Study (first author and year)	Participants (n)	Diagnostic accuracy (95% CI)
		Sensitivity (%)	Specificity (%)	LR+	LR–	PPV (%)	NPV (%)	Prevalence (%)	Concordance	Yield
Actim Partus
Brik (2010)48	276	73.9 (51.6 to 89.8)	64.8 (58.6 to 70.7)	2.10 (1.56 to 2.82)	0.40 (0.20 to 0.81)	16.0 (9.6 to 24.4)	96.5 (92.5 to 98.7)	8.3 (5.4 to 12.2)	0.66 (0.60 to 0.71)	0.38 (0.33 to 0.44)
Goyal (2016)52	60	65.7 (47.8 to 80.9)	56.0 (34.9 to 75.6)	1.49 (0.90 to 2.47)	0.61 (0.34 to 1.09)	67.6 (49.5 to 82.6)	53.8 (33.4 to 73.4)	58.3 (44.9 to 70.9)	0.62 (0.48 to 0.74)	0.57 (0.43 to 0.69)
Lembet (2002)53	36	93.3 (68.1 to 99.8)	81.0 (58.1 to 94.6)	4.90 (2.01 to 11.96)	0.08 (0.01 to 0.55)	77.8 (52.4 to 93.6)	94.4 (72.7 to 99.9)	41.7 (25.5 to 59.2)	0.86 (0.71 to 0.95)	0.50 (0.33 to 0.67)
Ting (2007)56	94	100.0 (47.8 to 100.0)	74.2 (63.8 to 82.9)	3.87 (2.72 to 5.50)	0.00 (N/A)	17.9 (6.1 to 36.9)	100.0 (94.6 to 100.0)	5.3 (1.7 to 12.0)	0.76 (0.66 to 0.84)	0.30 (0.21 to 0.40)
Tripathi (2016)57	468	95.5 (89.7 to 98.5)	82.1 (77.8 to 86.0)	5.34 (4.26 to 6.69)	0.06 (0.02 to 0.13)	62.1 (54.4 to 69.5)	98.3 (96.1 to 99.5)	23.5 (19.7 to 27.6)	0.85 (0.82 to 0.88)	0.36 (0.32 to 0.41)
Vishwekar (2017)58	30	84.6 (54.6 to 98.1)	82.4 (56.6 to 96.2)	4.79 (1.67 to 13.74)	0.19 (0.05 to 0.68)	78.6 (49.2 to 95.3)	87.5 (61.7 to 98.4)	43.3 (25.5 to 62.6)	0.83 (0.65 to 0.94)	0.47 (0.28 to 0.66)
PartoSure
Werlen (2015)41	41	0.0 (0.0 to 97.5)	97.5 (86.8 to 99.9)	0.00 (N/A)	1.03 (0.98 to 1.08)	0.0 (0.0 to 97.5)	97.5 (86.8 to 99.9)	2.4 (0.1 to 12.9)	0.95 (0.84 to 0.99)	0.02 (0.00 to 0.13)

A ROC plot for the six studies assessing Actim Partus against the 48-hour delivery reference standard is provided in Figure 5. Pooled analyses were conducted for these data and provided a pooled sensitivity of 87% (95% CI 74% to 94%) and a pooled specificity of 73% (95% CI 62% to 82%).

FIGURE 5.

The ROC plot for Actim Partus against the 48-hour delivery reference standard.

In the six studies evaluating Actim Partus against the prevalence of preterm birth within 48 hours, the PPV was lower in the studies by Brik et al. 48 and Ting et al. 56 [16.0% (95% CI 9.6% to 24.4%) and 17.9% (95% CI 6.1% to 36.9%), respectively] than in the other four studies [range 62.1% (95% CI 54.4% to 69.5%) to 78.6% (95% CI 49.2% to 95.3%)]. This is likely to be linked to the prevalence also being low in these two studies (8.3%, 95% CI 5.4% to 12.2% in the study by Brik et al. 48 and 5.3%, 95% CI 1.7% to 12.0% in the study by Ting et al. 56). Conversely, NPV was lowest in the study by Goyal et al. 52 (53.8%, 95% CI 33.4% to 73.4%), with NPV in the other five studies ranging from 87.5% (95% CI 61.7% to 98.4%) in the study by Vishwekar et al. 58 to 100% (95% CI 94.6% to 100.0%) in the study by Ting et al. 56 Looking at the diagnostic yield, the study by Goyal et al. 52 was the only study in which > 50% of the population had a positive Actim Partus result (57%, 95% CI 43% to 69%); all other studies had a diagnostic yield of ≤ 50%. Data from these six Actim Partus studies were also used to calculate LR+, LR– and concordance. These values are provided in Table 7.

PartoSure

In the study evaluating PartoSure against the prevalence of preterm birth within 48 hours,41 prevalence of preterm birth within 48 hours was lower (2.4%, 95% CI 0.1% to 12.9%) than in any of the Actim Partus studies discussed in the previous section. Sensitivity was 0.0% (95% CI 0.0% to 97.5%) and specificity was 97.5% (95% CI 86.8% to 99.9%); the total sample size was 41 and only one test result was positive (a false positive). 41 These data, along with calculated values for PPV, NPV, LR+, LR–, concordance and yield, are given in Table 7.

Data derived from the systematic review of diagnostic test accuracy

Summary of the data available across the systematic review and overview

Table 8 summarises the results from the systematic review of DTA for the three index tests (PartoSure, Actim Partus and qfFN).

As can be seen in Table 8, Actim Partus and PartoSure were assessed in the same sample in one study (Hadzi-Lega et al. 44) and Actim Partus and qfFN were assessed in the same sample in one other study (APOSTEL-142–44). No studies were identified that assessed PartoSure and qfFN in the same sample.

As well as being assessed in the same sample as Actim Partus in the study by Hadzi-Lega et al. ,44 PartoSure was assessed in the same sample as the QuikCheck test in one study (Nikolova et al. 61).

Summary of test accuracy data across the systematic review

Table 9 summarises the sensitivity and specificity data for the index tests in the systematic review of test accuracy (PartoSure, Actim Partus and qfFN at thresholds other than 50 ng/ml). 24

With regard to the reference standard of preterm delivery within 7 days:

• Data set 1 (see Table 9) reports test accuracy data obtained from the two studies that assess two index tests (included in the systematic review).

• Data set 2 (see Table 9) reports the sensitivity and specificity values from all included studies (those evaluating only one index test and those evaluating more than one index test); for all index tests, data set 2 (see Table 9) reports the range of sensitivities and specificities across the individual studies. Meta-analyses were conducted to calculate pooled sensitivity and specificity for the 16 studies assessing Actim Partus and for the four studies assessing PartoSure. Meta-analyses were not conducted for qfFN because only two studies of this test were included.

With regard to the reference standard of preterm delivery within 48 hours:

• Data set 3 (see Table 9) reports the range of sensitivities and specificities across the six Actim Partus studies included in the systematic review and also reports the sensitivity and specificity derived from the one PartoSure study providing test accuracy data against the 48-hour reference standard. For Actim Partus, meta-analyses were conducted.

Identification of studies

The same searches as for the review of diagnostic accuracy studies were conducted (see Chapter 2, Identification of studies). In brief, these included searches of electronic databases (these were designed to identify all studies assessing PartoSure, Actim Partus and qfFN), all systematic reviews identified by the electronic searches, trial registries, Google Advanced Search, reference lists of included DTA studies, studies citing the included DTA studies and industry submissions to NICE.

As with the review of DTA (see Chapter 2, Methods of the systematic review), screening for relevant studies was in two stages (screening of titles and abstracts and then screening of papers obtained in full). At both stages, screening was conducted concurrently with the screening for the review of test accuracy studies and this was done independently by two reviewers (two of JVC, SD, MB and HC). Prespecified inclusion and exclusion criteria were used (see Inclusion and exclusion criteria). Disagreements were resolved by discussion.

Inclusion and exclusion criteria

Other methods

Further aspects of the review methods (data extraction strategy, critical appraisal strategy and methods of data synthesis) are not described as there were no included studies.

Quantity and quality of research available

After screening 2623 items, no studies were identified that met the inclusion criteria for the review of clinical effectiveness. This was because none of the studies compared the tests of interest with a comparator regarding the clinical outcomes of interest; there were no studies identified in which some women received one test and some received another and even in the studies identified in the test accuracy review, in which women received more than one test (see Chapter 2, Summary), there was no clear indication that treatment decisions were based on the results of one test for some women and based on the results of the other test for other women. Indeed, these studies did not provide data on the clinical outcomes of interest.

Assessment of effectiveness

We were not able to draw any conclusions on the effectiveness of PartoSure, Actim Partus or fFN from the systematic review of end-to-end studies.

It is important to consider that this review was looking for evidence from controlled study designs. For the systematic review of clinical effectiveness, we did not look for evidence of clinical effectiveness from other designs (e.g. uncontrolled pre–post studies). This decision was made because these designs may be too open to bias to be worth including in a systematic review of end-to-end studies, even if they provide the only available evidence.

Identification of studies

The methods followed those reported in Chapter 2, Methods of the systematic review, for study identification. Studies were screened by two reviewers (RMM and JVC).

Inclusion and exclusion criteria

The population, index test and reference standard matched those reported in Chapter 2, Inclusion and exclusion criteria, and the inclusion criteria include the protocol amendment that twin or multiple pregnancies could make up 20% of the total population. However, for the review of economic evaluations, the criteria for inclusion permitted studies that reported health-care costs of an index test without restriction in terms of the design of the effectiveness study. In the following sections, any reference to exclusions owing to study design mean economic study design (i.e. exclusion of studies that are not economic evaluations).

Data extraction strategy

Data were extracted by one reviewer (RMM) using standardised data extraction templates.

Critical appraisal strategy

The quality of the studies was assessed in detail by an experienced health economist (RMM) in accordance with the criteria specified by the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) checklist. 64 This represents a deviation from the study protocol, which stated that the Drummond and Jefferson’s checklist would be used. 65

Methods of data synthesis

Data were narratively reported. Methods and results were tabulated using the prices and currencies as reported by the identified studies.

Critique

Although the study by Gibson et al. 67 did not aim to assess the cost-effectiveness of the different diagnostic strategies, it did provide information with which to model the cost-effectiveness of the following two sets of comparisons: (1) no test and treat all with steroids versus qualitative testing with fFN and treat those with positive results, and (2) testing options investigated at the qfFN thresholds of 10, 50, 200 and 500 ng/ml.

Given the available data from the study, the costs per patient adequately treated with steroids (i.e. within 7 days) were calculated by the assessment group (AG) using the following formula (for the comparison between no test and treating all with steroids):

Where C_ACS = cost of steroids, C_fFN = cost of fFN test, fFN₊ = probability of a positive test result, C_H = costs of hospital admission, N = total sample size, N_TA = number of mothers given ACS within 7 days and at least 1 day before delivery in the treat-all strategy, N_fFN = number of mothers given ACS within 7 days and at least 1 day before delivery in the testing strategy, fFN_F– = probability of a false-negative test result and NNT = number needed to treat to avoid one case of inadequate treatment without testing.

Thus the incremental cost per patient adequately treated is equal to the number needed to treat to avoid one case of inadequate treatment without testing (NNT) multiplied by the incremental cost per patient of treating all versus testing and treating positive cases. By combining the data from Gibson et al. 67 with treatment effectiveness data and cost and utility values used by previous models (e.g. NICE 2015,24 discussed below), we can obtain the incremental cost per case of RDS avoided, cost per life saved, and, subject to the natural reservations about projecting long-term outcomes, cost per quality-adjusted life-year (QALY) gained. The resulting formulae are:

where ARR_RDS = the absolute risk reduction (i.e. the difference between the absolute probability with and without appropriate ACS treatment administration) of RDS occurring, ARR_IVH = the absolute risk reduction of IVH occurring after steroid treatment, ARR_Death = the absolute overall death risk reduction (which includes the reduction in death mediated through RDS and IVH), D_IVH = the conditional probability of death in neonates with IVH, D_RDS = the conditional probability of death in neonates with RDS, Disu_IVH = the QALY loss from IVH and Disu_RDS = the QALY loss from RDS.

By adopting the values in the NICE guideline model24 discussed below and summarised in Table 11, one may calculate the relevant incremental cost-effectiveness measures using the diagnostic test results reported by Gibson et al. 67

These values result in an incremental cost per case of IVH avoided of £1,548,291, an incremental cost per case of RDS avoided of £466,622, an incremental cost per death avoided of £473,967 and an incremental cost per QALY gained of £20,942 with the ‘no-test and treat-all’ strategy relative to fFN. These figures do not account for any negative effects of inappropriate use of steroids on the infant’s health, and, therefore, may be considered a lower bound estimate.

Gibson et al. 67 found that variation of the threshold from 10 to 50 to 200 ng/ml resulted in the same number (two) of false-negative cases of women who delivered within 7 days of testing. In terms of cost-effectiveness analysis, this finding means that qualitative testing using the 10 ng/ml and 50 ng/ml thresholds is dominated by testing at the 200 ng/ml threshold because the latter results in lower resource use for testing and hospital admissions than the lower threshold strategies. However, moving from the 200 ng/ml to the 500 ng/ml threshold resulted in one additional missed preterm birth case. 67 Comparing the qfFN test with the 200 versus the 500 ng/ml threshold, the AG calculates incremental costs per event avoided of £221,115, £770,000 and £235,714 for the RDS, IVH and death outcomes, respectively. The incremental cost per QALY is £10,415. Therefore, at the £20,000 NICE cost-effectiveness threshold, the optimal, cost-effective diagnostic strategy is to use the qfFN with a threshold of 200 ng/ml.

It is evident from the small numbers of false-negative cases presented above that the findings from the study by Gibson et al. 67 are highly uncertain. This also highlights the need for evidence synthesis over multiple studies in order to derive meaningful evidence.

We highlight that the above formulae (equations 7–11) allow for a separate treatment of the costs of hospital admission and steroid treatment, in contrast to other models discussed below. This may be important because diagnostic guidelines or protocols being used in some centres [e.g. Guy’s and St Thomas’ Hospital (London)] suggest that the fFN threshold concentrations used by clinicians in the obstetrics department to decide when to admit a patient may be different from those used by them for deciding when to administer steroids. Therefore, the cost of treatment (i.e. the sum of steroid costs, C_ACS, and hospital costs, C_H) may vary across different qfFN thresholds.

Summary

One abstract was identified that investigated some measure of costs or cost-effectiveness of the interventions of interest to this assessment. In this section, we have reviewed the abstract67 and corresponding dissertation68 that reported results in terms of the number needed to treat to achieve a desired neonatal outcome, and we have shown how these data may be used in conjunction with the literature to derive useful information about the cost-effectiveness of different thresholds for the qfFN test. We note that current treatment protocols in some hospitals may allow for the use of different qfFN thresholds to decide whether or not to administer steroids and admit to hospital.

The National Institute for Health and Care Excellence 2015 guidelines model

The RCOG developed a decision-analytic model to inform the NICE 2015 guidelines24 on the diagnosis and treatment of preterm labour and birth. The authors of this model concluded that the quality of the diagnostic accuracy data was low for the different tests considered relevant at the time for women presenting with symptoms of preterm labour (cervical length measurement by ultrasound scan, Actim Partus, qualitative fFN). Consequently, they presented a ‘what-if’ analysis comparing the testing versus the no testing and treat-all strategies, which consisted of identifying the levels of specificity and sensitivity at which a hypothetical test became cost-effective in accordance with the NICE cost-effectiveness threshold of £20,000 per QALY gained.

Unlike previous analyses, the NICE evaluation accounted for the effect of gestational age on the trade-off between sensitivity and specificity (i.e. costs of treating more patients unnecessarily versus missing patients at high risk of neonatal adverse events, including death). This analysis set the cost of the test equal to that of cervical length measurement, and found that testing was not cost-effective at gestations of < 30 weeks. This served as the basis of the NICE recommendations about the use of testing to rule out preterm labour.

The NICE guidelines model structure is illustrated in Figure 7. In accordance with this model, the causal pathway from diagnostic results to neonatal outcomes is mediated by tocolysis treatment, which can delay premature delivery by ≥ 48 hours. This would generate a window of opportunity for appropriate steroid administration (i.e. ≥ 24 hours and up to 7 days before delivery) and transfer to a tertiary hospital, thus reducing the risk of RDS, IVH, and death. The risk reduction parameter values used in this model are based on treatment effect estimates for calcium channel blockers versus placebo, from three separate network meta-analyses of RCTs (one per model outcome) (NICE 2015). 24 It is worth noting that two of these treatment effect parameters, the odds ratio (OR) for death, 0.62 (95% CI 0.21 to 1.80), and the OR for RDS, 0.81 (95% CI 0.50 to 1.34), are imprecisely estimated. The estimated OR for IVH was 0.40 (95% CI 0.21 to 0.74). None of these estimates used data from a direct head-to-head RCT. Furthermore, these treatment effects were assumed to be constant across the gestation in the model (24 to 34 weeks), so that the absolute risk reduction (ARR = relative risk × baseline risk) for the three types of event with tocolysis varies by gestation only because the baseline (i.e. without tocolysis) risk declines with gestational age (Figure 8 and see Table 11). IVH and RDS each contribute to the risk of neonatal mortality; the probability of death is 0.300 conditional on the former and 0.054 conditional on the latter. Infants who did not die following these adverse events would contribute additional costs and QALY losses over their expected lifetime.

FIGURE 7.

The NICE 2015 guidelines model. 24 Reproduced from: Royal College of Obstetricians NICE Guideline 25 Preterm Labour and Birth, London, ROCG, November 2015, with the permission of the Royal College of Obstetricians and Gynaecologists. 24

Reproduced from: Royal College of Obstetricians NICE Guideline 25 Preterm Labour and Birth, London, ROCG, November 2015, with the permission of the Royal College of Obstetricians and Gynaecologists. 24

FIGURE 8.

Baseline (without tocolytics) risks in the NICE guideline model. Reproduced from: Royal College of Obstetricians NICE Guideline 25 Preterm Labour and Birth, London, ROCG, November 2015, with the permission of the Royal College of Obstetricians and Gynaecologists. 24

Reproduced from: Royal College of Obstetricians NICE Guideline 25 Preterm Labour and Birth, London, ROCG, November 2015, with the permission of the Royal College of Obstetricians and Gynaecologists. 24

The analysis projected lifelong QALY values based on the neonatal adverse events. Infants who survived the neonatal phase and were discharged home were assumed to have average expected QALYs that varied with gestational age from 19.92 at 24 weeks to 22.61 at 34 weeks. These were calculated as the gestational age-specific proportion of infants surviving the first year of life multiplied by the expected QALYs of these infants, which, in turn, was equal to the life expectancy in England and Wales of 80 years valued at the population norm health-state utility of 0.82 and discounted at an annual rate of 3.5%. 77 Deaths in the first year of life were assumed to generate zero QALYs. In the event of RDS, an average QALY loss would apply, arbitrarily set at 3.85 (based on RDS providing a slightly lower QALY loss than IVH). The occurrence of IVH incurred an average QALY loss of 4.5, based on the assumption that the IVH would incur the same quality-of-life loss as intracranial haemorrhage (ICH), which, in turn, was assumed to incur one-third of the QALY loss of the cerebral palsy value reported by Cahill et al. 78

The analysis accounted for the costs of tests, including test acquisition and staff time, drug treatment [nifedipine at a loading dose of 40 mg and subsequent dose of 240 mg, at British National Formulary (BNF) prices of £0.008 per 1 mg, from a 90-capsule pack of 10 mg] and administration (5 minutes of doctor and 5 minutes of nurse time) and downstream neonatal hospital costs of adverse events (RDS and IVH). The downstream RDS costs were set at the NHS Reference Costs of neonatal intensive care unit (NICU) care [British Association of Perinatal Medicine (BAPM) level 1] with extracorporeal life support/extracorporeal membrane oxygenation, and the costs of IVH were assumed to be equal to the lifetime health-care costs of ICH, which, in turn, were assumed to be equal to the health-care costs of severity grade III or grade IV cerebral palsy. 79 Regardless of how valid these clinical assumptions are, the cost of IVH appears to be underestimated, as it was calculated with a higher discount rate than the 3.5% recommended by NICE, and underweighted:

It was additionally assumed that Grade III and Grade IV ICH would be similar in cost to cerebral palsy. A European paper80 estimated in year 2000 prices that the lifetime healthcare costs for cerebral palsy using an annual discount rate of 5% was €66,155 for men and €65,288 [for women]. The mid-point of this estimate was used and converted into GBP [Great British pounds] using an exchange rate of £0.83 = €1 . . . It was then converted into 2011/12 prices using the HCHS (Hospital and Community Health Service) Index. One study81 suggested that 30% of ICH is of severity Grade III and Grade IV and therefore the cost of ICH was estimated as 0.3 × £79,000.

© NICE 2015. 24 Preterm Labour and Birth. Available from www.nice.org.uk/guidance/ng25. All rights reserved. Subject to Notice of rights. NICE guidance is prepared for the National Health Service in England. All NICE guidance is subject to regular review and may be updated or withdrawn. NICE accepts no responsibility for the use of its content in this product/publication

The parameter estimates for the different elements of cost appeared to be estimated in prices of different years. Treatment costs were expressed in 2015 prices, costs of drug administration were in a price year prior to 2014 and adverse events were in 2011/12 (IVH) and 2012/13 (RDS) prices. The long-term adverse event cost of IVH was derived from a European study that reported results in 2000 euros,80 converted to GBP (Great British pounds) using the exchange rate of £0.83 = €1.00 (the year this rate applied to was not provided), and reflated to 2011/12 prices.

The key assumptions of the NICE guidelines model are summarised in Table 13. As in other models in this literature, maternal outcomes are not measured. The model assumes full adherence to the diagnostic protocol, thus abstracting from individual disparities in clinician behaviour. Critically, the model assumes that all patients with positive test results are treated with tocolytics, which may not happen in routine practice (Professor Andrew Shennan, King’s College London, 2017, personal communication). The model does not explicitly account for the use and effect of corticosteroids, only implicitly within the treatment effect estimates obtained from the network meta-analysis of tocolysis studies discussed previously in this section. This feature makes the model less suitable for obtaining generalisable results for situations in which the corticosteroids are used without tocolytic therapies. A major limitation of the analysis is the high degree of uncertainty associated with the calculation of costs and QALYs, which were extrapolated to lifetime values from neonatal morbidity outcomes. Thus, this model’s advantage in terms of producing results in terms of QALYs for informing NICE decisions may have come at the cost of heroic assumptions about the ability to predict lifetime costs and benefits from neonatal outcomes. In fact, the extrapolation was inadequately calculated (by multiplying a life expectancy times a constant population norm) because it did not account for the survival curve profile in population life tables and the varying utility with age,84 and the utility norms were derived from a study that predates the time that EuroQol-5 Dimensions (EQ-5D) scores were developed. It is unclear if by choosing to model treatment based on tocolysis as opposed to steroids the model failed to account for outcomes in terms of other neonatal adverse events, such as necrotising enterocolitis, sepsis and retinopathy. On the other hand, the model’s ability to account for outcomes by gestation at presentation make this model the most relevant among those available for guiding clinical decisions on individual patients, because other models did not produce results by length of gestation. A summary of the main features of the model is provided using the CHEERS checklist64 in Table 14.

Model structure

In common with all previous studies (see Chapter 5), we used a decision tree to model the economic evaluation of the diagnostic choice problem. As in the model supporting the 2015 NICE guidelines on diagnosis and treatment of preterm labour,24 the only case that included both of these aspects of patient management, our model includes an initial diagnostic phase followed by treatment and long-term outcomes. Although our protocol stated the plan to use a decision tree model for the diagnostic phase and another decision tree for the treatment phase, the available data did not allow us to populate the decision tree for the treatment phase. Therefore, our model is a decision tree with lifetime costs and QALY pay-offs. The model accounts for the costs incurred starting from the time women present to a maternity hospital with symptoms suggestive of preterm labour, through hospital admission or discharge home, to neonatal discharge or death in hospital. The health consequences to the offspring are measured in terms of QALYs based on neonatal morbidity and mortality outcomes. The main features of our model are that it:

• accounts for the costs and QALYs of the infant (as well as QALYs for the mother, in a scenario analysis)

• differentiates costs and benefits by gestational age

• distinguishes between hospital levels of designation – level 3 hospitals have NICUs providing all types of care to the local population and care for the most severe infants transferred in utero or after birth, level 2 hospitals or local neonatal units provide all types of neonatal care except long-term intensive care and care for complex cases and level 1 hospitals provide specialised care for their local population, stabilisation and non-invasive monitoring

• accounts for the costs and benefits of steroids and the costs of tocolysis and hospital transfer for neonatal transfers

• determines long-term QALYs and costs by neonatal morbidity (RDS and IVH) and mortality outcomes.

The model builds on that used to inform the NICE 2015 guidelines24 on the diagnosis and treatment of preterm labour, as the only prior model allowing for variation in health risks, and thus costs and benefits, of inaccurate diagnosis by gestational age. By adopting this general structure, we are able to account for the increasing neonatal health risks posed by an attending obstetrician’s failure to identify a woman in preterm labour earlier in gestation. Unlike the NICE model, which assumed that the diagnosis of preterm labour was intended to guide the decision of whether or not to administer tocolysis, we model the treatment pathway following a diagnosis of preterm labour around the decision of whether or not to treat with corticosteroids and/or admit to hospital or discharge home. This methodological variation in our approach is motivated and informed by the very recent evidence quantifying the positive effects of antenatal corticosteroid (ACS) administration for accelerating the maturation of the fetus’s lungs as a function of the time of administration relative to delivery, the limited use of tocolysis reported in the literature and the emerging consensus on its potential risks to the infant and side-effects to the mother. 30,33,34,87 For example, audit data for the period from September 2016 to May 2017 from the level 2 hospital in Exeter show that tocolysis was administered in only one out of nine patients (11%) presenting with symptoms of preterm labour at 24–34 weeks with fFN of ≥ 50 ng/ml. In our analysis, we assume that tocolysis is used only for all in utero transfers at gestational ages of < 28 weeks (see Results).

Given the importance of the timing of ACS administration, the aim was to develop a model capable of accounting for the different diagnostic test options’ capabilities to distinguish between those women likely to deliver imminently following presentation and those who would deliver preterm in a week or later, among presumptive cases of preterm labour. Despite our initial aim to explicitly model treatment administration at intervals of < 2 days, 2–7 days and > 7 days before delivery, few studies of DTA reported outcomes for the time-to-delivery interval of < 2 days (see Chapter 2, Results of the systematic review). These intervals have been discussed in the literature as most relevant to ACS effectiveness, with 7 days before delivery considered the earliest time for effective use of steroids in terms of fetal and neonatal mortality and RDS. 88,89 The latest evidence suggests that the effectiveness of ACSs in terms of mortality risk reduction may be optimal within 2 days of delivery, and that it diminishes with time before delivery. This observation also applies to the risk of RDS and IVH. 33,87 Thus, we decided on the model structure illustrated in Figure 9.

FIGURE 9.

The Peninsula Technology Assessment Group cost-effectiveness model structure of diagnosis and treatment of women with symptoms of preterm labour. The dotted line indicates that the subsequent branch structure after ‘fFN’ is identical to the structure shown for ‘New test’. New test is patient management in accordance with one of the interventions or ‘index tests’. fFN is the comparator (admit and treat when fFN is ≥ 50 ng/ml) status quo. We also consider the no-test, treat-all comparator. Greek and Latin letters are parameters populated with data from diagnostic accuracy test studies. π, pre-test probability of preterm birth in ≤ 7 days; p, test sensitivity of PTB in ≤ 7 days; µ, pre-test probability of PTB; r, test specificity of PTB. PTB, preterm birth.

This structure shares the features of previous models of diagnosis and treatment and is determined by (1) the available DTA data and (2) the latest evidence on the time window relative to delivery when ACS treatment is most effective. The model assumes that the decision to admit and treat or transfer to another hospital is driven by the test result (positive or negative). The model makes a distinction in terms of effectiveness between diagnostic tests in accordance with their ability to correctly predict whether or not a woman will deliver before term, and whether or not that will happen before or after 7 days.

In this model, the costs and health benefits of following one of at least two mutually exclusive courses of action for managing a woman presenting with signs and symptoms of preterm labour are evaluated. Figure 9 shows that a new test (PartoSure, Actim Partus and qfFN used qualitatively at thresholds other than 50 ng/ml) or the no-test, treat-all strategy may be compared against the status quo of qualitative fFN (or qfFN used qualitatively at a 50 ng/ml threshold). The starting point of the model is when the decision between diagnostic strategies is made, that is, immediately after clinical assessment of symptoms that have not ruled out preterm labour. Thus, in the absence of further testing, all women would be admitted or transferred to another hospital. A woman tested may deliver a preterm baby within 7 days of testing or may deliver in more than 7 days from the time of testing; if the latter happens, birth may be preterm (before 37 weeks’ gestation) or full term. Therefore, women may be classified in one of these three subgroups in accordance with the time of delivery. Within each of these, the results of the new test will determine how the patient is managed, and consequently the woman’s ability to benefit from ACS treatment. Thus, if a woman tests positive, the obstetrician would be expected to treat her by admitting her to the hospital and administering steroids (under fFN testing, in some hospitals women may be admitted for observation above one threshold and admitted and administered steroids at another higher threshold: we do not consider this case). The model distinguishes the type of hospital setting by level of specialisation: if a woman attends a tertiary level hospital at < 28 weeks’ gestation and tests positive, she is admitted into hospital, whereas if testing takes place at a lower-level hospital, she would be given tocolysis (we assume that tocolysis is considered only for women undergoing in utero transfer at < 28 weeks’ gestation) and transferred to a level 3 hospital for her care. Women who test negative are sent home without treatment; owing to a lack of any test-specific data on this parameter, we do not allow for partial compliance with treatment guidelines in contrast with what is suggested in Figure 1. The same structure is assumed for the status quo ‘fFN’ testing option with one and the same threshold for admission and treatment.

In accordance with the model in Figure 9, a symptomatic woman who goes on to deliver within 7 days has a positive test result with probability p (the sensitivity of the test) and a negative test result with probability 1 – p (the false-negative rate). Among women who deliver after 7 days, the probability of a positive test result is equal to the false-positive rate (FPR), and the probability of a negative test result equals the test specificity for delivery within 7 days of testing.

Some women have a positive test result, receive treatment and deliver after 7 days of testing, but before 37 weeks of gestation. In the base-case analysis, we assume that ACS produces no benefit when administered > 7 days before preterm birth. 88,89 In scenario analyses, these women are assumed to benefit from ACS, but less so than those who are treated within 7 days of preterm delivery33,87 (we assume throughout that no multiple courses of steroids are given, based on obstetricians’ advice on routine practice and the perceived lack of proven benefit and risks to the neonate). The frequency of such cases is calculated as:

where P(PTB > 7 days and +ve result) is the probability of having a positive test result and delivering preterm > 7 days after testing, ρ is the incidence of delivery within 7 days, Specificity_7d is the test specificity for delivery within 7 days and r is the test specificity for delivery at < 37 weeks (i.e. the proportion of women testing negative among those who deliver after 37 weeks), which has an incidence of ρ_37w. Thus the proportion of women who receive treatment more than 7 days before preterm birth (and, therefore, derive partial benefit from ACSs) is equal to the difference between the FPR for delivery within ≤ 7 days and the FPR for delivery is < 37 weeks, weighted by their respective incidences. The benefit from ACSs for this group of women is also reduced by the fact that the baseline mortality and adverse event (IVH and RDS) risk of preterm birth is lower, as the infant is delivered at a higher gestational age, than for mothers who deliver within 7 days. We assume that delivery takes place at the midpoint between gestation at presentation and 36 weeks.

Population

The population was defined as women presenting with symptoms of threatened preterm labour (abdominal pain and contractions) with intact membranes between 24 and 36 weeks’ gestation for whom a transvaginal ultrasound scan was not available or acceptable.

Interventions and comparators

We evaluated the following diagnostic test strategies immediately following an initial clinical investigation that had not ruled out preterm labour:

• testing with PartoSure

• testing with Actim Partus

• testing with qfFN at thresholds of 10, 200 and 500 ng/ml

• comparator testing with fFN at 50 ng/ml, from quantitative or qualitative versions of the test device

• treat all without testing (i.e. because clinical investigation could not attribute symptoms to other causes, women are managed as presumptive cases of preterm labour).

These were the options for which evidence was available in the literature. Combinations of these options were not considered, as they were not part of the NICE scope.

In addition, we explored the scenarios of evaluating (1) qualitative fFN in accordance with current treatment protocols in Guy’s and St Thomas’ women’s hospital, where different thresholds are used to admit to hospital (at 50 ng/ml) and treat with steroids (200 ng/ml), and (2) qfFN (qfFN) as observed at the level 2 maternity hospital in Exeter. 90 These additional analyses are intended to reflect the spectrum of variation in local current practice across the country.

Perspective, time horizon and discounting

The analysis adopted the perspective of the NHS and Personal Social Services. In accordance with the requirements of the NICE methods guide, the time horizon is taken as the entire lifetime, and the projected long-term health-care costs and utilities associated with avoiding an adverse neonatal outcome (death, RDS and IVH) were measured. 91 All previous models of diagnosis in preterm labour assumed much shorter time horizons (up to neonatal death or discharge from hospital), except for the model informing the NICE 2015 guideline in this area24 (see Chapter 4 for a critique of these models). An annual discount rate of 3.5% for costs and benefits was used, as set by NICE. We present results limited to neonatal death or discharge from hospital in scenario analyses.

Model parameters

The model consists of two parts, the diagnostic phase and the treatment phase, each with a characteristic set of parameters and sources of evidence. The diagnostic phase parameters are populated from diagnostic accuracy studies of the interventions of interest, complemented by data on patient management, such as admission rates conditional on test results, which are obtained from audit data or modelling studies. The treatment phase is derived from large observational studies of the effects of steroids on neonatal health outcomes and evidence synthesis of RCTs of ACS treatment. Cost estimates are obtained from detailed costing studies in individual hospitals or routine national sources, and health-related quality-of-life utilities have been obtained from observational and health-state preference elicitation studies.

Treatment effectiveness and extrapolation

Diagnostic test accuracy

We limit our analyses to evaluate the diagnostic tests assessed by individual comparative diagnostic accuracy studies, as identified in Chapter 2. These studies are:

• APOSTEL-1, comparing Actim Partus with qualitative fFN at the 10, 50, 200 and 500 ng/ml thresholds (Bruijn et al. 42,43)

• a comparison of Actim Partus with PartoSure (Hadzi-Lega et al. 44).

Other studies were identified as providing relevant data but they were of lower quality than the data from these two studies. One was a comparison of Actim Partus with fFN at 50 ng/ml. 49 The study by Cooper et al. 49 was the second largest of the three identified studies that compared Actim Partus with fFN at 50 ng/ml and reported test accuracy data for delivery within 7 days and at < 37 weeks (see Chapter 2, Results of the systematic review). However, as discussed in Chapter 2, it was unclear what version of the fFN test was used, but the ELISA version of the qualitative fFN test, which is no longer used in clinical practice, was presumed to be used. Therefore, we consider this study in scenario analyses, thus limiting the base-case analysis to include only the APOSTEL-1 study, which evaluated a non-laboratory-based fFN test, and the study by Hadzi-Lega et al. 44 In scenario analyses, we also evaluated the comparison of Actim Partus with fFN at 50 ng/ml (non-ELISA) tests based on a meta-analysis of 7-day results reported by four comparative studies of these technologies. 42,43,51,56,57 We did not consider this meta-analysis in the base case owing to the heterogeneity between the combined studies, especially in terms of preterm birth rates, which probably drove their differences in test accuracy results. Furthermore, the pooled results for each index test presented in Chapter 2, Results of quantitative data synthesis (test accuracy data), were not considered in the economic analysis, because comparisons between tests in terms of those results are probably confounded by the heterogeneity between the studies.

In addition, although it was excluded from the assessment of test accuracy section (see Chapter 2) because it did not provide published test accuracy data within 7 days, we evaluated the diagnostic test considered in the only UK study (Abbott et al. 92) using data provided by the study authors for this review (Professor Andrew Shennan, King’s College London, 2017, personal communication). This test was an assessment of the Rapid fFN 10Q analyser (Hologic, Inc.) at the qualitative thresholds of 10, 50, 200 and 500 ng/ml.

Table 15 summarises the diagnostic test options compared in cost-effectiveness analyses, and the diagnostic accuracy study sources used to populate the economic model.

TABLE 15 - Diagnostic test options compared in cost-effectiveness analyses and their sources

Diagnostic test option	Comparator/intervention	Base-case analysis		Scenario analysis
		APOSTEL-142,43	Hadzi-Lega et al.44	Cooper et al.49	Abbott et al.92	Meta-analysis42,43,51,56,57
PartoSure	Intervention		✓
Actim Partus	Intervention	✓	✓	✓		✓
Rapid fFN 10Q Cassette Kit thresholds other than 50 ng/ml	Intervention	✓			✓
fFN, threshold of 50 ng/ml	Comparator	✓		✓	✓	✓
No test, treat all	Intervention	✓	✓	✓	✓	✓

The sensitivities and specificities used for these analyses are presented in Table 16. Two sets of accuracy parameter values for each study were required for the model, one for predicting delivery within 7 days and another predicting delivery before 37 weeks’ gestation. We could not obtain 37-week data for one of the studies involving the comparison of fFN with Actim Partus (Bruijn et al. 42,43). For this study, therefore, we imputed specificity at 37 weeks from the corresponding 7-day specificity rate so as to obtain a 37-week FPR aligned with the UK study of fFN and the Italian study of Actim Partus. 54,92 Similarly, we imputed 37-week sensitivity values for the analysis of Actim Partus versus fFN (50-ng/ml threshold) based on our meta-analysis of 7-day accuracy data, using the Italian study data. 54 We varied these values in sensitivity analyses.

TABLE 16 - Diagnostic accuracy values used in the Peninsula Technology Assessment Group model

Assumption: the FPR for delivery before 37 weeks was 6% lower than the FPR for delivery within 7 days of testing.

Assumption: the FPR for delivery before 37 weeks was 11% lower than the FPR for delivery within 7 days of testing.

Assumption: the FPR for delivery before 37 weeks was 18% lower than the FPR for delivery within 7 days of testing.

Assumption: the FPR for delivery before 37 weeks was 35% lower than the FPR for delivery within 7 days of testing.

7-day FPR of test multiplied by ratio of FPR for delivery before 37 weeks relative to FPR for delivery in ≤ 7 days of corresponding test in Riboni et al. 54

Imputed: 7-day FPR of test multiplied by ratio of FPR for delivery before 37 weeks relative to FPR for delivery in ≤ 7 days of corresponding test in Cooper et al. 49

Note

Further details are provided in Chapter 2, The 7-day delivery reference standard, and Table 6.

Study (first author and year)	Delivery within 7 days						Delivery in < 37 weeks
	Diagnostic test	N	Sensitivity	Specificity	Probability distribution sensitivity [Beta(α,β)]	Probability distribution specificity [Beta(α,β)]	Specificity
Bruijn (2016) (APOSTEL-1)42,43	fFN at 10 ng/ml	350	0.957	0.423	Beta(66,3)	Beta(119,162)	0.458a
	fFN at 50 ng/ml	350	0.913	0.648	Beta(63,6)	Beta(182,99)	0.686b
	fFN at 200 ng/ml	350	0.710	0.836	Beta(49,20)	Beta(235,46)	0.866c
	fFN at 500 ng/ml	350	0.420	0.957	Beta(29,40)	Beta(269,12)	0.972d
	Actim Partus	350	0.783	0.893	Beta(54,15)	Beta(251,30)	0.929e
Hadzi-Lega (2017)44	PartoSure	57	0.833	0.902	Beta(5,1)	Beta(39,12)	0.919f
	Actim Partus	57	0.833	0.765	Beta(4,1)	Beta(46,5)	0.764f
Cooper (2012)49	Actim Partus	349	0.333	0.741	Beta(2,4)	Beta(254,89)	0.740
	fFN at 50 ng/ml	349	0.333	0.898	Beta(2,4)	Beta(256,29)	0.946
Abbott (2013)92	Confidential information has been removed	Confidential information has been removed	Confidential information has been removed	Confidential information has been removed	Confidential information has been removed	Confidential information has been removed	Confidential information has been removed
	Confidential information has been removed	Confidential information has been removed	Confidential information has been removed	Confidential information has been removed	Confidential information has been removed	Confidential information has been removed	Confidential information has been removed
	Confidential information has been removed	Confidential information has been removed	Confidential information has been removed	Confidential information has been removed	Confidential information has been removed	Confidential information has been removed	Confidential information has been removed
	Confidential information has been removed	Confidential information has been removed	Confidential information has been removed	Confidential information has been removed	Confidential information has been removed	Confidential information has been removed	Confidential information has been removed
Meta-analysis by AG42,43,51,56,57	Actim Partus	963	0.832	0.879	Beta(150,30)	Beta(689,94)	0.920e
	fFN at 50 ng/ml	963	0.683	0.872	Beta(123,57)	Beta(683,100)	0.909e

Figure 10 depicts the differences in diagnostic accuracy parameters for predicting delivery within 7 days across individual studies in Table 16 by index test and, for qfFN, by fFN threshold. The results from the study by Cooper et al. ,49 which compared Actim Partus with fFN at 50 ng/ml, are the two outlying points at the bottom of the graph; as discussed in Chapter 2, it is unclear whether this study used a laboratory or non-laboratory fFN test. 49 Because none of the new tests appears to be superior in terms of both accuracy measures, adopting any one of them implies a trade-off of specificity against sensitivity relative to fFN at 50 ng/ml. Not reflected in the figure is the extent of sampling uncertainty, which may be exemplified by the case of the data for PartoSure, derived from a single study of 57 subjects. 44

FIGURE 10.

Empirical summary ROC points across evaluated studies (7-day). These data (apart from those from Abbot et al. 92) come from test accuracy studies assessed in Chapter 2 and are therefore subject to the issues discussed in relation to the quality of evidence (the implications of the quality of these data for the economic conclusions are discussed in Chapter 9).

Background neonatal risks parameter values

Our model included underlying risks of neonatal mortality and adverse events in terms of RDS and IVH, similar to the model that informed the NICE 2015 guidelines24 on preterm labour diagnosis and treatment, but with data adjusted for steroid use in routine practice (Table 17 and Figure 11). We use the latest estimates of baseline mortality risks by gestational age from the ONS. 13 The risks of RDS and IVH were derived from Medscape data compiled by Michael Ross (author; MD, MPH Distinguished Professor of Obstetrics and Gynecology, University of California, Los Angeles, David Geffen School of Medicine; Distinguished Professor, Department of Community Health Sciences, Fielding School of Public Health at University of California at Los Angeles), which are available at https://emedicine.medscape.com/article/260998-overview#a5 (accessed November 2017).

TABLE 17 - Baseline risk of neonatal adverse events

Note

Source: UK stillbirth and neonatal mortality rates (ONS 201613) and US data from Medscape (https://emedicine.medscape.com/article/260998-overview#a5; accessed November 2017). The adjusted analysis used data on ACS treatment effects from Travers et al. 93 and steroid use in routine practice from the Neonatal National Audit Programme. 94

Gestational age (weeks)	Adverse event
	Death		RDS		IVH
	Event probability	Probability distribution [Beta(α,β)]	Event probability	Probability distribution [Beta(α,β)]	Event probability	Probability distribution [Beta(α,β)]
24	0.57	Beta(571,163)	0.70	Beta(408,326)	0.25	Beta(251,483)
25	0.44	Beta(480,244)	0.90	Beta(590,134)	0.30	Beta(309,415)
26	0.32	Beta(403,424)	0.93	Beta(695,132)	0.30	Beta(468,359)
27	0.24	Beta(362,537)	0.84	Beta(650,249)	0.16	Beta(280,619)
28	0.20	Beta(402,731)	0.65	Beta(650,483)	0.04	Beta(71,1062)
29	0.13	Beta(310,975)	0.62	Beta(818,467)	0.04	Beta(78,1207)
30	0.10	Beta(217,1368)	0.55	Beta(808,777)	0.02	Beta(59,1526)
31	0.08	Beta(318,1715)	0.37	Beta(776,1257)	0.02	Beta(60,1973)
32	0.05	Beta(236,2653)	0.28	Beta(771,2118)	0.01	Beta(40,2849)
33	0.04	Beta(320,3738)	0.34	Beta(1414,2644)	0.00	Beta(0,4058)
34	0.03	Beta(429,6368)	0.14	Beta(892,5905)	0.00	Beta(0,6797)
35	0.02	Beta(516,9518)	0.12	Beta(1128,8906)	0.00	Beta(0,10034)
36	0.01	Beta(547,19561)	0.07	Beta(1319,18789)	0.00	Beta(0,20108)
37	0.01	Beta(687,43773)	0.03	Beta(1458,43002)	0.00	Beta(0,44460)

FIGURE 11.

Baseline risks of neonatal adverse events. Source: UK stillbirth and neonatal mortality rates (ONS 201613) and rates of RDS and IVH in US data from Medscape (https://emedicine.medscape.com/article/260998-overview#a5; accessed November 2017).

The baseline risk values in the model are intended to measure neonatal risks in the absence of ACS treatment; thus, the values in Table 17 have been adjusted to subtract the effect of steroids use in routine practice, using the formula:

⁽¹³⁾ Baseline risk = Unadjusted risk 1 + P ANS × ( RR ANS – 1 ) ,

where ‘Unadjusted risk’ is the risk estimate as reported in the data source, P_ANS is the prevalence of ACS use in routine practice, RR_ANS is the relative risk of mothers given ACSs relative to those not given ACSs and ‘baseline risk’ is the adjusted risk estimate for each outcome and gestational age reported in Table 23. The adjustment acknowledges the fact that the observed risk in the national statistics is a weighted average of the risk of those who receive and do not receive ACSs, in which the weights are given by the proportion of women receiving and not receiving ACSs. At the lowest extreme, the baseline risk will be equal to the unadjusted risk when no women are treated with ACSs or when ACSs have no effect on the risk (i.e. RR_ANS = 1) and increases with ACS use and the effectiveness of ACSs to a maximum of 1/(1 – P_ANS) multiplied by the unadjusted risk (i.e. when relative risk = 0). According to the National Neonatal Audit Programme (NNAP),94 the most representative data source on ACS use in England, Scotland and Wales, 83% of mothers of babies born between 24 and 34 weeks’ gestation in 2013 (the year of our neonatal mortality data) were given at least one dose of ACS. Because NNAP does not produce data by gestational age, we assume such value applies to all gestational ages. We identified one study (by Travers et al. 93) that reports treatment effects of ACSs by gestational age (range 23 to 34 weeks) for death before discharge and used that to derive RR_ANS (Grant95) for neonatal death in Table 23; we adopted the ACS treatment effects on severe ICH by gestational age from the same source to approximate the RR_ANS for IVH. We could not find estimates for ACS treatment effects on RDS by gestational age and, thus, assumed a constant value of RR_ANS for this outcome, from the source described in Steroid treatment (Travers et al. 93 report treatment effects estimates for bronchopulmonary dysplasia, but the effects were so imprecisely estimated that point estimates implied that the RR_ANS was > 1 for all but three gestational ages).

Because some IVH and RDS cases result in fatality, to calculate the number of infants who live to adulthood with these conditions we follow the NICE 2015 guideline model24 and multiply the incidence of RDS and IVH by 1 minus the probability of neonatal mortality among neonates with these events, which is assumed to be constant across gestational ages. We searched the literature and identified a new source of data on the probability of death related to IVH, which we adopted in our model. 96 We found no new data on the probability of neonatal death among RDS cases and, thus, used the value from the NICE guidelines,24 which was obtained from US data for 2004 [Centers for Disease Control and Prevention (CDC) 200797]. The values are 0.054 [beta(875,15393) in probabilistic sensitivity analyses] for RDS and 0.205 [beta(76,394)] for IVH.

Steroid treatment

In the model, women who test positive are treated with ANSs. In the scenarios in which women present at level 1 or 2 maternity hospitals at < 28 weeks’ gestation, they also receive tocolysis and are transferred in utero to a level 3 unit. The model does not account for any possible effects of tocolysis in terms of delaying preterm delivery.

For our base-case analysis, we used treatment effects parameter values for ACS administration from results reported by the Effective Perinatal Intensive Care in Europe (EPICE) study,33 a prospective cohort study that collected data from 19 regions in 11 European countries in 2011 and 2012. This study was selected as the largest and most representative source of data on ACS effectiveness in reducing neonatal mortality and morbidity of very preterm infants by time to delivery. The EPICE study produced an analysis of the association of administration-to-birth intervals with morbidity and mortality in 4594 infants born at gestational ages of between 24 and 31 weeks. Given its large sample size, the study was able to analyse the outcomes associated with corticosteroids given a few hours before birth relative to outcomes at longer administration-to-birth intervals. The study concluded that ACSs may be effective even when administered up to 3 hours before delivery, which was expected to reduce mortality relative to no ACS by 26%. 33 The authors reported that 77.9% of the 1111 women who received ACSs < 24 hours before delivery received only one dose of ACS. Treatment effects on IVH were also derived from this source. For treatment effects on RDS, we used data from the Cochrane Database Systematic Review of RCTs of the effectiveness of ACSs relative to no treatment or placebo,34 which was also the source of values for sensitivity analyses. The main findings from the Cochrane review are summarised in Table 18. Because subgroup analysis produced no evidence that rupture of membrane status led to different rates of neonatal death, fetal death, RDS, IVH or birthweight in infants exposed to corticosteroids, we decided to use the overall treatment effect estimates in our model.

The effectiveness of steroids depends on the time from ANS administration to delivery. Table 19 displays the treatment effect model parameter values for the base-case and sensitivity analysis, which reflect the reduced effects of ACSs when given ≤ 7 days before birth.

Health-related quality of life

We conducted a systematic search of the literature for utility values of neonatal outcomes in the model: mortality, RDS and IVH. The details of the search strategy, identification and data extraction from the identified studies are provided in Appendix 4, in which additional tables highlight the parameter values used in the model. In this section, we summarise our findings.

Summary of identified studies

A total of 28 studies were identified from screening full texts as containing information useful for obtaining or deriving utility parameters for the model, given the populations studied (i.e. either preterm children or mothers). These studies are broadly summarised in Appendix 4.

Of these 28 studies, 24 assess the outcomes of children born preterm. The details of these studies are summarised in Appendix 5, Table 39. Nine additional papers were cited as sources for utilities in some of these studies. Parameter values from these additional papers are presented in Appendix 5, Table 40. The remaining four papers assess the outcomes of mothers; these studies are summarised in Appendix 5, Table 41.

Short Form questionnaire-36 items mapping and extraction of utilities

None of the studies that were found directly measured utilities based on the EQ-5D. However, various mapping functions exist that allow Short Form questionnaire-36 items (SF-36)99 summary measures to be converted into EQ-5D utilities. 100 We made use of a mapping function obtained from Rowen et al. 101 to undertake this conversion, as it was deemed the most appropriate study based on regression variables and the population sample used. A more detailed discussion of mapping studies can be found in Appendix 4.

Relevant studies for utilities of intraventricular haemorrhage and respiratory distress syndrome and for mothers

Only one paper considers the quality of life for preterm children with IVH, separated into two severity groups: level 0–2 IVH with no periventricular leukomalacia (PVL) and level 2–4 IVH with/without PVL. 102 However, a suitable mapping to EQ-5D utility for the health-related quality-of-life measure used in this paper could not be found.

Likewise, only one paper103 considers quality of life for preterm children with RDS. This study measures SF-36 scores, but does not report them. We were unable to obtain the SF-36 data after contacting the corresponding author.

The evidence on the quality of life of mothers of preterm children is sparse. Only two studies consider mothers of preterm children specifically. 104,105 The first is an abstract that reported only physical and mental health SF-36 mean summary scores,104 and the second reported Maternal Postpartum Quality of Life (MAPP-QOL) scores. 105 Neither of these could be reliably mapped to EQ-5D utilities.

Couto et al. 106 assessed the quality of life for mothers in Brazil who have had at least one of four previous adverse pregnancy outcomes. Although preterm birth is one of the four outcomes that is an inclusion criterion (along with early neonatal death, recurrent abortion and fetal death), we are not provided with separate utilities for each outcome individually.

A more detailed discussion of all reviewed studies for the utilities of preterm survivors, IVH, RDS and mothers, is provided in Appendix 4.

Utilities for reduced birthweight

In order to assess whether or not there is any quality-of-life impact from reduced birthweight as a result of not receiving treatment, papers that were identified from title and abstract screening were searched to find studies that contained both sufficient birthweight data and utility data. One study was identified, and the author was able to provide the raw data on request. 107

A number of regression specifications were estimated using random-effects estimators, in order to find the effects of birthweight on utility (see Appendix 5, Statistical analysis of the effects of birthweight on utility). However, the coefficient estimates for birthweight and squared birthweight were not statistically significant at the 5% level in any model. Furthermore, the simplest specification (including only birthweight and squared birthweight) failed the LR test when compared with a specification that included gestational age, sex and time dummies. Therefore, the analysis found insufficient evidence of a birthweight effect on utility, and so we assumed that there is no utility loss from reduced birthweight alone. Further details of the statistical analysis can be found in Appendix 5, Statistical analysis of the effects of birthweight on utility.

Utility parameters selected for the economic model

Based on the discussion in Appendix 4, utilities deemed the most appropriate base-case values are reported in Table 20. Proxy utilities for RDS and IVH were obtained from the paper by Carroll and Downs,108 which was the source for the study by Bastek et al. 109

TABLE 20 - Utilities selected for the economic model

SG, standard gamble; TTO, time trade-off.

Range calculated as a 95% CI, based on the data from Carroll and Downs. 108

Range taken directly from the study by Bastek et al. 109 These represent the minimum and maximum values found in their literature search for these utilities.

SF-36 means for the eight dimensions were mapped onto EQ-5D utilities using a quadratic model. 101

Range taken from two of the five studies reporting SF-36 scores (the minimum and maximum utilities reported among the five studies). 112,113

Range is generated by generating a 95% CI for the eight SF-36 means, and mapping all lower bounds and all upper bounds to EQ-5D using a linear model. 101 These represent a wider estimate of the range than if a 95% CI was provided for the EQ-5D measure directly.

Variable	Subject	Source	Measure	Utility	Range
‘Severe’ RDS (severe persistent asthma used as proxy)	Child	Carroll and Downs108	TTO	0.85	0.84–0.86a
IVH grades III–IV (moderate cerebral palsy used as proxy)	Child	Carroll and Downs108 via Bastek et al.109	TTO	0.76	0.66–0.84b
Death	Child	Assumption. Upper bound from Vandenbussche et al.115	SG (upper bound only)	0	0–0.02
Preterm survivor	Child	Cooke111	SF-36c	0.879	0.846–0.901d
Mother with previous adverse child outcome	Mother	Couto et al.106	SF-36c	0.644	0.556–0.652e
Mother with no adverse child outcome	Mother	Couto et al.106	SF-36c	0.834	0.768–0.843e

In practice, not all children with RDS go on to develop severe persistent asthma (the proxy for RDS used in the model). Based on feedback from a neonatologist, we applied this proxy utility to 56% of all RDS cases. This figure is from a UK-based study that found that 56% of children born extremely preterm had abnormal baseline spirometry when they were 11 years old. 110 The remaining 44% of RDS cases are assumed to incur no additional QALY loss, relative to a preterm survivor.

For IVH, the proxy used (moderate cerebral palsy) is too severe for infants with IVH grades of below III. On consultation with a clinical expert, long-term outcomes from IVH grades below III are thought to not differ greatly from those of preterm survivors in general. Therefore, we apply the utility of moderate cerebral palsy only to incidences of IVH that are at grade III or IV.

The utility for preterm survivors was obtained using mapped SF-36 scores. 101 The utility computed from the UK study111 was used for the base case, and the minimum and maximum utilities in the remaining papers were selected to provide a range. 112,113 This follows the principles outlined in the NICE technical support document. 114

We also considered measuring the health-related quality-of-life outcomes of the mother. We identified published evidence on longer term utility for mothers with previous adverse pregnancy or neonatal outcomes from the study by Couto et al. 106 This can be used as a proxy for mothers who have preterm children who suffer an adverse outcome, with the caveat that it is likely to be an overestimate for infant mortality and an underestimate for IVH or RDS. The lower (upper) bound for the two utilities imputed from the study by Couto et al. 106 is calculated by taking the lower (upper) value of the 95% CI for each of the eight SF-36 dimensions and generating a mapped EQ-5D from this vector. This provides a relatively pessimistic (wide) estimate of the range of utilities, which is desirable given the caveat attached to the adverse outcome utility value.

Therefore, the utilities of mothers and infants were linked by the occurrence of neonatal mortality and morbidity events both in deterministic and stochastic analyses. Further details on how discounted QALYs were obtained from these utility values, and a comparison of these to the QALY values used in the NICE guidelines model,24 can be found in Appendix 5, Quality-adjusted life-year calculations and comparison with parameters used in the National Institute for Health and Care Excellence guidelines model. Details may be found therein for the basis of assumptions such as the 1-year duration of mortality effects.

Summary of all child-related quality-adjusted life-year values used in the model

The base-case total discounted lifetime QALYs for children are summarised in Figure 12. For comparison, total discounted lifetime QALYs calculated using the NICE guidelines method24 (i.e. without age and survival adjustments across the lifespan) are denoted by dashed lines in Figure 12. As can be seen by comparing these values, the NICE guidelines method for obtaining QALYs appears to:

• overestimate total QALYs for preterm survivors

• overestimate the QALY losses from RDS and IVH.

FIGURE 12.

Total lifetime QALYs for preterm children by gestational age and health state, using age and survival adjustment (solid lines) and NICE’s unadjusted method24 (dashed lines). QALY losses in the graph are not scaled to apply to only severe cases of RDS and IVH; this is to enable clearer visual comparison between the two QALY calculation methods.

This illustrates that the NICE guideline method is likely to overestimate the benefits of treatment-intensive options. It must be noted that this comparison of the two methods to extrapolating outcomes is undertaken within the Peninsula Technology Assessment Group (PenTAG) model structure, which is based on the effects of ACS treatment, as opposed to the original NICE model’s24 tocolytic-based mechanism of effect. Therefore, a comparison of the overestimation of QALY gains from treatment-intensive options (e.g. treat all vs. qfFN) in the original NICE model24 is slightly smaller because, as discussed in Chapter 5 the NICE model underestimated the benefits of treatment as it was based on tocolysis as opposed to ACSs.

Base-case analyses

In the base-case analysis, we consider the case of women presenting to a level 2 care hospital. This is based on level 2 hospitals having the highest frequency of cases, given the mean gestational age at presentation of 30 weeks. We set the prevalence rate of preterm birth within 7 days of testing at 3.0% and the prevalence rate of preterm birth at < 37 weeks’ gestation at 12.1%. 92 In order to derive the costs and QALYs of PartoSure for comparison with fFN at 50 ng/ml (and other options in the full incremental analyses), we conducted an indirect comparison whereby the incremental costs and QALYs of PartoSure relative to fFN at 50 ng/ml based on the test accuracy data from the study by Hadzi-Lega et al. 44 were added to the costs and QALYs of Actim Partus based on the APOSTEL-1 data. 42,43 Table 22 summarises the main model assumptions.

Scenario analyses

We explore the following scenario analyses:

• alternative study sources of test accuracy data

• women presenting at tertiary-level unit hospitals

• limiting costs to the neonatal phase

• limiting costs to the diagnostic phase (until delivery)

• assuming that ACSs have (partial) benefits when administered earlier than 7 days before preterm delivery

• excluding the neonatal hospital costs of infant death

• including mothers’ QALYs.

Probabilistic sensitivity analyses

We present probabilistic analyses using information on sampling uncertainty for test accuracy, costs and utilities presented in Tables 16, 17 and 19. These analyses are presented in terms of the relative frequency with which each diagnostic test option had the highest net monetary benefit of all competing options. 127

Base-case results

The base-case deterministic results are presented in Table 23. These are based on the preferred comparative studies APOSTEL-142,43 and Hadzi-Lega et al. 44 The base case considers women presenting at 30 weeks’ gestation to a level 2 hospital. Although all incremental cost-effectiveness ratios (ICERs) are positive, they should be interpreted with caution because, other than ‘treat all’ and fFN of 10 ng/ml versus fFN of 50 ng/ml, they represent a reduction in both costs and QALYs. Actim Partus results in £56,030 of cost savings per QALY lost relative to fFN of 50 ng/ml, which are higher than those of fFN of 200 ng/ml (£25,209) and fFN of 500 ng/ml (£17,025). The costs savings and QALY losses for PartoSure versus fFN of 50 ng/ml are the result of an indirect comparison between the studies by Bruijn et al. 42,43 and Hadzi-Lega et al. ,44 because no included study directly compares these two tests. Subject to this caveat, PartoSure would produce the same QALY loss but more cost savings than Actim Partus, relative to fFN of 50 ng/ml.

Table 24 shows the base-case results as a full incremental analysis. The rows of the table are ordered from the most to the least effective testing option in terms of total QALYs. Incremental costs, QALYs and cost-effectiveness for each test are shown in comparison with the following option in the table; for example, fFN at 50 ng/ml is immediately above Actim Partus in the QALY ranking, but has lower costs than this test and therefore dominates it. The relevant ICER of fFN at 50 ng/ml is, therefore, the ICER calculated relative to PartoSure (i.e. £81,922; see Table 23). A graphical depiction of these results is presented in Figure 13.

FIGURE 13.

Incremental costs and benefits of index tests against comparator (fFN at 50 ng/ml).

Table 25 breaks down the base-case results shown in Table 23 in terms of their component discounted costs and QALYs. It should be noted, as in Table 23, that the cost saving per QALY loss for PartoSure versus fFN at 50 ng/ml is the result of an indirect comparison via Actim Partus. More specifically, the relative differences between PartoSure and Actim Partus obtained using the study by Hadzi-Lega et al. 44 were applied to the results for Actim Partus using the study by Bruijn et al. ,42,43 and then compared with fFN at 50 ng/ml.

The ICERs for women presenting at 26 weeks’ gestation and 33 weeks’ gestation at a level 2 hospital are provided in Appendix 6.

Tornado analysis

For the tornado analysis, the parameter base-case values were increased and decreased by 20% (the upper and the lower variations, respectively) and the ICERs versus the comparator of fFN at 50 ng/ml were plotted, with the intersection of the vertical and the horizontal axes at the ICER base case. The tornado plots for each of the interventions in the Bruijn et al. 43 (APOSTEL-1) study and the Hadzi-Lega et al. 44 study are presented here and in Appendix 6.

The tornado plot for PartoSure (imputed) versus fFN at 50 ng/ml is shown in Figure 14. The remainder can be found in Appendix 6 (see Figures 19–23). There is a consistent pattern across all comparisons. The results are sensitive to the health-related quality of life (state utility) of preterm survivors. Much less influential are the cost of hospital admission, the prevalence of preterm birth within 7 days, the effectiveness of steroid treatment and the baseline mortality risks. Other parameter values appear to have no discernible influence on the results.

FIGURE 14.

Tornado diagram for model parameters with the most influential variation in value: PartoSure. PTB, preterm birth; RR, risk ratio.

Probabilistic sensitivity analyses

At a willingness to pay per QALY threshold of £20,000, Actim Partus has a probability of being cost-effective of 14%, 21% and 21% for women presenting at gestational ages of 33, 30 and 26 weeks, respectively (Figure 15, and see Appendix 6, Figures 24 and 25). PartoSure has probabilities of being cost-effective of 83%, 76% and 75% for women presenting at gestation of 33, 30 and 26 weeks, respectively, but these values are based on indirect comparison and have a lower strength of evidence than for other diagnostic options.

FIGURE 15.

Probabilistic analysis: women presenting at 30 weeks’ gestation. The highest net monetary benefit is for a gestational age at presentation of 30 weeks.

The treat-all option has a probability of being cost-effective of 0% at a willingness to pay per QALY threshold of £20,000 for all women, and becomes the option with the highest likelihood of being cost-effective for women presenting at 26 weeks’ gestation only at a willingness to pay per QALY threshold of > £180,000.

Scenario analyses

Strengths

The strengths of this systematic review are that it was conducted by an independent, experienced research team using the latest evidence and working to a prespecified protocol (PROSPERO CRD42017072696) that follows a robust methodology.

The search strategy was devised by a dedicated information specialist. The strategy did not restrict by study design and included both forward and backward citation chasing, web searching and cross-checking with studies provided by the companies. The studies were independently screened by two reviewers, with data extraction and quality appraisal conducted by one reviewer and checked by a second.

Weaknesses

The primary weakness of the review of test accuracy was the substantial methodological and clinical heterogeneity between included studies. There was considerable heterogeneity in the following areas: prevalence of preterm birth, mode of delivery, gestational age, definition (symptoms) of preterm labour (including dilatation threshold), inclusion of multiple gestations, participant characteristics and provision of treatments. As a consequence, the reported accuracies of individual tests varied widely, and hence the CIs in the pooled analyses are also wide. Subsequently, we have limited confidence in the mean pooled test accuracy results.

A limitation to our review was the lack of published studies in which two or more index tests were administered to the same population. Such studies allow us to have more confidence in any differences between the accuracy results of the tests as differences would not be attributable to population or study design. Only two studies, APOSTEL-142,43 (2016) and Hadzi-Lega et al. 44 (2017), assessed the DTA of two different index tests in the same population; APOSTEL-142,43 assessed both Actim Partus and qfFN and Hadzi-Lega et al. 44 assessed Actim Partus and PartoSure. We did not identify any studies in which all three tests were used in the same population.

Our review was also limited by the lower number of published studies for qfFN (two studies) and PartoSure (four studies) than for Actim Partus (16 studies) against the 7-day reference standard. In addition, fewer studies published data against the 48-hour reference standard, with only seven studies being identified (six for Actim Partus and one for PartoSure). Meta-analysis of test accuracy data requires a minimum of four studies; therefore, the scope for meta-analysis was restricted. We are aware of three studies published after our searches were run that assessed PartoSure (Wing et al. ,130 Lofti et al. 131 and Melchor et al. 132) and (confidential information has been removed). We also identified seven relevant ongoing trials, four of which are UK based, of which two plan to enrol > 1000 participants. Personal communications with trial organisers (Dr Sarah Stock, University of Edinburgh, 2017, and Professor Andrew Shennan, King’s College London, 2017) indicated that data from the two large UK trials (QUIDS/QUIDS-2 assessing PartoSure, Actim Partus and qfFN and PETRA assessing qfFN) were expected in 2018. There is the potential, should our analyses be re-run using the data from these trials, that the estimates of relative test accuracy may change.

The scope issued by NICE12 asked for an assessment of test accuracy of qfFN at thresholds other than 50 ng/ml. Our capabilities to look at different thresholds was limited by those reported in the published studies. The two qfFN studies both used thresholds of 10, 50, 200 and 500 ng/ml. Without access to the individual patient data, we were unable to assess any other thresholds.

Owing to the paucity of published test accuracy studies, we made two protocol amendments. The first was to include women with multiple gestations (up to 20% of the total population). Without this amendment, there would have been no includable qfFN studies. The second was to include studies in which testing was not carried out in line with clinical practice (i.e. the samples were frozen and analysed at a later date). Without this amendment, we would have had only one includable qfFN study (Bruijn et al. 62).

Dependent on how the data were reported in each study, we were required to conduct some data manipulations. Most studies reported the raw true-positive, true-negative, false-positive and false-negative data, enabling us to calculate additional test accuracy statistics, such as sensitivity, specificity, PPV and NPV. However, four studies48,54,56,57 reported only sensitivity, specificity, PPV and NPV. We back-calculated from these statistics to derive the true-positive, true-negative, false-positive and false-negative values, which were required in our review. All the raw data as reported in the published studies are available in Appendix 3 (see Tables 33 and 34).

Our review limited included studies to those published in the English language. This may be considered a limitation; however, a systematic review assessing the bias of excluding studies that were not in the English language found no evidence of bias but that there may have been an impact on precision. 133 Our review did include a French study (Werlen et al. 41), as a certified translation was received from the manufacturer (Parsagen Diagnostics Inc.).

Areas of uncertainty

There is considerable uncertainty surrounding the generalisability of the studies to the UK population. Most specifically, none of the included studies was conducted in the UK. In addition, the prevalence rates of preterm birth in our included studies ranged from 2% to 73%. UK prevalence is approximately 8%. These differences in prevalence between studies are probably attributable to the differences in the populations recruited into the studies (e.g. differences in gestational age, in presenting symptoms of preterm labour and in recruitment of high- or low-risk women). It is likely that the prevalence of preterm birth will have an impact on the DTA data presented in Chapter 2, Results of quantitative data synthesis (test accuracy data). 134 Indeed, Leeflang et al. 134 explored how sensitivity and specificity vary with disease prevalence, and suggested using prevalence as a guide when selecting studies that most closely match the situation under assessment.

Our ongoing trial searches identified four relevant ongoing UK trials, two of which are very large (> 1000 participants) and their results are anticipated to be published in 2018 (PETRA and QUIDS).

There is some uncertainty around whether or not the studies included in the review will be representative of women who do not (or cannot) have access to cervical length measurement. No studies were identified that were specifically based on such a population; the majority of studies did not mention access to cervical length measurement, seven studies used (but did not select participants based on) cervical length measurement and two studies only included women with a transvaginal cervical length measurement of ≤ 30 mm. It is in these final two studies (APOSTEL-142,43 and Danti et al. 50) in which there is most uncertainty; selection based on cervical length measurement would probably increase the prevalence of preterm birth in these studies. However, given that 15 of the 20 included studies had a prevalence rate of < 25% and both studies had a prevalence of < 25% (APOSTEL-1,42,43 19.7%; Danti et al. ,50 6.7%), it is unlikely that these criteria had an impact on prevalence rates.

There was also uncertainty around whether or not any management strategies (e.g. treatments) would incorrectly inflate false-positive rates. As described in Chapter 2, Summary of any treatments given, the types of treatments offered to women in threatened preterm labour differed between each study, as did the level of detail describing what and who received the treatments. More often than not, the treatment options were at the clinician’s discretion. It is most likely that the tocolytic treatments would have the biggest impact on incorrectly inflating the false-positive rate, as their purpose is to delay delivery.

It is understood that clinicians would use the results of these tests in combination with other clinical information to make clinical decisions. A mobile application called QUIPP is used in local clinical practice to assist with decision-making. The QUIPP app generates a risk score from the following information: whether or not the mother is symptomatic, number of fetuses, gestation in weeks and days, qfFN value and/or cervical length measurement. In our review, we did not consider combining test results with such clinical data, because no studies were identified that assessed this combination.

We also acknowledge four studies of PartoSure and quantitative/qualitative fFN that have been recently published or are due to be published. These studies were published after our searches were run and consequently were not eligible for the review. The studies are:

• Wing et al. 130 (2017), n = 796 – PartoSure and fFN at 50 ng/ml

• Lofti et al. 131 (2017), n = 132 – PartoSure

• Melchor et al. 132 (2017), n = 420 – PartoSure and fFN at 50 ng/ml

• (confidential information has been removed).

Strengths

The review was conducted by an experienced research team and the conducted searches were very sensitive as no study design filters were used. All citations were screened by at least two members of the review team. The review team worked to a prespecified prospective protocol.

Weaknesses

The review focused on published literature indexed by bibliographic databases, meaning that grey literature was not identified. The searches were conducted in July 2017, so it is possible that studies have been published and indexed subsequently that have not been identified.

Areas of uncertainty

Because no studies were identified for inclusion, we were unable to assess whether or not using these biomarker tests for predicting preterm labour is clinically effective (i.e. whether or not they would improve health outcomes).

Strengths

Our review was able to highlight the major developments in methodological practice. Studies have evolved from evaluating only the cost differences of diagnostic strategies, so that competing options were selected solely on the basis of their ability to rule out cases of unnecessary treatment and admission, to evaluating the neonatal health implications of missing the rare cases of true preterm labour. A clear finding from the review was the limited information that most previous economic analyses provide for guiding decision-making. With one exception (the model that informed the 2015 NICE guidelines24 on diagnosis and treatment of preterm labour), the previous studies do not account for the gestational age gradient in neonatal mortality and morbidity risk exposure and its consequences for cost-effectiveness.

Furthermore, previous models did not account for the variation in costs and benefits of diagnostic testing across hospital settings.

Our results also highlight the need to account for the neonatal hospital cost implications of saving a preterm infant, which no previous study has addressed.

Our analysis also suggests that, since the 2015 NICE guidelines24 on preterm labour diagnosis and treatment, the evidence on the risk–benefit profile for tocolysis and steroids has changed. Tocolysis is now used sparingly, whereas the importance of providing steroids within 2 days of preterm delivery has gained consensus, especially following articles published in 2017. Unlike the NICE guidelines model, which used tocolysis as the mediator of improved diagnosis and clinical effectiveness in terms of neonatal mortality and morbidity, cost-effectiveness analysis should be considered with reference to the timing of corticosteroid administration. Despite the 2015 NICE guideline recommendation that women presenting with symptoms of preterm labour at < 30 weeks’ gestation should be admitted to hospital without testing, current practice has not followed this recommendation. Instead, the fFN and Actim Partus tests are commonly used to guide admission and treatment decisions in women as early as 22 weeks’ gestation. The qualitative fFN test that previously produced a binary result has now been replaced with a test that provides a concentration level. Clinicians are using the new test in more flexible ways than the older binary test, in some cases applying different thresholds for admission and steroid treatment. This warrants new analysis that takes into account the emerging evidence and updated testing practices.

Weaknesses

We focused on economic studies in symptomatic women with intact membranes, and did not cover studies of asymptomatic women, which might have provided important relevant evidence on utilities, costs, epidemiological parameters and modelling methods. For example, we learned that the only study measuring generic health-related quality of life for outcomes of mothers (EQ-5D utilities) was a RCT in asymptomatic women [the OPPITIMUM (dOes Progesterone Prophylaxis To prevent preterm labour IMprove oUtcoMe) trial by Norman et al. 135 in 2016]. We contacted the authors to request utility data, but these were not yet available to external researchers to the trial.

At the time of writing, there is no published full economic evaluation of new index tests. The existing studies have addressed only the question of the cost-effectiveness of testing versus no testing. We are aware that there is a research article being prepared for publication on the basis of the QUIDDS project that evaluates the use of qfFN in 1500 individuals across the UK (Dr Sarah Stock, University of Edinburgh, 2017, personal communication). This will constitute the largest known economic study to date on a biochemical test in the population of interest to this review, and will enable the investigation of key outcomes for clinical effectiveness, such as the ability to predict delivery within 2 days of presentation with symptoms suggestive of preterm labour. Previous diagnostic studies of fFN tests have provided limited data on such outcomes.

In terms of methodological economic evaluation practice, older studies evaluating the at-the-time standard practice of treating all symptomatic women assumed that the economic value of diagnostic testing depended solely on a test’s ability to rule out cases in which patients were not in preterm labour or likely to deliver within 7 days. More recent studies have extended the analysis from a purely cost-minimisation framework to a cost-effectiveness framework, by recognising that, however few false-negative cases may be missed by testing, they are placed at a risk relative to the treat-all alternative management option, so that there is a trade-off between cost savings and increased risks to life and quality of life for a few cases with testing. The model that informed the 2015 NICE guideline24 on diagnosis and treatment of preterm labour represented a methodological advance in that it explicitly recognised that such trade-offs varied by gestational age and were more favourable to testing at older gestational ages.

Since the 2015 NICE guidelines24 on the topic, new evidence has emerged on the value of timely use of ACSs, which may still confer benefits on the neonate if given within 1 day of delivery and have a maximum benefit when given between 1 and 2 days of delivery. However, existing models are ill suited to account for this emerging evidence as few diagnostic studies are large enough to include sufficient numbers to measure such outcomes reliably. Current clinical guidelines and recommendations maintain that steroids need to be given within 7 days of delivery to be effective, although empirical findings published in 2017 by separate independent groups of researchers suggest residual beneficial effects when steroids are given more than 7 days before delivery. 34,35,94 On the other hand, existing studies have not accounted for the risks posed to the neonate by ACSs in terms of birthweight, despite the common perception by health professionals of the importance of these risks as manifested through the existing tendency to not give repeated courses of ACSs to women who do not deliver within 7 days of an initial course.

Finally, although the screening was carried out by two reviewers, the data extraction and critical appraisal were conducted by only one. This may increase the risk of bias associated with the findings of this review.

Areas of uncertainty

Our review could not inform the cost-effectiveness of PartoSure, Actim Partus and fFN. The only evidence available was found for fFN in an unpublished study (a MSc dissertation68), which suggests that fFN at a threshold of 50 ng/ml for treatment and hospital admission may be an inefficient use of NHS resources and that restricting treatment and admission by raising the threshold to 200 ng/ml may be cost-effective. These findings were derived by the AG from the number needed to test to adequately treat a woman with steroids, as reported by the unpublished study of fFN, using a decision tree model and costs and utilities as used by the NICE 2015 guideline model. 24 In view of the limitations of the unpublished model highlighted previously, it is unclear whether or not these findings are robust to sampling and structural model uncertainty.

Key areas of structural uncertainty include the maintained assumption in the NICE model24 that false negatives (i.e. those who test negative but deliver within 7 days of testing) miss treatment and, therefore, are placed at increased risk of neonatal death and of experiencing adverse chronic events including RDS and IVH. In fact, some of those ‘missed’ cases are likely to return to the maternity hospital and receive ACSs closer to delivery, thus, paradoxically deriving more benefit from the treatment than if they had been detected in the first place. Another key unknown is the effect of accounting for neonatal costs on the results, because previous studies have ignored the costs associated with neonatal deaths and have used low-quality data on costs of neonatal morbidity. Better quality data on neonatal hospital costs (length of stay at different levels of care) are available for the UK from the National Neonatal Research Database. 124

Strengths

We provide new evidence on the cost-effectiveness of new and existing diagnostic tests in use in the NHS. We model the costs and benefits of diagnostic testing on the basis of recent evidence on the optimal time-to-delivery intervals for effective ACS administration in terms of neonatal mortality and morbidity (RDS and IVH). Furthermore, we adopted the best modelling practice in the field and introduced some innovations by accounting for:

• gestational age

• hospital setting (level 1, 2 or 3)

• costs of in utero transfers at very preterm gestation

• neonatal hospital costs of saving a preterm infant

• long-term QALYs and costs of additional preterm birth survivors with and without adverse events (RDS and IVH)

• mothers’ QALYs (in exploratory analysis).

We also conducted extensive scenario analysis and probabilistic analysis to reflect sampling uncertainty in model parameter values.

Weaknesses

We were unable to consider multipurpose uses of some of the tests (i.e. relevant for other indications). In our analysis, we did not account for the effect of costs of any deals offered by suppliers for the purchasing of combinations of diagnostic tests or in bulk over a hospital network.

The scope of our analysis is limited by the fact that we do not consider diagnostic strategies involving combinations of tests. The population was defined as women for whom a transvaginal ultrasound scan was not indicated or who were attending maternity hospitals in which it was unavailable.

We had to approximate the neonatal hospital costs of RDS and IVH, assuming that they are the same and equal to the additional neonatal hospital costs incurred by infants who were admitted to BAPM level 1 care (this led to costs similar to those used by the NICE 2015 guideline model24).

Critically, the evidence on accuracy was limited to two comparative studies, one of which may be considered too small (n = 57) to reliably detect differences between index tests; in fact, the study found PartoSure and Actim Partus to have the same sensitivity, which in our model determines clinical effectiveness. Therefore, our findings must be considered with caution and point to the need to conduct further research before drawing any conclusions on the relative cost-effectiveness of Actim Partus and PartoSure. On firmer ground are the findings that, relative to fFN at 50 ng/ml, Actim Partus produced larger savings per QALY lost than fFN at 200 ng/ml and fFN at 500 ng/ml, whereas ‘treat all’ and fFN at 10 ng/ml had ICERs above the £80,000 mark in women presenting at 24–34 weeks’ gestation; these results are based on diagnostic accuracy data from one study42,43 in 350 women. However, it must be noted that our results were based on the assumption that the overall test accuracy reported in these studies is the same across gestational ages, owing to a lack of data by length of gestation.

It is worth noting that the only UK diagnostic study was a non-comparative study of qfFN, in which fFN at 200 ng/ml and at 500 ng/ml resulted in the same sensitivity as that of fFN at 50 ng/ml. This was considered in scenario analyses, and fully incremental analysis suggests that lowering the fFN threshold for diagnosing preterm labour from 500 to 200 ng/ml has an ICER of < £20,000, whereas lowering it from 200 to 50 ng/ml has an ICER of > £20,000.

There is an absolute lack of evidence on the outcomes of the status quo diagnostic test, qfFN, in routine practice. Our analysis was largely based on the modelling of the strict adherence to local hospitals’ treatment guidelines with fFN. We were able to obtain audit data on treatment practice over 6 months (2016–17) in a local level 2 hospital (n = 75); however, this is too small an administrative sample to draw definitive conclusions.

A methodological limitation of our analysis is the working assumption that false-negative cases miss treatment altogether with the associated increased exposure to risks of neonatal mortality and morbidity. It may be argued that at least some false negatives may actually end up benefiting from being ‘missed’ because they may deliver between 2 and 7 days and, therefore, as an unintended consequence get the chance to return before delivery and closer to the target optimal window for ACS administration of 1 to 2 days before delivery. 33 Thus, many if not most false negatives may possibly end up having the same clinical outcomes as true positives. This would appear to justify applying a simpler modelling approach that focused only on costs (i.e. cost-minimisation analysis) or perhaps on 48-hour as opposed to 7-day diagnostic outcomes. Given the limited existing data and large sample sizes required to measure 48-hour diagnostic outcomes, the cost-minimisation analysis may seem the only practicable alternative and we conducted scenario analyses that limited the analytical horizon to the time of delivery, effectively providing a cost-minimisation analysis of the decision problem of interest (i.e. because we do not measure any clinical outcomes for the diagnostic phase). This scenario analysis also favoured Actim Partus and suggested that PartoSure may be cost-effective subject to the strong caveat discussed previously. Another argument in support of cost minimisation is provided by the perceived harm of multiple courses of ACSs; in this case, the key diagnostic outcome measure for clinical effectiveness would be the FPR because not only would it result in ineffective use of ACSs to start with but it would measure the extent to which women were precluded from any use of ACSs and thus exposed to the risk of adverse neonatal health outcomes. This suggests that the cost-minimisation analysis may be thought of as a conservative alternative scenario to the base case.

We found no reliable data to account for the side effects of ACSs in terms of birthweight. The only source of data on long-term quality-of-life outcomes by birthweight that we identified was from a Canadian study by Saigal et al. ,107 which involved measured quality-of-life outcomes in 286 extremely low-birthweight survivors at adolescence and young and mature adulthood. Our analysis showed that data produced no detectable relationship between birthweight and quality of life, which may be attributed to the sample being too small to reliably measure long-term outcome differences by gestational age at birth.

There was little evidence on quality-of-life outcomes of mothers, and we could conduct only some exploratory analysis of results including such evidence. Another limitation is that costs and utility values were independent of gestational age at birth; for example, we assumed that, provided a preterm infant survives until their first birthday, they will experience the same long-term quality of life as in the average population. Owing to a lack of data on the quality-of-life effects of IVH, we had to use utility values from other patient populations (i.e. individuals with ICH).

The mapping algorithm that we used to derive EQ-5D utility values from SF-36 produced predicted EQ-5D utilities that were consistently higher than (overpredicted) actual EQ-5D utilities for more severe EQ-5D states;101 however, this prediction error is large only for health-state utilities of < 0.25. The lowest mapped utility we consider is 0.64, which was for mothers, and not used in base-case analysis. The only mapped utilities included in the base case were for preterm survivors, and these were all > 0.8. Therefore, although there is certainly a limitation with the mapping function, we do not believe that the limitation is a concern given the ranges of utility values we deal with in this review.

Areas of uncertainty

There is a high degree of uncertainty associated with the costs and health benefits of PartoSure relative to other tests, as we had access to only one comparative study of this test, which involved < 100 individuals. A study of this size is unlikely to provide reliable results on diagnostic accuracy on delivery outcomes at 48 hours and 7 days. The uncertainty of our findings is compounded by the lack of a study of all relevant tests in the same patient sample, which led us to resort to indirect comparisons using a common comparator (i.e. Actim Partus) approach.

More generally, as discussed previously, the ability to predict delivery within 48 hours has become critical, given recent evidence on the importance of good neonatal outcomes of administering ACSs within such a short period before birth. In addition to prediction within 7 days, we aimed to differentiate in our model for the ability of a test to predict delivery within 48 hours but we abandoned that analysis owing to a lack of the required data for the great majority of studies. Further DTA studies in preterm labour should be undertaken with sufficiently large samples to measure and report diagnostic outcomes for the within 48 hours delivery end point in addition to the conventional 7-day outcomes.

Given the importance of this element for the cost-effectiveness of diagnostic options, further research is warranted to produce the data required to calculate more precise measures of long-term QALYs gained by saving a neonatal life, which distinguishes by the gestational age at birth. Furthermore, evidence is required on the long-term implications of birthweight reductions on quality of life, so that an adequate picture of the implications of the benefit–risk profile of ACSs may be accounted for in economic assessments.

There is a high degree of heterogeneity in the evidence on DTA, as illustrated by the very different prevalence of preterm birth between the largest comparative study (APOSTEL-1,42,43 20% delivery within 7 days) and the UK study (Abbott et al. ,92 3% delivery within 7 days), which is not explained by the former study’s inclusion and the latter’s exclusion of non-spontaneous preterm births. More obvious differences in the selection of patients for study participation is also a problem, as highlighted by the high preterm birth prevalence in one of the four available studies comparing Actim Partus and fFN at 50 ng/ml,57 which, by including patients at gestational ages of 28–36 weeks, rather than the more common 24- to 34-week range, led to a 7-day prevalence of preterm birth of 32.5% versus 19.7% in the largest comparative study of these tests (APOSTEL-1) (see Chapter 2, Differences between studies in mode of delivery).

We did not conduct a value-of-information analysis because, as with any model of intervention at birth, there is a degree of structural uncertainty associated with the need to extrapolate from neonatal to lifetime outcomes that would render parameter uncertainty meaningless. Thus, probabilistic sensitivity analysis needs to be considered as exploratory and value of information could not be considered reliable in informing research priorities.

Some of the uncertainties discussed previously may be addressed by the findings of two large (> 1000 participants) ongoing trials of the predictive utility of qfFN (PETRA) and qfFN, PartoSure and Actim Partus (QUIDS 2) in the UK. Until these new data become available, the value of PartoSure is unlikely to be settled. This and other studies may also address a key issue of uncertainty in our analysis, namely whether or not the use of the new tests do affect routine patient management. Our model was limited by the lack of these data and the consequently simplifying assumption that mothers were treated in strict accordance with the test results, which we know does not happen with the current status quo test (fFN at 50 ng/ml).