A prognostic model, including quantitative fetal fibronectin, to predict preterm labour: the QUIDS meta-analysis and prospective cohort study

Quantitative fetal fibronectin

The QUIDS study evaluates the Rapid fFN 10Q system, which provides a concentration of fFN (ng/ml or invalid) from a vaginal swab sample within 10 minutes. 16 It is now the only commercially available fFN test system and replaces the TLiQ^® system (Hologic, Inc.), which provided a qualitative fFN result (positive or negative) based on a threshold of 50 ng/ml.

Vaginal swab samples are analysed by lateral flow solid-phase immunochromatographic assay [the Rapid fFN Cassette Kit (Hologic, Inc.)] and interpreted in the Rapid fFN 10Q Analyzer (Hologic, Inc.). 16 A total volume of 200 µl of the sample is pipetted into the sample application well of the Rapid fFN cassette using a polypropylene or polyethylene pipette. 16 The sample flows from an absorbent pad across a nitrocellulose membrane via capillary action through a reaction zone containing murine monoclonal anti-fFN antibody conjugated to blue microspheres (conjugate). 16 The conjugate, embedded in the membrane, is mobilised by the flow of the sample. 16 The sample then flows through a zone containing goat polyclonal antihuman fibronectin antibody that captures the fibronectin–conjugate complexes. 16 The remaining sample flows through a zone containing goat polyclonal antimouse immunoglobulin-G antibody that captures unbound conjugate, resulting in a control line. 16 After 10 minutes of reaction time, the intensities of the test line and control line are interpreted with the Rapid fFN 10Q Analyzer and a printed result provided as a concentration in ng/ml (0–500 ng/ml) or as invalid. 16 The result is invalid if the test does not meet internal quality controls that are performed automatically with every test. 16 In the event of an invalid result, the test can be repeated with any remaining clinical specimen. A quality control can be performed by a reusable Rapid fFN 10Q QCette^® (Hologic, Inc.), which verifies that the analyser performance is within specification. 16

Actim Partus

The Actim Partus test is a visually interpreted, qualitative immunochromatographic dipstick test that detects the presence of phIGFBP-1 (insulin-like growth factor-binding protein 1) in cervical secretions during pregnancy. 17 It gives a qualitative (positive or negative) result within 5 minutes. The lowest detectable amount of phIGFBP-1 in the extracted sample is approximately 10 µg/l. 17 Samples are taken using the Actim Partus test kit as per the manufacturer’s instructions. 17 The test kit comprises an Actim Partus dipstick in an aluminium foil pouch with desiccant, a sterile polyester swab and a tube of specimen extraction solution (bovine serum albumin, protease inhibitors and preservatives; 0.5 ml). 17 The sample is collected from the cervix using the sterile polyester swab during a speculum examination. 17 The swab should be left in the cervix for 10–15 seconds to allow it to absorb the secretions. 17 The sample is then placed into the provided specimen extraction solution and swirled vigorously for 10 seconds. 17 The swab is then pressed against the wall of the tube to remove any remaining liquid from the swab before it is discarded.

The test involves two monoclonal antibodies for human insulin-like growth factor-binding protein 1 (IGFBP-1):17 one is bound to the blue latex particles (the detecting label) and the other is immobilised on a carrier membrane to catch the complex of antigen and latex-labelled antibody and indicate a positive result. 17 When placed in the sample, the dipstick absorbs the liquid, which starts to flow up the dipstick. If the sample contains phIGFBP-1 it binds to the antibody labelled with latex particles. 17 The particles are then carried by the liquid flow and, if IGFBP-1 is bound to them, they bind to the catching antibody. 17 A blue line (test line) will appear in the result area if the concentration of phIGFBP-1 in the sample exceeds the detection limit of the test. 17 A second blue line (control line) confirms the correct performance of the test. 17 The yellow dip area of the dipstick is placed into the extracted sample and held until the liquid is seen to enter the result area. 17 The dipstick is then removed and placed on a horizontal surface. Test results will be reported as positive, negative or invalid. The presence of two lines (test line and control line) indicates a positive result, however strong the line is. A negative result is shown by only one line (control line); an invalid result is shown by either no lines or the sample line only (i.e. no control line).

PartoSure

The PartoSure test provides a qualitative result (positive or negative) within 5 minutes. It is a rapid, non-instrumented, qualitative immunochromatographic test for the in vitro detection of PAMG-1 in vaginal secretions of pregnant women. 18 The test employs monoclonal antibodies that are sufficiently sensitive to detect 1 ng/ml of PAMG-1. 18 Samples are taken using the PartoSure test kit as per the manufacturer’s instructions. 18 The test kit comprises a PartoSure test strip in a foil pouch with desiccant, a sterile flocked vaginal swab and a plastic vial with solvent solution (0.9% sodium chloride, 0.05% sodium azide and 0.01% triton X-100). 18 The swab is inserted into the vagina (between 5 cm and 7 cm) without speculum and withdrawn after 30 seconds. 18 The sample is then placed into the provided solvent vial and rinsed by rotating it for 30 seconds. 18 The swab is then removed and discarded.

For testing of the sample, the sample flows from an absorbent pad to a nitrocellulose membrane passing through a reactive area containing monoclonal anti-PAMG-1 antibodies conjugated to a gold particle. 18 The antigen–antibody complex flows to the test region where it is immobilised by a second anti-PAMG-1 antibody. 18 This event leads to the appearance of the test line. Unbound antigen–antibody complexes continue to flow along the test strip and are immobilised by a second antibody. 18 This leads to the appearance of the internal control line. The test strip is inserted into the sample and held there until either two lines are present or 5 minutes have elapsed. 18 The strip should then be placed on a horizontal surface to read the results. Test results are reported as positive, negative or invalid. The presence of two lines (test line and control line) indicates a positive result, however strong the line is. 18 A negative result is shown by only one line (control line); an invalid result is shown by either no lines or the sample line only (i.e. no control line). 18

Model discrimination

Model discrimination is the ability of a prognostic model to correctly differentiate between those with and those without the outcome of interest. 23 We present this as the c-statistic, which is identical to the area under the receiver operating characteristic curve (AUC). In the context of our prognostic model, it represents the chance that in two women, one with and one without spontaneous preterm birth within 7 days, the predicted risk will be higher for the woman with spontaneous preterm birth within 7 days than for the one without. A c-index of 0.5 represents no discriminative ability, whereas a c-index of 1.0 indicates perfect discrimination. 23

Model calibration

Model calibration is the degree of agreement between the risk predicted by the model and the actual risk observed. 23 For example, if a prognostic mode gives a 5% risk of preterm birth within 7 days, then approximately 5 out of 100 women with this predicted risk should give birth within 7 days. Calibration is less relevant at internal validation because you would expect that any model will give correct predictions for the cohort it is derived from.

We present the calibration slope as a measure of calibration, which is a measure of agreement between observed and predicted risk of the outcome across the full range of predicted values. A value of 1 suggests perfect calibration and a value much lower than 1 suggests overfitting of the model to the data. 23

We present calibration plots as a visual representation of the expected/observed number of events, which summarises the overall calibration of risk predictions from the model in the validation data. 23 The expected/observed number of events provides the ratio of the total number of women expected to have a spontaneous preterm birth within 7 days to the total number of women observed to have spontaneous preterm birth within 7 days, with an ideal value of 1. Values of < 1 indicate that the model is underpredicting the number of events in a population, and values of > 1 indicate that the model is overpredicting the number events in a population. 23 In the calibration plots that we present (see Figures 1–3 and 8–11), individuals are ranked by deciles of predicted probability of spontaneous preterm birth within 7 days and plotted on the x-axis. Observed outcome frequencies are plotted on the y-axis. The 45-degree lines represents perfect calibration. We also refer to ‘calibration in the large’, which represents the intercept of the calibration plot and is, thus, a measure of whether the predictions are systematically too low or too high.

Overall performance

Overall performance is a statistical representation of the distance between the predicted outcome and the actual outcome. 24 We express this using the Nagelkerke R², which has a range from 0 to 1 and is a measure of how much of the variation is explained by the model. 24 It is not intuitive to interpret but can be used when comparing the performance of different models on the same data, with an aim of maximising the Nagelkerke R².

Net benefit

Net benefit is a type of decision curve analysis and a measure of the potential clinical value of a prognostic model. 24 In a formal decision analysis, a single optimal decision threshold is calculated from the quantified harms and benefits of the use of a prognostic model. However, defining a single threshold may be difficult at a population level because harms and benefits from treatments may be difficult to quantify. 24 It may also be undesirable because the relative weight of harms and benefits are likely to vary across individuals and health-care settings, and the perception of benefits and harms are likely to be influenced by personal values and experience. For example, from a clinical perspective, the relative harm of incorrectly predicting a woman’s probability of spontaneous preterm birth is greater at earlier gestation than at later gestation (owing to the higher risk of complications of prematurity at early gestations, which are worsened by lack of treatment); thus, a lower-risk threshold may be recommended to indicate hospital admission and antenatal corticosteroid use in women at early gestation than women at late gestation. Alternatively, when admission to hospital is offered solely because there is a risk of preterm birth (which is likely to incur family disruption and personal cost),4 a woman may have a higher risk threshold before accepting transfer if she lives in a geographically remote location with other caring responsibilities than if she lives near to a neonatal unit and has other family support.

Net benefit analysis allows the harms (from ‘missing’ a case of preterm birth) and benefits (from avoiding unnecessary treatment) of the model to be considered across a range of risk thresholds for spontaneous preterm birth. The potential benefits from correct identification of women at low probability of spontaneous preterm birth within 7 days (‘true negatives’) are put on the same standardised scale as potential harms from unnecessary treatments (‘false negatives’) to allow direct comparison, and are presented for a range of risk thresholds. 25 This is akin to receiver operator characteristic (ROC) curves in that the full range of thresholds are included, rather than a single threshold for a sensitivity/specificity pairing. 24

In our analysis, the range of decision thresholds are for ‘ruling out’ treatment and assume that if the model gives a predicted risk at or below a threshold then no treatment will be given, and that above this threshold steps will be taken to ameliorate outcomes of preterm birth should it occur (i.e. standard or usual care). Approaches of ‘treat all’ or ‘treat none’ are presented alongside the model net benefit for comparison.

Design

This study adopted a qualitative, interpretive approach, using semistructured interviews and focus groups to explore the decisional requirements and experiences of participants. This enabled a focused investigation of the a priori aim, while encouraging participants to tell their own stories. Service users were involved in the development of the protocol and study resources. The study was carried out in three NHS tertiary referral centres in England and Scotland.

Participants and recruitment

Participants were purposively sampled to cover different personal and professional experiences of preterm labour and birth. Inclusion criteria were pregnant women at high risk of preterm birth or who had experienced threatened preterm labour, and postnatal women who had experienced preterm birth (at < 34 weeks’ gestation). Clinicians with experience of caring for women in preterm labour and making decisions about their care were eligible, including midwives and obstetricians. Exclusion criteria included being aged < 16 years and non-English speaking. Women were identified by staff in the maternity department and clinicians were identified by members of the research team. Verbal and written information was provided and informed, written consent was gained.

Data collection

Data were collected between January and May 2016 via semistructured interviews, using different topic guides for each group. Women were invited to attend a focus group and those unable to attend were interviewed individually, face to face, in a hospital setting or over the telephone. Individual interviews were preferred for clinicians to avoid dominating participant bias or false consensus and to enable flexibility to interview clinicians when they were available. Demographic details were collected prior to the interview. Interviews were audio-recorded and field notes were taken. The focus group was facilitated by two female researchers and all individual interviews were facilitated by one researcher. Researchers were not part of the clinical care team. Recapping and summarising were used to clarify meaning and avoid misinterpretation. Reflexivity and acknowledgement of personal bias were maintained by regular debriefing between researchers and written reflective accounts following interviews.

Data analysis

Data were analysed independently by three researchers using a framework approach. Analysis of women’s and clinicians’ data was conducted separately then brought together. Data were transcribed verbatim then checked for accuracy against the original recordings. Data were anonymised and labelled using a study identification number and, later, a pseudonym. One researcher analysed all of the data using NVivo version 11 (QSR International, Warrington, UK) and a large sample of the data was analysed separately by two researchers. Consensus was reached regarding meaning and the final framework confirmed by discussion.

The framework approach enabled the large number of data to be managed and interpreted within the focused primary and exploratory secondary aims of the study. 30 The approach to analysis was underpinned by the theory that knowledge is constructed by social interchange;31 hence, themes emerged based on participants’ emphasis rather than on the a priori aims. Following verbatim transcription of the interview recordings, the researchers became familiar with the data by reading the transcripts and field notes several times. Recurring characteristics were recognised and related to decisional and informational requirements and emergent themes. The data were coded then mapped into themes and subthemes according to the participants’ emphasis, creating a framework for each group. The frameworks were refined and interpreted based on the original transcripts. At all stages the transcripts were reviewed to ensure that the thematic framework reflected the original context. Having multiple analysts ensured that themes were interpreted directly from the data, thus minimising interpretation bias.

Separate ethics approval was granted for this part of the study by the North-West Liverpool East NHS Research Ethics Committee (reference number 15/NW/0945).

Decision-making

Communication

Communication between women and care providers permeated all narratives, emphasising its importance. Positive or negative experiences of communication appeared to influence women’s overall judgement of care.

Women valued the communication of information, particularly because they felt that they had little knowledge of preterm birth or mid-trimester loss prior to their experiences. Women listed numerous informational requirements. Clinicians recognised the challenge of providing the vast amount of complex information required at such a sensitive time.

Some women reported discovering that the information provided to them during their experiences was incomplete; this damaged the trust they held in caregivers. Woman wanted honesty even when the information was negative, such as a poor prognosis:

. . . nobody was actually saying to me that you’re dilated, the likelihood is your baby is going to be born soon and she is not going to live – which sounds brutal but that’s what a woman needs to know.

Donna

Despite this, women wanted clinicians to deliver information to them sensitively, balancing honesty with empathy.

Clinicians listed some terms that they do not use, which broadly corresponded with terms that the women highlighted as upsetting. There were not many terms that the women objected to, but where they did object they did so strongly owing to the upset and distress caused. These terms included ‘fetus’, ‘miscarriage’, ‘viable’ and ‘abortion’:

I didn’t want to hear that word [‘miscarriage’], you know, at the end of the day I know I wasn’t really far gone, I was only 5 months, but I was still 5 months pregnant, frightened and . . . You know, it was just not a word that I wanted to hear.

Hatti

These terms should not be used in the decision support tool, and clinicians should be cautious in using them in their discussions with women. Indeed, using the words the women themselves use is optimal, as they are less likely to be upsetting for the women.

The experiences of women indicate that technical jargon should be avoided in the decision support tool, or fully explained if there is no alternative. Clinicians reported being mindful of this when communicating with women and families and cited useful techniques to enhance women’s understanding, for example relating care to previous experiences:

I ask them if they’ve ever had a smear test done before – if they have, I say, ‘it will feel a bit like having a smear test done’.

Obs1

The women and clinicians agreed that verbal communication was the most appropriate for information and results provision, so that the discussion could be individualised and for the checking of understanding. However, both women and clinicians felt that written or interactive forms of communication were also helpful to enable women to revisit what they had been told, especially complex concepts such as preterm birth risk:

I sometimes think clinicians aren’t maybe very good at putting it simply. I think sometimes patient information leaflets, when people are sent home, thinking about actually how to put it in terms of maybe using words but also using visual aids sometimes can be more helpful.

Obs7

. . . to be told but also to have some information to go back to in case you forgot or you didn’t really understand it. I think it just makes it better.

Beth

The method of communicating results was also discussed and there was general agreement that, although verbal communication was essential, having the ability to print out results for notes and for the women would be valuable. Some element of reassurance was gained from seeing the result:

I mean, it was nice obviously just . . . ‘cause it did say ‘negative’ on the top, so just backed up what she was saying. Not that she would have lied, but . . . yeah.

Fran

One clinician suggested that the results printout could include robust, high-quality information to back up what the clinicians discuss with women verbally. This may include information about the prognostic model, what their level of risk means, recommended care for them, evidence-based prognostic information or signs and symptoms to look out for if they are considered low risk and discharged home. Some women also considered the experience of waiting for fFN results. The idea of using that time to provide women with information about the test and the decision support tool was rated positively. This could be in a written or an electronic/interactive format, and should be used in addition to face-to-face discussions with a clinician.

The decision support tool and any associated resources, such as a patient information leaflet or interactive videos and results printouts, offer an opportunity to provide robust and high-quality information in a format that women and clinicians will understand. Some clinicians provided examples of formats that are effective, such as visual analogue scales or infographics. Any written, interactive or visual information should be presented clearly and simply, without language or imagery that the women might find emotive or upsetting. Such resources should be considered adjuncts to verbal communication with a skilled and knowledgeable clinician.

Accessing and negotiating care

Impact

Primary end point

The primary end point, consistent with the findings of QUIDS qualitative (see Chapter 3), was the binary outcome of whether or not spontaneous preterm birth occurred within 7 days of quantitative fFN.

A secondary outcome was the binary outcome of whether or not spontaneous birth occurred within 48 hours of quantitative testing.

Inclusion criteria

We prespecified inclusion of prospective cohort studies or RCTs of women with signs and symptoms of preterm labour (as defined by investigators) that included quantitative fFN results determined by the Rapid fFN 10Q analyser system and pregnancy outcome data if the principal investigator (PI) was in agreement to collaborate and provide full data. 41

Exclusion criteria

We excluded studies in which fFN concentration was measured by an enzyme-linked immunosorbent assay (ELISA) and studies in which IPD were not available for meta-analysis. 41

Search strategy

When applying for funding for this study (April 2014), we completed a literature search for completed and ongoing cohort studies of quantitative fFN using search terms for quantitative fFN and preterm birth. The databases that we searched include MEDLINE, EMBASE, Cochrane Database of Systematic Reviews, Database of Abstracts of Reviews of Effects, HTA database, Cochrane Central Register of Controlled Trials and ClinicalTrials.gov. We also used general search engines, such as Google (Google Inc., Mountain View, CA, USA), and searched references of systematic reviews. We consulted preterm birth researchers and networks, for example Royal College of Obstetricians and Gynaecologists Clinical Study Groups, British Maternal Fetal Medicine Society (BMFMS), Preterm Birth International Collaborative (PREBIC) and the manufacturers of Rapid fFN (Hologic, Inc.) to capture all relevant studies. 41

Study manuscripts and/or protocols were screened by two researchers. We contacted the PIs of all eligible studies and invited them to participate. De-identified data were transferred and stored in a bespoke database on a secure server at the University of Edinburgh.

Data items and sample size

A prespecified set of factors thought to influence the probability of spontaneous preterm birth, as agreed by the experts on the project management group, were requested and considered for inclusion as predictors in the prognostic model. 41

These candidate predictors included fFN concentration (ng/ml), previous spontaneous preterm birth, nulliparity (no previous pregnancy of > 24 weeks), gestational age at fFN test (weeks), maternal age (years), ethnicity, body mass index (BMI) (kg/m²), smoking status, deprivation index, number of uterine contractions in set time period, cervical dilatation (cm), vaginal bleeding, previous cervical treatment for cervical intraepithelial neoplasia (CIN), cervical length (transvaginal cervical length measurement) (mm), singleton or multiple pregnancy and tocolysis. 41

We specified that only variables available in every study would be used for model development, and we planned to further refine the list of potential predictors by ranking them by probable clinical relevance as agreed by consensus in the project management team. 41

In model development, the number of predictor parameters that can be considered is limited by the number of events, with guidance (at the time of our study design) suggesting that at least 10 events are required for each predictor parameter. 43,44 We deemed it sensible to limit predictors for potential inclusion in our model using this rule of thumb. 41

Data cleaning

Study quality was assessed (MB and SJS) using a checklist modified, as recommended by Chang et al.,45 from the quality assessment of diagnostic accuracy studies 2 (QUADAS-2) assessment tool46 (see Appendix 2). Prior to analysis, data were checked for outliers, with nonsensical values removed, and missing data were identified. The characteristics of the population of the eligible studies were summarised using means and standard deviations (SDs), and medians and interquartile ranges (IQRs) for continuous variables, and using counts and percentages for categorical variables. 41 The baseline participants characteristics were summarised first by individual study and second for the entire database. A summary of the number of events, total participants and the median gestational age at birth for each of the studies was also presented to describe the events in the population. The percentage of missing data in the entire database was also presented by each candidate prognostic variable.

Missing data

Under the assumptions of a missing at random (MAR) mechanism, multiple imputation was used to impute missing values of IPD for the predictors included in the final model so as to avoid excluding participants from the analysis. 47 Multiple imputation was performed in R version 3.6.1 (The R Foundation for Statistical Computing, Vienna, Austria) using the ‘mice’ package. 48 Multiple imputation was performed for each original study separately, before the meta-analysis, to recognise clustering of participants within studies and retain any potential heterogeneity across studies. Rubin’s rules were used to combine parameter estimates across the analyses within each set of imputed IPD meta-analysis data sets.

As there was more than one predictor with missing data to be included in the model, multiple imputation by chained equations was used. This approach uses a set of imputation equations, including one for each of the predictors with missing data; all equations include all of the predictors of interest. Missing values for the first predictor are imputed by initially regressing the predictor on all other predictors and the outcome of interest and then drawing from the corresponding posterior predictive distribution of the predictor. 49 The second predictor with missing values is imputed in the same manner, but includes the imputed values of the first predictor in the regression model. The imputation is repeated for all predictors with missing values and this forms one cycle; cycles are repeated to stabilise the results and then the whole process is repeated to create a set of m imputed data sets. We performed 60 imputations, based on the rule of thumb that the number of imputed data sets should equal the largest proportion of incomplete data observed in individual study populations. 49

Model development for the primary outcome

As the outcomes of interest was binary (spontaneous preterm birth within 7 days), a logistic regression modelling framework was used to develop the models. 41 Only predictors from the predefined selection that were available in each study separately were used for inclusion in our prognostic model. 41

We wanted to minimise heterogeneity across studies to ensure that our final model had more robust and generalisable risk predictions across settings and populations. 23 Therefore, we began by assessing the heterogeneity of the predictor effects after full adjustment, using a two-stage approach. We fitted a model with all candidate predictors in each study (i.e. no variable selection) and performed a random effects meta-analysis of each predictor’s adjusted effect separately. Heterogeneity of the predictor effects was quantified using the estimated variance (τ²) and the I² statistic. Predictors considered to have large heterogeneity in the prognostic effect across studies were removed to ensure that summary beta terms (adjusted log-odds ratios) in the model are meaningful (accurate) for individual populations. 50

For the model development we used a one-stage approach, combining all five data sets with a fixed-effect assumption. Estimation of effects was performed in R using the ‘glm’ function (binomial model). A separate intercept term per study was included in the model to account for clustering and to gauge how predictions may require tailoring to different populations (owing to differences in baseline risk). A backwards selection procedure was used to decide which of the candidate predictor variables should be included in the final prediction model (with a p-value of < 0.1 taken to warrant inclusion and prevent omission of important predictors). For categorical variables, we used the lowest p-value of any category (relative to the reference category) to indicate inclusion or exclusion. The model was refitted after dropping each individual predictor. The models were fitted in each of the imputed data sets and the parameter estimates combined using Rubin’s rules to form the prognostic model equation (see the ‘mice’ package48).

Continuous predictors were analysed on their continuous scale and non-linear associations with the outcome examined. The formulae [(quantitative fFN + 1)/100]^0.5 and [(cervical length + 1)/10]^0.5 were used to deal with non-linearity and zero values. We used the Multivariable Fractional Polynomial (MFP) package51 in R to identify non-linear terms for continuous variables. The package was applied to each of the multiply imputed data sets individually, with the pattern that optimally predicted the outcome variable in the majority of multiply imputed data sets being the one that was used.

Sensitivity analysis

A prognostic model that included tocolysis as a categorical variable (administered/not administered) was prespecified as a sensitivity analysis to explore any potential treatment effect of tocolysis in delaying birth. Tocolytics are often given to delay birth but have not been shown to delay birth beyond 48 hours. 52

Apparent model performance

After model development, the apparent performance of models was assessed by estimating their performance in the same data used to develop the model (using the means from the pooled imputed data sets). We calculated the overall fit (expressed by Nagelkerke R²) and the observed discrimination and calibration in the data set used to develop the model. The ability of models to discriminate between women with and women without spontaneous preterm birth within 7 days was determined by the AUC, also known as the c-statistic. Calibration was assessed for each tenth of predicted risk by calculating the ratio of predicted (expected) to observed probability of spontaneous preterm birth within 7 days, and visualised using a calibration plot with a non-parametric (locally weighted scatterplot smoothing) calibration curve. We also measured calibration across all participants by calculating the calibration slope and calibration in the large.

Internal validation and adjustment for overfitting

Apparent performance is likely to be optimistic because it is examined using the same data used for model development, for which there is likely to be overfitting. Therefore, internal validation was undertaken using a non-parametric bootstrap resampling technique. 53,54 Each modelling step was repeated in each of the bootstrap samples to obtain a new model based on each bootstrap sample. The apparent performance statistics (e.g. AUC and calibration slope) of each bootstrap model was compared with its performance in the original data set. The ‘optimism’ was the mean difference (across all bootstrap samples) between the apparent value in the bootstrap sample and the observed value in the original data set. This optimism estimate was then subtracted from the original model’s apparent performance to give an optimism-adjusted estimate of each measure of performance for the original model.

To adjust the model for overfitting, the optimism-adjusted calibration slope was used as a uniform shrinkage factor to shrink (penalise) the predictor effects (beta coefficients) of the original model. Then, while holding fixed the shrunken beta coefficients (via an offset term), the study-specific intercept terms were re-estimated to ensure that perfect overall calibration-in-the-large was maintained in each study separately. 50 This produced our final model containing the updated intercepts and the shrunken beta coefficients.

Calculation of net benefit

Net benefit was calculated as the proportion of true negatives minus the proportion of false negatives, weighted by the odds for high risk designation at the selected threshold. 25 We used the package ‘rmda’55 in R, using the means from the pooled imputed data sets to plot the net benefit across a range of thresholds for which a woman would be designated at high risk of preterm birth within 7 days. To compare the potential clinical value of including cervical length in a model, we plotted the net benefit of model A (clinical risk factors + quantitative fFN) against model C (clinical risk factors + quantitative fFN + cervical length) against strategies of ‘treat all’ and ‘treat none’. For comparison, we included diagnostic test accuracy results for cervical length only (based on a single threshold of 15 mm). At any given threshold, the preferred model is that with the higher net benefit. 25

Software

Data were cleaned in IBM Statistical Product and Service Solutions (SPSS) version 24 (IBM Corporation, Armonk, NY, USA). All analyses were done in R.

Description of data

We identified a total of 10 studies of quantitative fFN that were potentially eligible. Four early data sets (in three publications) used ELISAs to determine the concentration of fFN56–58 and were excluded because the different method of analysis and earlier period of study would increase heterogeneity. Six studies fulfilled the eligibility criteria (at the time of study identification only one study59 had been published, but three studies60–62 have subsequently published; Table 3). Five PIs agreed to provide data [Mol, EUFIS (European Fibronectin Study) data;61 van Baaren, APOSTEL-1 (Alleviation of Pregnancy Outcome by Suspending of Tocolysis in Early Labour – 1) study data;60 Khalil, unpublished QFCAPS (Quantitative fetal fibronectin, Cervical length and Actim Partus for the prediction of Preterm birth in Symptomatic women) study data; Shennan, EQUIPP (Evaluation of Fetal Fibronectin with a Quantitative Instrument for the Prediction of Preterm Birth) study data;59 and David, unpublished UCLH/Whit (University College London Hospital/Whittington) study data]. The PI (Elovitz) of the sixth study [Screening to Obviate Preterm Birth (STOP)]62 indicated that data were available only after publication of the study, which occurred after completion of our analysis. The participating studies are from consultant-led maternity units in the UK (three studies)59 and mainland Europe (two studies). 60,61 All women in the included studies provided informed consent for participation in the clinical research and for their data to be used in subsequent analyses. The studies were rated as having a low risk of bias (see Appendix 2).

Table 4 shows the availability of the prespecified candidate predictors in each study. Only maternal age, BMI, ethnicity, smoking, nulliparity, multiple pregnancy, gestational age at assessment, previous spontaneous preterm birth before 34 weeks’ gestation, cervical length and fFN test results were available in each study and, therefore, these 10 candidate predictors were included in the model development. Tocolysis was included in sensitivity analysis to explore any potential treatment effect on delaying birth.

Summary statistics for the baseline participant characteristics and available predictors in the IPD database are shown in Table 5. In total there were 139 events of spontaneous preterm birth within 7 days of fFN test among 1783 women with signs and symptoms of preterm labour; thus, there was an overall outcome proportion of 7.8%. There was a higher rate of spontaneous preterm birth within 7 days in the mainland European studies than in the UK studies. Given the 139 events, and the 10 events per predictor parameter rule of thumb, this suggested that about 14 predictor parameters could be considered. Because we had only 10 candidate predictors, this was appropriate and allowed us to consider non-linear trends.

A summary of the percentage of missing data for each study, and across all studies, is presented by candidate predictor in Table 6. Multiple imputation was performed for missing data on all predictors excluding cervical length.

There were high levels of missing data for cervical length in two of the UK studies (76% missing in EQUIPP,59 86% missing in UCLH/Whit), which reflects the fact that cervical length is not routinely available for assessment of women with symptoms of preterm labour in the UK despite being recognised as a predictor of spontaneous preterm birth. 13 With the agreement of the project management group and study steering committee, to explore the potential prognostic value of combining quantitative fFN with cervical length we decided on the following analysis strategy for the primary end point (spontaneous preterm birth within 7 days):

• Primary analysis – prognostic models with quantitative fFN without cervical length as candidate predictor, based on data from all five studies (relevant to current UK practice, in which cervical length assessment is not consistently routinely available). Model A is the model with variable selection (most parsimonious model). Model B is the full model with all predictors forced into the model.

• Secondary analysis (i) – prognostic models with cervical length as the candidate predictor, based on data from three studies with complete cases of cervical length (EUFIS,61 APOSTEL-160 and QFCAPS) [exploring the potential added value of including cervical length in the prognostic model but based on a mainly mainland European (and higher-risk) population]. Model C is the model with variable selection (most parsimonious model). Model D is the full model with all predictors forced into the model.

• Secondary analysis (ii) – a prognostic model with cervical length as the candidate predictor, based on data from all five studies, with multiple imputation of cervical length and other missing participant data (exploring the potential added value of including cervical length in the prognostic model, including data from a UK population, but with added uncertainty from the multiple imputation of large numbers of missing data). Model E is the model with variable selection (most parsimonious model). Model F is the full model with all predictors forced into the model.

Heterogeneity of predictor effects

The meta-analysis of the candidate predictor effects for the primary analysis showed low to moderate heterogeneity of their adjusted log-odds ratios across studies (see Appendix 3). Only previous spontaneous preterm birth showed moderate heterogeneity (τ² = 0.6 and I² = 38%). The other values were all close to zero. Therefore, we did not exclude any of the candidate predictors in our primary analysis because of heterogeneity.

Cervical length showed high heterogeneity (I² = 75%) across all the studies, probably reflecting the recognised higher-risk population in the mainland European studies (EUFIS61 and APOSTEL-160) and regional differences in clinical practice. A plan was made to explore the effect of cervical length in model development in the secondary analyses while recognising the inherent uncertainty resulting from population differences and imputation of data.

Prognostic models for spontaneous preterm birth within 7 days

Primary analysis: prognostic model based on quantitative fetal fibronectin and clinical risk factors (models A and B)

For multiple imputation of predictors, we based the number of imputed data sets on the largest proportion of incomplete cases observed in an individual study. The EUFIS study60 had 59.5% incomplete cases, so we created 60 imputed data sets. We performed the multiple imputation in each data set separately before merging them together. Because QFCAPS did not have any missing data, we replicated the data from this study 60 times to merge the multiple imputed data sets.

After merging the five data sets, a prognostic model with all available candidate predictors (quantitative, maternal age, BMI, ethnicity, smoking, nulliparity, multiple pregnancy, gestational age at assessment, previous spontaneous preterm birth before 34 weeks’ gestation) was fitted in each of the imputed data sets and the results were combined using Rubin’s rules. Predictor variables were dropped stepwise based on the largest p-value > 0.1. After every step, the MFP procedure was used within each imputation set to allow the selection of non-linear terms for continuous variables. The final list of predictors for the logistic model was quantitative fFN, smoking, ethnicity, nulliparity and multiple pregnancy. Quantitative fFN was transformed (square root) because of non-linearity.

Table 7 shows the logistic models before (model B) and after (model A) variable selection prior to adjustment for optimism. The more parsimonious model A identified that high quantitative fFN levels, South Asian ethnicity, nulliparity and multiple pregnancy were associated with increased probability of spontaneous preterm birth within 7 days. East Asian and African/Caribbean/Middle-Eastern and other ethnicity, and smoking, were associated with a reduced risk. The apparent AUC for the model before variable selection (averaged across all multiply imputed data sets) was 0.90 [95% confidence interval (CI) 0.88 to 0.93] and the Nagelkerke R² was 0.39. For the final model after variable selection (model A) the AUC was 0.90 (95% CI 0.87 to 0.93) and the Nagelkerke R² was 0.38, indicating very little additional benefit of the omitted variables.

TABLE 7 - Multivariable logistic analysis of candidate predictors (without cervical length) before (model B) and after (model A) variable selection

[(quantitative fFN + 1)/100]^0.5.

Variable	Model including all variables (model B)		Model after variable selection (model A)
	Intercept	95% CI	Intercept	95% CI
Study
1 (APOSTEL-1)60	–7.849	–11.24 to –4.45	–5.019	–6.43 to –3.60
2 (EUFIS)61	–8.529	–11.58 to –5.10	–5.697	–7.11 to –4.28
3 (EQUIPP)59	–9.019	–12.44 to –5.59	–6.152	–7.61 to –4.69
4 (QFCAPS)	–8.700	–12.40 to –5.00	–5.874	–7.85 to –3.90
5 (UCLH/Whit)	–9.324	–12.87 to –5.78	–6.493	–8.21 to –4.78
Variable	Model including all variables (model B)		Model after variable selection (model A)
	Beta	OR (95% CI)	Beta	OR (95% CI)
Quantitative fFNa	2.033	7.64 (5.68 to 10.28)	2.042	7.71 (5.74 to 10.34)
Age	0.024	1.02 (0.98 to 1.07)	–	–
BMI	0.018	1.02 (0.96 to 1.13)	–	–
Smoking	–0.656	0.52 (0.24 to 1.13)	–0.729	0.48 (0.23 to 1.03)
Ethnicity
White	Reference	–	Reference	–
South Asian	1.066	2.90 (0.93 to 9.10)	1.020	2.77 (0.89 to 8.61)
East Asian	–1.184	0.31 (0.04 to 2.49)	–1.087	0.34 (0.04 to 2.65)
African, Caribbean, Middle-Eastern	–0.216	0.81 (0.42 to 1.54)	–0.224	0.80 (0.46 to 1.50)
Other	–0.252	0.78 (0.20 to 3.00)	–0.330	0.72 (0.18 to 2.82)
Nulliparity	0.527	1.69 (1.06 to 2.71)	0.394	1.483 (0.95 to 2.31)
Multiple pregnancy	0.852	2.34 (1.00 to 4.07)	0.900	2.46 (1.44 to 4.20)
Previous spontaneous preterm birth at < 34 weeks’ gestation	0.427	1.53 (1.25 to 3.03)	–	–
Gestational age at assessment	0.031	1.031 (0.49 to 1.11)	–	–
Apparent predictive performance
Nagelkerke R2	0.39		0.38
AUC (95% CI)	0.90 (0.88 to 0.93)		0.90 (0.87 to 0.93)

Internal validation using a non-parametric bootstrap resampling technique resulted in a uniform shrinkage factor of 0.92, which suggested some slight overfitting during model development as expected. Table 8 shows the final model (model A) after shrinkage to adjust for overfitting. The optimism-adjusted AUC for this model was 0.89 (95% CI 0.87 to 0.93) and the optimism-adjusted Nagelkerke R² was 0.39. Figure 1 shows the calibration plot for model A with predicted versus observed risk in the complete data set.

TABLE 8 - Final multivariable logistic analysis for model A after adjustment for optimism

[(quantitative fFN + 1)/100]^0.5.

Mean shrinkage factor = 0.92.

Variable	Final model after variable selection and adjustment for optimism (model A)
	Beta	OR (95% CI)
Study
1 (APOSTEL-1)60	–4.640
2 (EUFIS)61	–5.267
3 (EQUIPP)59	–5.688
4 (QFCAPS)	–5.431
5 (UCLH/Whit)	–6.003
Quantitative fFNa	1.888	6.61 (4.92 to 8.87)
Age	–	–
BMI	–	–
Smoking	–0.674	0.51 (0.24 to 1.08)
Ethnicity
White	Reference	–
South Asian	0.943	2.57 (0.84 to 7.88)
East Asian	–1.005	0.37 (0.05 to 2.77)
African, Caribbean, Middle-Eastern	–0.207	0.81 (0.43 to 1.52)
Other	–0.305	0.74 (0.19 to 2.82)
Nulliparity	0.364	1.44 (0.92 to 2.24)
Multiple pregnancy	0.832	2.30 (1.35 to 3.92)
Previous spontaneous preterm birth at < 34 weeks’ gestation	–	–
Gestational age at assessment	–	–
Performance
Nagelkerke R2	0.39
AUC (95% CI)	0.89 (0.87 to 0.93)

FIGURE 1.

Calibration plot with predicted vs. observed risk for model A as applied to the complete data set.

Secondary analysis (i): prognostic model based on quantitative fetal fibronectin and clinical risk factors with cervical length data from three European studies (models C and D)

The multiple imputed data sets (m = 60) from the three studies without missing values of cervical length (EUFIS,61 APOSTEL-160 and QFCAPS) were used to develop this model.

The same model development methods were used. The final list of predictors for this logistic model included quantitative fFN, cervical length, maternal age, smoking and gestational age at assessment. Quantitative fFN was not transformed in this model, hence the coefficients for quantitative fFN appear markedly different from those of models A and B.

Table 9 shows the logistic model before and after variable selection prior to adjustment for optimism. The final model identified that high quantitative fFN levels, high maternal age and high gestational age at assessment were associated with increased probability of spontaneous preterm birth within 7 days. Longer cervical length and smoking were associated with a reduced risk. The apparent c-statistic for the model before variable selection (averaged across all multiply imputed data sets) was 0.92 (95% CI 0.90 to 0.94) and the Nagelkerke R² was 0.52. For the final model after variable selection (model C) the c-statistic was 0.92 (95% CI 0.89 to 0.94) and the Nagelkerke R² was 0.51. Internal validation using a non-parametric bootstrap resampling technique resulted in a uniform shrinkage factor of 0.92. Table 10 shows the final model (model C) after adjustment for optimism. The apparent c-statistic for this model was 0.91 (95% CI 0.89 to 0.94) and the Nagelkerke R² was 0.51. Figure 2 shows the calibration plot for model C with predicted versus observed risk in the complete data set.

TABLE 9 - Multivariable logistic analysis of candidate predictors (with cervical length) before (model D) and after (model C) variable selection in three data sets

[(cervical length+ 1)/10]^0.5.

Variable	Model including all variables (model D)		Model after variable selection (model C)
	Intercept	95% CI	Intercept	95% CI
Study
1 (APOSTEL-1)60	–2.932	–7.23 to1.37	–1.960	–5.33 to 1.41
2 (EUFIS)61	–3.535	–7.85 to 0.78	–2.615	–6.00 to 0.77
4 (QFCAPS)	–4.267	–8.89 to 0.36	–3.382	–7.26 to 0.49
Variable	Beta	OR (95% CI)	Beta	OR (95% CI)
Quantitative fFN	0.007	1.012 (1.010 to 1.015)	0.007	1.012 (1.010 to 1.015)
Cervical lengtha	–3.141	0.04 (0.02 to 0.09)	–3.195	0.04 (0.02 to 0.08)
Age	0.039	1.04 (0.99 to 1.09)	0.047	1.05 (1.00 to 1.01)
BMI	0.013	1.01 (0.94 to 1.10)	–	–
Smoking	–1.078	0.34 (0.12 to 0.95)	–1.076	0.34 (0.13 to 0.92)
Ethnicity
White	Reference	–	Reference	–
South Asian	1.440	4.22 (0.72 to 24.80)	–	–
East Asian	–0.686	0.50 (0.05 to 5.24)	–	–
African, Caribbean, Middle-Eastern	–0.505	0.60 (0.23 to 1.58)	–	–
Other	0.119	1.13 (0.21 to 5.93)	–	–
Nulliparity	–0.128	0.88 (0.49 to 1.60)	–	–
Multiple pregnancy	0.490	1.63 (0.85 to 3.14)	–	–
Previous spontaneous preterm birth at < 34 weeks’ gestation	0.328	1.39 (0.56 to 3.45)	–	–
Gestational age at assessment (weeks)	0.100	1.11 (1.00 to 1.22)	0.100	1.11 (1.00 to 1.22)
Performance
Nagelkerke R2	0.52		0.51
AUC (95% CI)	0.92 (0.90 to 0.94)		0.92 (CI 0.89 to 0.94)

TABLE 10 - Final multivariable logistic model (model C) of candidate predictors with cervical length before and after variable selection in three data sets after adjustment for optimism

[(cervical length+ 1)/10]^0.5.

Mean shrinkage factor = 0.92.

Variable	Model including all variables (model C)
	Intercept	95% CI
Study
1 (APOSTEL-1)60	–1.805	–
2 (EUFIS)61	–2.408	–
4 (QFCAPS)	–3.114	–
Variable	Beta	OR (95% CI)
Quantitative fFN	0.006	1.01 (1.01 to 1.01)
Cervical lengtha	–2.942	0.05 (0.03 to 0.11)
Age	0.043	1.04 (1.00 to 1.10)
BMI	–	–
Smoking	–0.991	0.37 (0.14 to 0.99)
Ethnicity
White	Reference	Reference
South Asian	–	–
East Asian	–	–
African, Caribbean, Middle-Eastern	–	–
Other	–	–
Nulliparity	–	–
Multiple pregnancy	–	–
Previous spontaneous preterm birth at < 34 weeks’ gestation	–	–
Gestational age at assessment	0.092	1.10 (0.99 to 1.21)
Performance
Nagelkerke R2	0.51
AUC (95% CI)	0.91 (0.89 to 0.94)

FIGURE 2.

Calibration plot with predicted vs. observed risk for model C as applied to the complete data set.

Secondary analysis (ii): prognostic model based on quantitative fetal fibronectin and clinical risk factors with cervical length data from three European studies (models E and F)

For this model we included all five studies, with imputation of missing values of cervical length in the two studies with missing data (EQUIPP59 and UCLH/Whit). However, in UCLH/Whit, cervical length was measured only in women at high risk of imminent birth (resulting in a shorter mean cervical length of 14 mm). As the missing cervical length data were not MAR in this study, multiple imputation was not feasible. Therefore, we first performed multiple imputation of missing data in all studies separately (m = 75 because of 75% incomplete cases in the EQUIPP study59), except for missing cervical length data in the UCLH/Whit study. The imputed data of all studies were then combined and a single imputation was performed for the completely missing cervical length data in UCLH/Whit study.

The same model development methods were used. The final list of predictors for this logistic model included quantitative fFN, cervical length, smoking, multiple pregnancy and gestational age at assessment. Cervical length was transformed because of non-linearity.

Table 11 shows the logistic model before and after variable selection prior to adjustment for optimism. The final model identified that high quantitative fFN levels, multiple pregnancy and high gestational age at assessment were associated with increased probability of spontaneous preterm birth within 7 days. Large cervical length and smoking were associated with a reduced risk. Quantitative fFN was not transformed in this model, hence the coefficients for quantitative fFN appear markedly different from those of models A and B.

TABLE 11 - Multivariable logistic analysis of candidate predictors with cervical length before (model G) and after (model F) variable selection (after multiple imputation of missing cervical length in five data sets)

[(cervical length + 1)/10]^0.5.

Variable	Model including all variables (model G)		Model after variable selection (model F)
	Intercept	95% CI	Intercept	95% CI
Study
1 (APOSTEL-1)60	–3.751	–7.58 to 0.07	–1.609	–
2 (EUFIS)61	–4.354	–8.19 to –0.51	–2.262	–
3 (EQUIPP)59	–5.391	–9.21 to –1.58	–3.252	–
4 (QFCAPS)	–5.045	–9.19 to –0.90	–3.029	–
5 (UCLH/Whit)	–4.992	–8.99 to –1.00	–2.754	–
Variable	Beta	OR (95% CI)	Beta	OR (95% CI)
Quantitative fFN	0.006	1.01 (1.01 to 1.01)	0.006	1.01 (1.01 to 1.01)
Cervical lengtha	–2.807	0.06 (0.03 to 0.12)	–2.815	0.06 (0.03 to 0.12)
Age	0.031	1.03 (0.99 to 1.1)	–	–
BMI	0.014	1.01 (0.95 to 1.1)	–	–
Smoking	–0.880	0.41 (0.17 to 0.99)	–1.005	0.37 (0.16 to 0.86)
Ethnicity
White	Reference	–	–	–
South Asian	1.165	3.21 (0.83 to 12.33)	–	–
East Asian	–0.696	0.50 (0.06 to 4.10)	–	–
African, Caribbean, Middle-Eastern	–0.116	0.89 (0.44 to 1.82)	–	–
Other	0.108	1.11 (0.28 to 4.42)	–	–
Nulliparity	0.150	1.16 (0.68 to 1.98)	–	–
Multiple pregnancy	0.509	1.66 (0.92 to 3.01)	0.560	1.75 (0.99 to 3.10)
Previous spontaneous preterm birth at < 34 weeks’ gestation	0.329	1.39 (0.65 to 2.95)	–	–
Gestational age at assessment (weeks)	0.101	1.11 (1.01 to 1.20)	0.097	1.10 (1.01 to 1.10)
Performance	Model including all variables (model G)		Model after variable selection (model F)
Nagelkerke R2	0.47		0.47
AUC (95% CI)	0.93 (0.91 to 0.95)		0.93 (0.90 to 0.95)

The apparent c-statistic for the model before variable selection (averaged across all multiply imputed data sets) was 0.93 (95% CI 0.91 to 0.95) and the Nagelkerke R² was 0.47. For the final model, after variable selection the c-statistic was 0.93 (95% CI 0.90 to 0.95) and the Nagelkerke R² was 0.47. Internal validation using a non-parametric bootstrap resampling technique resulted in a uniform shrinkage factor of 0.92. Table 12 shows the final model after adjustment for optimism. Figure 3 shows the calibration plot with predicted versus observed risk in the complete data set.

TABLE 12 - Final multivariable logistic model of candidate predictors with cervical length after variable selection (model F) (after multiple imputation of missing cervical length in five data sets and after adjustment for optimism)

[(cervical length + 1)/10]^0.5.

Mean shrinkage factor = 0.92.

Variable	Model including all variables (model F)
	Beta	OR (95% CI)
Study
1 (APOSTEL-1)60	–1.484	–
2 (EUFIS)61	–2.087	–
3 (EQUIPP)59	–3.000	–
4 (QFCAPS)	–2.794	–
5 (UCLH/Whit)	–2.541	–
Quantitative fFN	0.006	1.01 (1.01 to 1.01)
Cervical lengtha	–2.597	0.07 (0.04 to 0.14)
Age	–	–
BMI	–	–
Smoking	–0.927	0.40 (0.17 to 0.92)
Ethnicity
White	Reference
South Asian	–	–
East Asian	–	–
African/Caribbean/Middle-Eastern	–	–
Other	–	–
Nulliparity	–	–
Multiple pregnancy	0.517	1.68 (0.95 to 2.96)
Previous spontaneous preterm birth at < 34 weeks’ gestation	–	–
Gestational age at assessment	0.089	1.09 (1.01 to 1.19)
Performance measures	Model including all variables (model F)
Nagelkerke R2	0.48
AUC (95% CI)	0.93 (0.90 to 0.95)

FIGURE 3.

Calibration plot with predicted vs. observed risk for model F as applied to the complete data set.

Net benefit analysis for prognostic model with and without cervical length

Figure 4 shows the net benefit curves for the means of the pooled imputation data set for model A (quantitative fFN + clinical risk factors), model C (quantitative fFN + clinical risk factors + cervical length) and cervical length only (at a single threshold of 15 mm, as currently recommended by NICE). 13 Model A and model C allow more women to be correctly identified as low risk up to a risk threshold of around 25%. Model A performs better than model C at risk thresholds below 5%, and model A and Model C perform similarly at thresholds above 5%, suggesting that there is limited clinical value in adding cervical length to the model. Cervical length (using a cut-off point of ≤ 15 mm) appears to have less clinical utility across all risk thresholds than the prognostic models based on multiple predictors.

FIGURE 4.

Net benefit curves for model A, model C and cervical length. Treat all (all) and treat none (none) are plotted for comparison. Cx, cervical length.

Sensitivity analysis for prognostic models for spontaneous preterm birth within 7 days

The results of the sensitivity analysis including tocolysis in models A and B are shown in the Appendix 4, Tables 36 and 37. This was performed to explore any potential confounding effect on time to birth resulting from administration of tocolysis (administration may be associated with other characteristics of the women and may delay birth by up to 48 hours). 52 If tocolysis is effective, it could be expected that tocolysis would be associated with a reduced risk of preterm birth within 7 days. However, the model showed that tocolysis was associated with a significantly increased probability of spontaneous preterm birth rather than having any association with delayed birth.

Prognostic models for spontaneous preterm birth within 48 hours

We had proposed to develop a model for spontaneous preterm birth within 48 hours if there were sufficient numbers of events. However, there were only 71 births within 48 hours in the IPD meta-analysis cohort. Therefore, we decided to combine the IPD meta-analysis cohort with the prospective cohort study for development of this model, recognising that using the maximal sample size is preferred. 63 This model will require further validation in future (see Chapter 6).

Economic model

A literature review was undertaken to inform the model design and identify parameters. MEDLINE, EMBASE, the Cochrane Library and Paediatric Economic Database Evaluation were searched, on 17 January 2017 and with a date range of inception to January 2017, for all economic analyses that included the use of fFN testing in women with threatened preterm labour. Full details of the search strategy are given in Appendix 5, Tables 38–41, and Appendix 5, Figure 18.

A decision tree was developed to illustrate the clinical pathway for each prognostic strategy over the 7-day time horizon. The clinical pathway is the same for each of the three strategies (qualitative fFN, quantitative fFN and cervical length measurement); Figure 5 illustrates one branch of the decision tree (one strategy) for simplification. The decision tree shows the pathway for a pregnant woman presenting with preterm labour, incorporating the diagnostic test accuracy associated with each test (distinguishing between accurate and inaccurate prediction), short-term management and resultant neonatal outcomes (mortality, major morbidity, minor morbidity, full health and did not deliver) at 7 days post testing, and the quality of life and costs associated with these.

FIGURE 5.

Decision tree for alternative strategy pathways.

For each prognostic strategy the tree initiates with the true prevalence of preterm labour at the outset. Presentation of preterm labour that is identified as ‘positive’ by the strategy is diagnosed as preterm labour accurately (true positive) or inaccurately (false positive) and the pregnant woman is hospitalised and receives antenatal corticosteroids (admit and treat), which reduce the risk of neonatal morbidity and mortality. Presentation of preterm labour that is identified as ‘negative’ by the prognostic strategy is diagnosed as term labour accurately (true negative) or inaccurately (false negative) and the pregnant woman is not hospitalised and, therefore, does not receive antenatal corticosteroids. The model can allows for the possibility that a negative test result is over-ruled and the patient is admitted and treated and can also be run under a hypothetical ‘treat-all’ strategy in which everyone is admitted. The base-case analysis did not incorporate any over-ruling; this option to over-rule tests or treat all was explored in sensitivity analyses. The final destination in the decision tree is one of five possible states for preterm births: stillborn, minor morbidity, major morbidity, full health and did not birth within 7 days.

Given this structure, the model accounts for both the clinical and the economic impact of false-negative and false-positive results from the prognostic strategies. False negatives represent failures to treat women with risk-reducing antenatal corticosteroids, so infants associated with false-negative results will not receive antenatal corticosteroids and therefore have a greater probability of experiencing neonatal morbidity and mortality, incurring the associated costs and experiencing the associated quality-of-life and survival effects of these. False positives, on the other hand, will result in women being admitted to hospital unnecessarily, incurring the additional and unnecessary costs of hospitalisation, interhospital transfer and treatment. It is assumed that there are no quality-of-life side effects for receiving unnecessary treatment. This framework allows us to identify the correct diagnoses (true positives and true negatives) and capture the consequences of incorrect diagnoses [both false positives (women admitted and treated unnecessarily) and false negatives (women not admitted and not treated correctly)]; therefore, the alternative prognostic or diagnostic strategies that have the greatest predictive value will have the best outcomes in terms of improved morbidity and mortality outcomes and lower costs to the NHS.

Parameters for the economic model

The key parameters used to populate the economic model include the probabilities of the various neonatal outcomes (did not deliver, stillborn, minor morbidity, major morbidity and full health), admission to NICUs, morbidity risk reduction from antenatal corticosteroids, diagnostic test accuracy outcomes generated from (for part 1) the IPD meta-analysis and (for part 2) predicted prognostic outcomes from the prognostic models (model A, see Table 8, and model C, see Table 10), unit costs and health utilities. The full list of all parameters for the model, their values and sources is given in Appendix 6, Table 42.

Part 1: economic analysis of alternative diagnostic tests available in the UK

Table 14 provides a breakdown of the results for qualitative fFN compared with alternative risk thresholds obtained from quantitative fFN. We compare incrementally the cost-effectiveness of each strategy to obtain the optimal treatment strategy in which qualitative fFN, quantitative fFN and treat all are available options. Table 15 presents our comparison of qualitative fFN, treat all and cervical length measurement at alternative risk thresholds to obtain the optimal treatment strategy in which these three treatment options are available.

The probability of correct diagnosis at 7 days, mean costs and QALDs, incremental costs, incremental effects, ICERs and incremental NMB are presented in full in Tables 14 and 15.

The results in Table 14 show that, in terms of cost per QALD, quantitative fFN at a ≥ 2% risk threshold is associated with a reduction of 0.007 QALDs and a cost reduction of £863 compared with a treat-all strategy. Quantitative fFN at a ≥ 2% risk threshold extended dominates qualitative fFN and quantitative fFN at all other risk thresholds. Appendix 7 details the cost-effectiveness plane and CEACs (see Appendix 7, Figures 19 and 20). In terms of the cost-effectiveness plane, quantitative fFN at a ≥ 2% risk threshold is in the south-west and south-east quadrants, which indicates that this strategy is less costly than a treat-all strategy. Although there is uncertainty as to the distribution of QALDs, the cost-effectiveness plane suggests that quantitative fFN at a ≥ 2% risk threshold is either cost saving or cost-effective compared with a treat-all strategy. The CEAC suggests that, for low values of willingness to pay for QALD gains, quantitative fFN at a ≥ 2% risk threshold has a greater probability of being cost-effective than a treat-all strategy.

At the optimal threshold for admission to hospital using quantitative fFN (a ≥ 2% risk threshold), qualitative fFN has a cost saving of £123 per person, with a reduction of 0.0003 QALDs, resulting in an ICER of > £400,000 per QALD, which is well above the NICE-recommended threshold of £54 per QALD.

In terms of cervical length testing, the results suggest that the use of qualitative fFN is the most cost-effective strategy. Compared with a treat-all strategy, qualitative fFN is cost-effective and all other strategies (including all those involving the use of cervical length measurement) are extended dominated. The cost-effectiveness plane suggests that the use of qualitative fFN is associated with lower costs than a treat-all strategy, but with considerable uncertainty surrounding differences in QALDs. The CEAC suggests that, for very low values of willingness to pay for QALD gains, qualitative fFN has a greater probability of being cost-effective than a treat-all strategy. However, as willingness to pay for QALD gains increases, this reduces to a 50% probability for each strategy. Graphical outputs of the cost-effectiveness plane and CEAC are given in Appendix 7, Figures 22 and 23.

These results provide a clear economic rationale for exploring a prognostic model including the use of quantitative fFN. Cervical length measurement was dominated by qualitative fFN in our analysis, so there is a less convincing case for inclusion of this in a prognostic model. However, as cervical length is recommended for use in current NICE guidelines,13 we have also explored the potential cost-effectiveness of the use of cervical length measurement as part of a prognostic model.

The VOI analysis found that, when comparing quantitative fFN with treat all, the EVPI associated with using quantitative fFN at a ≥ 2% probability of spontaneous preterm birth was £1365 per person at risk in the UK. This is equivalent to 0.0.068 QALDs per person in decision uncertainty when valuing uncertainty on the QALD scale. When comparing qualitative fFN with treat all, the EVPI associated with using qualitative fFN at a ≥ 2% probability of spontaneous preterm birth was estimated at £1243 per person at risk in the UK, equivalent to 0.062 QALDs in decision uncertainty when valuing uncertainty on the QALD scale.

The expected value of perfect parameter information (EVPPI) information for both of these comparisons suggests that the majority of the value of reducing parameter uncertainty in our model would be generated from reducing uncertainty around the parameters relating to test accuracy (that is, the rate of true positives, false positives, false negatives and true negatives). The relative importance of health utilities, costs, probabilities of health states and test accuracy is presented visually in Appendix 7, Figures 21 and 24. The PSA and VOI results together suggest that there is significant uncertainty about which treatment strategy is likely to be cost-effective and, hence, there is considerable value is reducing this uncertainty.

Part 2: results from economic analysis of prognostic models (quantitative fetal fibronectin model versus cervical length model)

Tables 16 and 17 provide the results for models A and C, respectively, in terms of the probability of correct diagnosis at 7 days, mean costs and QALDs, incremental costs, incremental QALDs, ICERs and incremental NMB. The prognostic model results are presented at a range of alternative risk thresholds (probability of spontaneous preterm birth within 7 days), so that the optimal risk threshold to admit to hospital can be estimated.

Table 16 shows that, in terms of cost per QALD, model A (quantitative fFN and clinical risk factors) at a ≥ 2% risk threshold is associated with considerably lower costs and just slightly lower QALDs than a treat-all strategy, and would be considered a cost-effective strategy. Model A at a ≥ 2% risk threshold extended dominates treat all and model A at all other risk thresholds. At the optimal threshold for admission to hospital (a ≥ 2% risk threshold), model A dominates treat all, with a cost saving of £1045 per person and a loss of 0.001 QALDs.

The results (see Table 17) suggest that the use of model C (cervical length and quantitative fFN and clinical risk factors) at a ≥ 5% risk threshold is cost-effective compared with model C at a ≥ 2% risk threshold. All other strategies are extendedly dominated. At the optimal threshold for admission to hospital (a ≥ 5% risk threshold), model C has an incremental cost reduction of £321 per person and no QALD loss compared with treat all. This results in an ICER of £3.96M, far greater than the accepted NICE threshold of £54 per QALD.

Model A at a ≥ 2% risk threshold dominates qualitative fFN, with a cost reduction of £59 and an additional 0.001 QALDs; therefore, we eliminated qualitative fFN as an option and compared model A with model C. To determine the most cost-effective choice of prognostic model between model A and model C, we compared the optimal strategy involving model A with the optimal strategy involving model C. Hence, we compared model A at a ≥ 2% risk threshold with model C at a ≥ 5% risk threshold. Model A was associated with an increase of 0.005 QALDs and a cost reduction of £406 compared with model C; hence, model A dominates model C and is the more cost-effective strategy.

The cost-effectiveness plane is illustrated in Figure 6 and shows the clear increase in costs associated with model C (all points are north of the horizontal axis, showing an increase in cost), but considerable uncertainty surrounds the difference in QALDs (all points cross the vertical axis, showing both QALD gains and QALD losses). The CEAC (see Appendix 7, Figure 25) suggests that, despite this uncertainty, at the NICE willingness-to-pay threshold for QALD gains (£54 per QALD, equivalent to £20,000 per QALY), model A has 100% probability of being cost-effective. Only once willingness to pay is > £1500 per QALD does the probability that model A is the optimal strategy reduce to almost 50%.

FIGURE 6.

Cost-effectiveness plane: model A at a ≥ 2% probability of spontaneous preterm birth compared with model C at a ≥ 5% probability of spontaneous preterm birth.

The VOI analysis found that the EVPI associated with the comparison between using model A at a ≥ 2% risk threshold with using model C at a ≥ 5% risk threshold is £1364 per person at risk in the UK. This is equivalent to 0.068 QALDs per person in decision uncertainty when valuing uncertainty on the QALD scale.

As with the individual test comparisons in part A, the EVPPI for both of these comparisons suggests that the key drivers of uncertainty in the model is related to test accuracy/prognostic value (that is, the rate of true positives, false positives, false negatives and true negatives). The relative importance of health utilities, costs, probabilities of health states, and test accuracy has been presented visually (see Figure 26). These findings show that further research into the prognostic accuracy of the test is potentially worthwhile if it were to cost less that £1364 per person.

Strengths and weaknesses in relation to other studies

Over the 7-day time horizon, quantitative fFN had lower costs to the NHS and a greater probability of correct diagnosis than qualitative fFN, and also resulted in QALY gains. We found no previously published economic studies that included quantitative fFN. Previous studies have explored cost-effectiveness of cervical length measurement compared with usual care (risk factors and clinical judgement),13 or cost only and cost-effectiveness analyses of qualitative fFN,11,12,75,76 finding fFN testing to be cost saving in comparison with usual care. 12 Our findings corroborate this but have added new evidence that quantitative fFN is superior to qualitative fFN. The economic model adopts a structure similar to those used by other economic studies of tests for predicting preterm labour,11,12,75,76 capturing the diagnostic outcomes and the resultant clinical and economic impacts. Our literature search identified the same studies as those in a recent HTA report. 12 Inclusion of QALD in the model was of importance to account for the quality of life and mortality impacts of hospitalisation and receiving treatment to enable the analysis to capture the resultant management implications and resultant health impacts on women over the 7-day time horizon. A recent NICE overview of biomarker tests for predicting preterm labour77 also included QALY outcomes over a short time horizon and likewise found that moving from probability of diagnosis to inclusion of QALYs results in a reduction in QALYs with cost-savings for the NHS (results that reside in the south-west quadrant of the cost-effectiveness plane).

Strengths and weaknesses of the study

This study includes comprehensive and the most current research on quantitative fFN studies; limitations relate mainly to the primary data available for inclusion in our analyses. Our study excludes alternative biochemical tests that are now available (such as PartoSure and Actim Partus) because evidence on these was unavailable at the time of study commencement and they were outside the remit of the original QUIDS study. Our study has developed prognostic models for women with symptoms of preterm labour based on the six most promising prognostic strategies. These have been internally validated and have shown that model A, which includes quantitative fFN and clinical risk factors, is likely to be the most cost-effective strategy. We have found that inclusion of cervical length in a prognostic model in addition to quantitative fFN (model C) is unlikely to be cost-effective. However, there is some uncertainty about this because the data for this model come from studies performed outside the UK and in higher-risk populations. Therefore, it would also be reasonable to try to externally validate this model in a UK population.

Meaning of the study: implications for clinicians and policy-makers

Our study has several implications for clinical practice. Current NICE guidelines13 recommend the use of transvaginal ultrasonography cervical length measurement for predicting preterm birth in women at > 30 weeks presenting with signs and symptoms of preterm labour; however, a lack of specific equipment, skills and capacity around the clock means that this is rarely part of routine practice. Instead, qualitative fFN is the most commonly available test. 10 Our findings suggest that quantitative fFN in combination with clinical risk factors in a prognostic model (model A) is superior to also including cervical length (model C) in terms of probability of correct diagnosis and NMB outcomes, with a willingness to pay of £54 per QALD (equivalent to the £20,000/QALY threshold recommended by NICE). Model A was found to improve outcomes (0.005 QALD gains) compared with model C, with a cost saving of £406; hence, model A dominates model C and is the more cost-effective strategy.

Thus, it seems likely that use of the quantitative fFN prognostic model could replace the use of qualitative fFN and that efforts to implement 24-hour cervical length measurement are unnecessary. However, given these results and the uncertainty surrounding cervical length measurement included in our model, we concluded that it would be worthwhile to endeavour to include cervical length measurement in the prospective cohort study, where sites were able to do so, to strengthen the evidence base on cervical length measurement, its value within a prognostic model and the cost-effectiveness of this approach. Our findings from the economic analyses predict that a quantitative fFN prognostic model will improve predicative accuracy and reduce costs to the NHS compared with qualitative fFN.

Unanswered questions

Our analysis finds that quantitative fFN is superior to qualitative fFN, and a prognostic model of quantitative fFN and clinical risk factors is superior to a prognostic model of quantitative fFN and cervical length. The optimal choice between prognostic strategies that include quantitative fFN and cervical length measurement has considerable uncertainty and should be explored further in the UK clinical setting (i.e. in the validation cohort study). These findings provide a clinical and economic rationale for the prospective cohort study to validated the quantitative prognostic model and collect evidence on cervical length in the UK. A VOI analysis was undertaken and found that further research to reduce current uncertainty regarding the alternative prognostic strategies is worthwhile. In Chapter 6 we detail the results of this further research.

Ethics and registration

The study was conducted in accordance with the principles of Good Clinical Practice (GCP). The study was approved by the West of Scotland Research Ethics Committee (16/WS/0068). The study was registered with International Standard Registered Clinical/soCial sTudy Number (ISRCTN) ISRCTN41598423 and the National Institute for Health Research (NIHR) Central Portfolio Management System (CPMS31277), and the protocol has been published. 78

Population and eligibility

The prospective cohort study included women with signs and symptoms of preterm labour at 22⁺⁰ to 34⁺⁶ weeks’ gestation in whom admission, transfer or treatment for preterm labour was being considered. 78

Inclusion criteria at initial assessment were women:

• at 22⁺⁰ to 34⁺⁶ weeks’ gestation (or earlier gestation if the fetus was considered potentially viable)78

• with signs and symptoms of preterm labour including any or all of the following – back pain, abdominal cramping, abdominal pain, light vaginal bleeding, vaginal pressure, uterine tightenings and contractions78

• for whom hospital admission, interhospital transfer or treatment (antenatal steroids, tocolysis or magnesium sulphate) was being considered owing to signs of preterm labour78

• aged ≥ 16 years. 78

Additional inclusion criteria at speculum examination were:

• cervical dilatation of ≤ 3 cm78

• intact membranes78

• no significant vaginal bleeding, as judged by the clinician. 78

Exclusion criteria were:

• Contraindication to vaginal examination (e.g. placenta praevia). 78

• Higher-order multiple pregnancy (triplets or more). 78

• Moderate or severe vaginal bleeding. 78

• Cervical dilatation of > 3 cm. 78

• Confirmed rupture of membranes. 78

• Sexual intercourse, vaginal examination or transvaginal ultrasonography in the preceding 24 hours. These factors may invalidate results. Women who meet these criteria were initially excluded from the study but could be included if still symptomatic after 24 hours, when fFN test accuracy would be considered to be restored. 78

Once it was established that a participant met the above criteria, the fFN swab was taken.

The broad inclusion criteria reflect current clinical practice and aimed to help ensure the generalisability of the results of the study for routine clinical care. We included women who reattended ≥ 7 days after initial recruitment with signs and symptoms of preterm labour, as well as women who remained symptomatic but undelivered 7 days later in whom repeat testing by the clinician was deemed to be appropriate. This is in line with manufacturer’s recommendations for fFN testing. These data have not been analysed in this QUIDS report (which has focused on predictive ability of the first quantitative fFN result), but may be used in future supplementary analyses. 78

Co-enrolment in other non-interventional studies was allowed; however, co-enrolment in trials of tocolytic treatments or other management strategies that might influence the timing of birth was not allowed because it may have affected the primary outcome of birth within 7 days of testing. Participation in the QUIDS study did not, however, preclude babies being subsequently involved in interventional trials. 78

Setting

The prospective cohort study took place in 26 consultant-led obstetric units in the UK (see Appendix 8, Table 43). More than 93% of pregnant women in the UK deliver in consultant-led units,1 and the vast majority of women with symptoms of preterm labour present to a consultant-led unit for assessment, either directly or following advice from their community midwife or general practitioner. The study did not include any community maternity units (staffed by midwives, with or without involvement of non-obstetric medical staff). Community maternity units cover only a very small proportion of the population and are located mainly in remote and rural areas. In the Perinatal Collaborative Transport Study (CoTS) of perinatal transfers in Scotland,4 which involved over 50,000 births, only 69 (0.13%) women were transferred to a consultant-led obstetric unit from a community maternity unit. Because of the small number of women cared for in community maternity units, their inclusion was not felt to be an efficient use of study resources. 78

Given that management of women with symptoms of preterm labour and interhospital transfer patterns depend partly on the level of available neonatal care and distance to transfer, we included a mixture of hospitals with different levels of neonatal care facilities in both rural and urban settings. 78 We included units with three different levels of neonatal care: SCBU (providing special care for their own local population), LNU (providing special care and high-dependency care and a restricted amount of intensive care) and NICU (larger intensive care units providing the whole range of medical, and sometimes surgical, neonatal care for their local population and for babies and their families referred from the neonatal network in which they are based, and other networks where necessary). The hospitals were chosen from different geographical settings (rural/urban) and different regions of the UK to help to ensure generalisability of findings. 78

Participant selection and enrolment

Women with signs and symptoms of preterm labour were identified on presentation to obstetric services. 78 A member of clinical staff, usually the doctor or midwife assessing the woman, identified potentially eligible participants, provided a participant information leaflet and invited consent. A suitably trained member of clinical staff (doctor or midwife) or research team was responsible for gaining consent from participants. 78

Posters and leaflets were situated in antenatal areas of participating hospitals to alert women that the study was taking place, and women were allowed as much time as possible to consider participation without unduly delaying further clinical assessment.

Screening for eligibility

The clinical likelihood of preterm birth is usually evaluated by history and examination. This includes abdominal palpation to assess the strength and frequency of uterine contractions. 78 If preterm labour is suspected, a vaginal speculum examination is performed in which the cervix is inspected for dilatation and evidence of vaginal bleeding and membrane rupture is assessed. 78 Swabs for fFN (or another biochemical test of preterm labour) are usually taken at this point, and cervical length ultrasonography is performed if the facilities are available at point of care.

Potential participants in the QUIDS study were identified after the initial assessment and provided with information about the study at this stage. A combined ‘screening and consent form’ was used as a self-screening tool for potentially eligible participants. Informed consent was provided before speculum examination. This approach meant that samples were collected at routine speculum examination, as they are in clinical practice, and participants avoided an additional vaginal examination. However, this approach meant that certain exclusion criteria could be applied only at speculum examination (e.g. vaginal bleeding or evidence of ruptured membranes), so a proportion of women were not be eligible for fFN testing after consent was given. These data are reported but not included in analysis. Women were also able to withdraw consent for use of their data at any time until the end of the study. 78

Study assessments and data collection

Study assessments are shown in Appendix 9, Table 44. Baseline demographics were collected on participants, including height and weight (at booking), information on medical history, obstetric history, estimated date of birth and presenting signs and symptoms. Samples for fFN analysis were taken with a fFN specimen collection kit, as per the manufacturer’s instructions. 16 The sample was run at a near-bedside Rapid fFN 10Q analyser specially adapted for the QUIDS study, which revealed a qualitative fFN result (positive/negative/invalid based on a threshold of 50 ng/ml) for clinicians to base clinical decision-making on, in accordance with local protocols. 78 The quantitative fFN result was masked from clinicians and stored as a three-letter code. 78 Samples were run as per the manufacturer’s instructions (described in Chapter 2). Hologic, Inc. offered training on sample collection and analysis to staff at sites participating in the study.

Screening data and data on quantitative fFN were collected on paper-based case report forms (CRFs), which were then inputted into a web-based electronic database by research staff. 78 All other data were collected from the participant records and recorded on the study database.

Data were collected on the following candidate predictors: fFN concentration, previous spontaneous preterm labour, gestation at fFN test, age, ethnicity, BMI, smoking, deprivation index (derived from postcode), number of uterine contractions in 10 minutes, cervical dilatation, vaginal bleeding, previous cervical treatment for CIN, cervical length (measured by transvaginal cervical length when available), singleton/multiple pregnancy, tocolysis and fetal sex. 78

Although we wanted to explore the added prognostic value of cervical length in our model validation, and we purposively selected sites that reported that facilities for cervical length were available, we did not make cervical length measurement mandatory. Doing so would have made the study extremely difficult to implement in the concurrent NHS setting. The majority of units do not have 24-hour availability of transvaginal ultrasonography and/or trained personnel to perform scans at point of care when women present with symptoms of preterm labour. Inclusion of cervical length could also have decreased recruitment rate (owing to the need for additional transvaginal ultrasonography examination) and would have required significant additional resources. Data on cervical length were thus, collected only when available.

Outcome and resource use data included78 gestational age at birth, date and time of birth, administration of treatments for preterm labour (steroids, antibiotics, tocolysis or magnesium sulphate), duration of hospital admission, hospital transfer, onset of labour [preterm prelabour rupture of membranes (PPROM); idiopathic preterm birth; medically indicated preterm birth (and indication)], place of birth (base hospital, other hospital or not in hospital), mode of birth, neonatal admission, neonatal complications, perinatal mortality, congenital anomaly, sex and birthweight.

Additional data were collected on alcohol use, employment group and education level, domestic violence, preterm labour symptoms and cervical dilatation.

At recruitment, participants were invited to complete an optional baseline State-Trait Anxiety Inventory (STAI) questionnaire and were provided with a second questionnaire to be completed 24–48 hours later (with a prepaid envelope to be returned by post). Acceptability of fFN testing and the decision support were assessed using follow-up interviews with a subgroup of participants (n = 30),78 which are detailed in Chapter 8.

Quality assessments

The Rapid fFN 10Q analyser has integrated quality control measures; records of these and staff training were kept. A precalibrated reusable quality control cassette was used to verify that analyser performance was within specification. Quality control checks were mandatory, that is a test sample would not be analysed if quality control had not been performed in the preceding 24 hours. Logs of results were stored on the machine and monthly paper logs of quality control tests were also kept. 78 Each fFN test has an internal quality control: a procedural control line that checks the threshold level of signal by the instrument. Sample flow detection ensures that the sample travels across the cassette properly and confirms absence of conjugate aggregation. All participating sites were requested to also enrol in the Wales External Quality Assurance Scheme (WEQAS) point-of-care quality assurance scheme. 69 The WEQAS provided a sample for analysis to each site bimonthly and provided confirmation of analyser performance and variability.

Sample size calculation

Our initial sample size calculation (n = 1602) was based on an anticipated event rate of between 6% and 12%. However, it quickly became apparent that our event rate was lower than estimated (stabilising at around 3%). Furthermore, new guidance emerged recommending a minimum of around 100 events and 100 non-events for prognostic model validation. 79 Therefore, we revised our sample size calculation during the study, aiming for 3000 participants, to obtain approximately 100 events of preterm birth within 7 days of testing.

Validation of the QUIDS prognostic model

A statistical analysis plan was drawn up and agreed prior to database lock and analysis.

Validation of models for preterm birth within 7 days of test

Table 19 shows external validation of models A (after variable selection) and B (including all variables). The c-statistic for model A (pooled from the multiple-imputation data sets) was 0.89 (95% CI 0.85 to 0.93). The c-statistic for model B before variable selection (pooled from the multiple-imputation data sets) was the same at 0.89 (95% CI 0.85 to 0.93). Figures 8–11 show the calibration plots for models A and B before and after recalibration with updated intercepts. The majority of participants in the study had a very low risk of preterm birth within 7 days, and the majority of events happened in the highest decile of risk. The calibration plots indicate that the models were underfitted, with a higher proportion of observed events than expected events (see Figures 8 and 10). Recalibration with update of the intercept improved the calibration, as shown by a calibration-in-the-large close to 0 and a calibration slope close to 1 (see Figures 9 and 11).

FIGURE 8.

Calibration plot with predicted vs. observed risk for model A (after variable selection) as applied to the prospective cohort study data (after multiple imputation; multiple imputation set 6 used to generate plot). LOWESS, locally weighted scatterplot smoothing.

FIGURE 9.

Calibration plot with predicted vs. observed risk for model A (after variable selection) as applied to the prospective cohort study data (after multiple imputation; multiple imputation set 6 used to generate plot) after recalibration with updated intercept. LOWESS, locally weighted scatterplot smoothing.

FIGURE 10.

Calibration plot with predicted vs. observed risk for model B (all variables) as applied to the prospective cohort study data (after multiple imputation; multiple imputation set 6 used to generate plot). LOWESS, locally weighted scatterplot smoothing.

FIGURE 11.

Calibration plot with predicted vs. observed risk for model B (all variables) as applied to the prospective cohort study data (after multiple imputation; multiple imputation set 6 used to generate plot) after recalibration with updated intercept. LOWESS, locally weighted scatterplot smoothing.

Sensitivity analyses

The model performance measures of models included in prespecified sensitivity analyses are presented in Appendix 11, Table 46, alongside those of the primary analysis for comparison. Model performance was similar across primary and sensitivity analyses.

Net benefit analysis

Figure 12 shows the net benefit of models A and B compared with treat-all and treat-none strategies. As would be expected, both models appear to have very similar net benefits, which are better than treat all or treat none up to a threshold of 20% risk.

FIGURE 12.

Net benefit curves for externally validated models A and B plotted across a range of thresholds used to identify women at low risk of preterm birth within 7 days. Treat all and treat none are plotted for comparison. All, treat all; none, treat none.

Development and internal validation of model for spontaneous preterm birth within 48 hours of test

We combined the original model development cohort with the prospective cohort in a new IPD meta-analysis for development and internal validation of a model for spontaneous preterm birth within 48 hours.

The baseline characteristics are shown in Table 20.

Table 21 shows the logistic model after adjustment for optimism (uniform shrinkage factor of 0.93). The final model identified that high quantitative fFN levels, gestational age at assessment (weeks), age, smoking, nulliparity, multiple pregnancy and previous spontaneous preterm birth at < 34 weeks’ gestation were all predictors.

Resource use and cost estimates

The following resource items were collected in the cohort study: maternal admission, neonatal admissions, complications, transfers and treatments given. The equation below illustrates the main cost components in terms of the total mean cost per patient:

Resource use data were collected via CRFs for women who were admitted to hospital with signs and symptoms of preterm labour. Data on subsequent birth in woman who were not admitted hospital following assessment were not captured. Appendix 12, Table 47, details the resource use items that were collected and recorded, the unit cost applied and the accompanying references. All unit costs were collected or converted into Great British pounds (£) for the price year 2017/18. Unit costs were collected from routine sources such as the British National Formulary,70 Personal Social Services Research Unit69 and NHS Reference Costs. 65 Some unit costs that are not available on the British National Formulary,70 such as the cost of a cervical length measurement, were obtained from published literature or from expert (e.g. the project management team) opinion.

Clinical outcomes

The prognostic model A (see Chapter 4) was validated in 26 maternity units across the UK. The data set included 3049 women. Following the exclusion of women who did not meet the eligibility criteria, the final data set was a cohort of 2924 women. The validation study generated data on quantitative fFN prognostic model performance, birth outcomes and demographic characteristics of the women involved. The cohort study results for the quantitative fFN prognostic model (the result of the linear predictor) were transformed into a percentage probability (between 0% and 100%) of spontaneous preterm birth within 7 days. Appendix 12, Table 48, provides all model parameters used in the cost-effectiveness analysis.

Cost-effectiveness analysis of cohort study: within 7 days

The base-case cost-effectiveness analysis compared the cohort study results for qualitative fFN with the quantitative fFN prognostic model results, reporting the incremental cost per QALD gained and NMB using a willingness-to-pay threshold per QALD of £54.79, equivalent to the UK threshold of £20,000 per QALY.

Lifetime analysis: lifetime horizon

To capture the longer-term cost and health outcomes associated with infants who did or did not receive treatment, the 7-day outcomes were extrapolated over a lifetime horizon. The 7-day time horizon does not capture the costs and quality-of-life impacts of minor and major morbidity that may endure beyond 7 days. For the lifetime horizon, QALYs rather than QALDs are presented and the key assumptions are that:

• Infants who are incorrectly not treated (i.e. false negatives) and experience major morbidity during the 7-day time horizon have lifetime cost and health implications. It is assumed that minor morbidity does not extend beyond 7 days.

• The quality of life for major morbidity is represented by the health utility associated with intraventricular haemorrhage (proxy for cerebral palsy). Lifetime costs associated with lifetime care for women with cerebral palsy (£115,000 lifetime care) are incorporated. 77 Lifetime healthy utilities are based on an infant’s state in the model at 7 days (dead, minor morbidity, major morbidity or healthy). These utilities67 are extrapolated over an average lifespan and discounted to the present value. We do not capture the natural decreasing time profile of health utility over a lifetime because this is not known for infants with minor or major morbidity at 7 days.

Sensitivity analyses

Scenario analysis

Owing to low levels of data on cervical length from the cohort study (see Chapter 6), we report summary statistics for the use of cervical length measurement for the prognosis of spontaneous preterm birth within 7 days.

Resource use estimates from cohort study

Table 22 provides a breakdown of the resource use and cost estimates relating to women admitted to hospital at risk of suspected preterm labour. These data were used in the final health economic model to estimate the cost-effectiveness of alternative prognostic strategies in a UK clinical setting. Full details of the assumptions made regarding resource use data are given in Appendix 13.

Cost-effectiveness results

Table 23 details the cost-effectiveness results of the cohort study comparing qualitative fFN with the quantitative prognostic tool over a 7-day time horizon. Table 24 details the results of the lifetime extrapolation, comparing qualitative fFN with the quantitative prognostic tool. In both tables, a scenario analysis of ‘treat all’ is included; this can be compared with the qualitative fFN results and an optimal quantitative fFN treatment threshold identified.

The results in Table 23 show that, in terms of cost per QALD over a 7-day time horizon, the prognostic tool (model A: quantitative fFN and clinical risk factors) at a ≥ 5% risk threshold dominates qualitative fFN and the tool at all other risk thresholds. At a risk threshold of ≥ 2% the prognostic tool (model A) gives the greatest incremental NMB and is the optimal choice. Compared with a treat-all strategy, the prognostic tool at a ≥ 2% risk is associated with a reduction in QALDs of 0.0005 and a cost reduction of £866. In terms of the cost-effectiveness plane, this strategy is in the south-west quadrant, which indicates that the most cost-effective option is the strategy that produces the greatest cost reduction per unit of health lost, in this case admitting to hospital at a ≥ 2% risk threshold predicted by the prognostic tool. All other options here are extendedly dominated.

Comparing the optimal threshold for admission to hospital (a ≥ 2% risk threshold) against qualitative fFN, the prognostic tool was found to increase costs by £41 per patient with a QALD gain of 0.002.

The results in Table 24 show that, in terms of cost per QALY, the prognostic tool (model A: quantitative ffN and clinical risk factors) at a ≥ 2% risk threshold is a cost-effective strategy. Compared with a treat-all strategy, the tool at a ≥ 2% risk threshold is associated with a reduction in QALYs of 0.0006 and a cost reduction of £840. In terms of the cost-effectiveness plane, this strategy is in the south-west quadrant, which indicates that the most cost-effective option is the strategy that produces the greatest cost reduction per unit of health lost, in this case admitting at a threshold of ≥ 2% risk. Qualitative fFN and the prognostic tool at all other risk thresholds are extendedly dominated. In comparison with qualitative fFN, the prognostic tool (model A) at a ≥ 2% risk threshold has an incremental cost of £40 with 0.008 QALY gain, resulting in an ICER of £5000 per QALY over a lifetime horizon. This is highly cost-effective given the recommended NICE threshold of £20,000 per QALY.

Sensitivity analysis

Probabilistic sensitivity analysis

The PSA was run for the optimal quantitative fFN strategy (≥ 2% probability of spontaneous preterm birth) compared with treat all. This analysis found that the majority of data points occurred in the south-west and south-east quadrants of the cost-effectiveness plane (i.e. where a treatment rule based on a ≥ 2% probability is less costly), but may lead to a small increase or decrease in QALDs. Hence, the PSA demonstrates that there is significant uncertainty regarding which is the most cost-effective strategy. This is illustrated in Figure 13. The CEAC is presented in Appendix 12, Figure 27.

FIGURE 13.

Cost-effectiveness plane: incremental cost and QALDs of the quantitative fFN prognostic tool at a ≥ 2% risk threshold vs. treat all.

Figure 14 illustrates the results of the qualitative fFN and the various quantitative fFN risk thresholds plotted on a ROC curve. This depicts the trade-off between the sensitivity and the specificity of alternative treatment probability thresholds used in the cost-effectiveness analysis. The strategy for which the data point is closest to the top-left side of the ROC space is regarded as the most accurate. This would suggest that an admit strategy of ≥ 15% probability would be regarded as the most accurate strategy.

FIGURE 14.

The ROC plot for the quantitative fFN prognostic tool (at a ≥ 2% risk threshold).

Figure 15 presents the NMB associated with each treatment threshold. However, in contrast to the ROC curve presented in Figure 14, this would suggest that a treatment threshold of ≥ 2% probability of spontaneous preterm birth is the optimal strategy. This is because the NMB approach considers both the costs and the quality of life consequences of false positive and false negatives. Under this approach, the results would suggest that a strategy which minimises false positives is likely to be the most cost-effective.

FIGURE 15.

The NMB at alternative admit probability thresholds.

Scenario analysis

Among the 2924 women included in the final analysis, 93 women (3.18% of the population) had cervical length measured. Owing to the lack of data on cervical length measurement in the cohort study (which provides insight into the logistic and technical restrictions of using this as a diagnostic tool in clinical practice), statistical analysis of these data was limited. Indeed, it would have been inappropriate to undertake multiple imputation for the remaining 97% of the study sample.

Strength of this study

This study was based on clinical and resource use data gathered from women being treated in a UK clinical setting. To our knowledge, this is the first UK study in which the qualitative fFN and quantitative fFN prognostic tools have been validated in a large population (2924 women in 26 maternity units in the UK).

Limitations of this study

Resource use data were collected for woman admitted to hospital with signs and symptoms of preterm labour. Data on the subsequent birth of woman who were not admitted to hospital following assessment were not captured. It may be the case that, among the woman who were incorrectly sent home (false negatives), subsequent hospital admission and treatment costs may have been higher. This would have the impact of increasing the cost of resource use in the economic model. However, because the number of false negatives is small (1–2% across all admit strategies), the impact of any additional resource use for these women is unlikely to have much impact on the overall resource use and cost estimates.

Estimating the quality of life for infants is challenging. Our model required health utility estimates for infants born with minor or major morbidity. We chose as a proxy for these states severe respiratory distress syndrome and moderate cerebral palsy, respectively. Because it is not possible to elicit health utilities directly from infants, these utilities were elicited from parents using the standard-gamble preference-elicitation technique. To estimate preferences for public health spending, this technique should be applied to the general population rather than to any specific patient group (the parents, in this case). However, no such health utility values were identified in our review. Alternative utility estimates from other recent studies were included in the sensitivity analyses. 81

Women’s outcomes in the model are determined from the diagnostic test or prognostic model outcomes, which then feed into the economic decision tree model to determine the proportion of women who enter into each of four possible health states (death, minor morbidity, major morbidity or healthy), according to which prognostic strategy is implemented and, hence, whether or not the patient receives treatment. The trade-off between unnecessary treatment (false positives) and missed cases (false negatives) is of importance in driving the model outcomes, specifically how these are weighted with costs and utilities. Because there are only a relatively small number of women who are classified as false positives or false negatives [compared with > 80% of women who are correctly sent home (true negatives)], the actual difference between the proportion of women who enter into the minor or major morbidity states, depending on which prognostic strategy is used, is in fact very small. This means that the overall difference in cost-effectiveness between strategies is small.

Summary of principal findings

• In comparison with qualitative fFN, the prognostic tool at a ≥ 2% risk threshold is highly cost-effective over a lifetime horizon. The results showed an incremental cost of £40 with a 0.008 QALY gain, resulting in an ICER of £5000 per QALY, which is highly cost-effective given the recommended NICE threshold of £20,000 per QALY.

• Current NICE guidance recommends using qualitative fFN or cervical length measurement (as individual tests). Our study suggests that the use of quantitative fFN as part of a prognostic tool is highly cost-effective and should, therefore, be considered for use in clinical practice.

• The use of the prognostic model validated at a treatment threshold of 2% probability of spontaneous preterm birth within 7 days dominated using qualitative fFN alone (i.e. it is more effective and less costly).

• There is only a small difference in costs and QALDs across alternative admit probability threshold strategies.

• This suggests that there remains uncertainty surrounding the optimal treatment probability threshold when using the prognostic model. This is driven by the uncertainty surrounding the true health and monetary cost of infants who are born preterm to mothers who are not given treatment (i.e. false negatives). Further research aimed at understanding the full cost and health outcome impact of preterm labour is warranted.

State-Trait Anxiety Inventory

The STAI assesses both state anxiety (situational anxiety) and trait anxiety (a person’s tendency to anxiety). At recruitment to the QUIDS prospective cohort study, participants were invited to complete an optional pre-test STAI questionnaire and provided with a second questionnaire to be completed 24–48 hours later (with a prepaid envelope to be returned by post). Pre- and post-test state and trait anxiety scores were compared (using Student’s t-test) and analysed using multiple logistic regression adjusting for qualitative fFN result, pre-test score and centre.

Sample and recruitment

We purposively sampled 30 women and 30 clinicians from a subset of trusts (n = 14) that had a high response rate to the STAI questionnaires88 as of January 2017. Women who had completed and returned both STAI questionnaires and had indicated an interest in being interviewed were sent a participant information sheet (PIS) by post and contacted by telephone. Clinicians were approached by research midwives or the local researcher, with the aim recruiting clinicians with a range of professional experience. Separate written consent was sought from all participants after they had been given time to read the PIS and the opportunity to ask questions.

Data collection

Data were collected via a semistructured interview conducted over the telephone. Separate structured topic guides were designed for women and clinicians; however, all participants were invited to speak freely about their experiences. Clinicians were also presented with a clinical scenario and asked to discuss their management plan based on clinical findings only, qualitative fFN, quantitative fFN and, finally, the QUIDS prognostic model result. All interviews were audio-recorded with consent and field notes were taken. Audio recordings were transcribed verbatim.

Data analysis

Interview data were analysed using a framework approach (see Chapter 3).

State-Trait Anxiety Inventory

Among the 2924 participants in the QUIDS study, 1876 (64.16%) completed a baseline STAI questionnaire and 412 (14.1%) returned both a baseline and a post-test STAI form. Baseline characteristics of the women who did not complete either questionnaire, those who completed baseline questionnaire only and those who competed baseline and post-test questionnaires are shown in Table 25. The proportion of smokers was lower in the group that completed both pre-test and post-test questionnaires, and there was a higher proportion of white women in the group that completed both questionnaires. However, mean baseline scores were similar in all women who competed a baseline questionnaire, whether or not they completed the post-test questionnaire, and there were similar proportions of women who had a spontaneous preterm birth within 7 days across the groups.

Table 26 shows the pre- and post-test STAI scores for the 415 women who completed both questionnaires. As would be expected, STAI trait anxiety scores (which represent inherent tendency to anxiety) were similar at baseline and posttest, with a mean change of –0.12 points (SD 5.99 points), and qualitative fFN results did not influence these. In contrast, post-test STAI state anxiety scores (which represent situational anxiety) were lower than at baseline overall, with a mean change of –5.6 points (SD 11.74 points), which is considered a clinically significant reduction. 89 However, women who had a positive qualitative fFN result did not have a reduction in their state anxiety score, and positive qualitative fFN results were associated with greater state anxiety than negative qualitative fFN results [adjusted OR 4.33 (95% CI 1.67 to 6.99); adjusted for baseline test score and centre], with a mean score above the cut-off point of 39–40 points, which has been suggested to detect clinically significant symptoms. 89

Acceptability

A total of 104 individuals from 14 NHS trusts across England, Scotland and Wales consented to take part in the acceptability interviews (32 women and 72 clinicians) and 61 participated [30 women and 31 clinicians (1 partial interview)] (Table 27). Forty-three individuals (two women and 41 clinicians) were unable to commit to a time for the interview or were uncontactable following consent. The trusts covered a range of geographical locations and included large tertiary-level maternity units and smaller district general hospitals.

Findings from women’s interviews

The women who participated in interviews had completed and returned the STAI questionnaires pre and post testing (Table 28).

Analysis of the women’s interviews revealed four main themes and five subthemes. The theme ‘anxiety and influencing factors’ ran across each of the other three themes, which were ‘deciding to access care’, ‘fFN testing’ and ‘impact of the fFN result’.

Anxiety and influencing factors

Anxiety, or a lack of it, was a theme that ran throughout the interviews. Some women reported feeling anxious at different time points during their experiences and some reported feeling calm throughout. This finding is also reflected in the results of the STAI questionnaires (see Table 26). Among those who did report feeling anxious, the main reason was concern for their baby. Women reported worrying about various morbidities and mortality:

You know, I thought the baby could have this or the baby could have that, or it could, you know, not survive.

11008: positive fFN, no event, term birth

However, some women explained that they did not feel anxious during their experiences because they felt that they were being cared for by the right professionals in the right place:

I wasn’t too overly stressed. I knew basically that I was in the best, safe possible hands and, you know, what will be will be, basically.

11195: positive fFN, no event, term birth

It is notable that this woman indicated high levels of state and trait anxiety at both time points in her STAI questionnaire (Table 29). This may demonstrate the effect of time on anxiety recall, especially when the pregnancy ended in a term birth without complication.

By contrast, another woman linked her lack of anxiety to her generally laid-back nature:

I was like what will be will be, if it’s positive it’s positive, and if it’s not it’s not, you know. Then literally I’m one of those sort of go with the flow.

26007: positive fFN, no event, term birth

This matched her STAI results, which indicated state and trait anxiety scores below the mean for anxiety in pregnancy. 90

There appeared to be a link between women’s perception of their symptoms and women’s anxiety. For some women anxiety heightened their perception of symptoms, whereas for others anxiety resolved when symptoms did.

Anxiety commenced or was heightened when clinicians demonstrated concern about women’s symptoms. The invitation to attend hospital or in some cases introducing the concept of fFN testing to rule out preterm labour caused anxiety:

They suggested having this test and that was when it first really . . . it never occurred to me that I might be going into preterm labour. So that was maybe a bit shocking but I never really thought that I was.

26087: negative fFN, no event, term birth

Deciding to access care

All women were invited to talk about their experience of having symptoms of preterm labour. Women talked about their experience of initial symptoms and their decision to contact maternity services. Symptoms included vague sensations that can be common to pregnancy – backache, abdominal tightening sensations, Braxton Hicks contractions, vaginal discharge or dampness and abdominal, vaginal or rectal pressure – and generalised feelings of being unwell, such as having an upset stomach and flu. For some women, symptoms were more clearly indicative of labour, such as contractions and having a ‘show’. The women who experienced more vague symptoms commonly talked about the feeling that ‘something wasn’t right’, indicating that they needed to contact maternity services:

I felt that something wasn’t quite right, like I felt that . . . yeah, I just felt something was different.

11001: positive fFN, event, preterm birth at 34 weeks’ gestation

A complicating factor in making the decision to contact maternity services was that women said that they were not expecting preterm labour and did not consider this to be the reason for their symptoms:

At that point it wasn’t something that even entered my head that it could be potentially going into labour.

11185: positive fFN, no event, term birth

Persistence or severity of symptoms was the driver for most women to contact maternity services. For some women, their previous experiences of childbirth helped them to decide when they needed to call. For some women the decision to contact maternity services was influenced by other people, who advised them to call.

Making contact and accessing face-to-face care was difficult for some women. Knowing who or where to call was sometimes unclear, especially ‘out of hours’:

I tried to phone the midwife, but they weren’t able to help me and they said I should phone the [NHS] and I phoned them. And they weren’t able to help me, ’cause I think it was over the weekend.

11008: positive fFN, no event, term birth

Once women contacted maternity services, some felt that they were being put off and encouraged to remain at home:

I just felt like it wasn’t normal but I felt like I was just getting told . . . I don’t know, I just kind of felt that I was getting . . . not that I was lying. I don’t know, I couldn’t really . . . I felt like, again, it was another trip to the hospital. That’s the only way I could put it, probably.

11027: negative fFN, no event, term birth

Some women felt that they needed to ‘fight their corner’ to be invited to attend for face-to-face care. However, others felt welcomed when they called maternity services.

The barrier to contacting maternity services was sometimes an internal struggle. Some women were worried about wasting clinicians’ time or being ‘dramatic’ about their experiences:

I just thought that maybe I was like being a bit . . . oh, what’s the word, not dramatic but, you know, oh, going for no reason, wasting their time I thought, do you know what I mean?

27007: positive fFN, event, preterm birth at 31 weeks’ gestation

Women with a prior childbirth experience found that negotiating access to care with clinicians over the telephone was easy because their judgement about whether or not they were in labour was believed readily.

Fetal fibronectin testing

Impact of the fetal fibronectin test result

Findings from clinician interviews

Clinicians who participated in the acceptability interviews covered a range of professional experience in different types of maternity unit (see Table 28).

Analysis of the clinician interviews revealed three main themes, and nine subthemes. The main themes were ‘factors influencing use of fFN’, ‘fFN as a tool for preterm labour diagnosis’ and ‘prognostic model to aid decision-making’.

Factors influencing use of fetal fibronectin

Use of the prognostic model in clinical practice

Ethics and registration

The study was conducted in accordance with the principles of GCP. The study was approved by the West of Scotland Research Ethics Committee (17/WS/0081). The study was registered with ISRCTN ISRCTN41598423 and the NIHR Central Portfolio Management System (36026) and the protocol was made available on the QUIDS study website in October 2018. It is now available at https://argoshare.is.ed.ac.uk/content/340/ (accessed 20 August 2020).

Population and eligibility

The eligibility criteria were identical to the QUIDS prospective cohort study (see Chapter 6): women with signs and symptoms of preterm labour at 22⁺⁰ to 34⁺⁶ weeks’ gestation for whom admission, transfer or treatment for preterm labour was being considered.

Once it was established that a participant met the eligibility criteria, preterm birth test swabs were taken. The fFN swab was taken first (from the posterior fornix), followed by the Actim Partus swab (from the endocervix), followed by the PartoSure swab (from the low vagina once the speculum was removed). All tests were carried out as per the manufacturer’s instructions.

Setting

The prospective cohort study took place in 19 consultant-led obstetric units in the UK (see Appendix 15, Table 49). All but two of these units were already participating in QUIDS and recruited women for QUIDS2 alongside QUIDS. To boost recruitment towards the end of the study we also included two additional QUIDS2-only sites: large teaching hospitals in London that already used quantitative fFN.

Participant selection and enrolment

Similar to QUIDS, women with signs and symptoms of preterm labour were identified on presentation to obstetric services. A member of clinical staff, usually the doctor or midwife assessing the woman, identified potentially eligible participants, provided a participant information leaflet and invited consent. A suitably trained member of clinical staff (doctor or midwife) or the research team was responsible for gaining consent from participants. 78

Posters and leaflets were situated in antenatal areas of participating hospitals to alert women that the study was taking place, and women were allowed as much time as possible to consider participation without unduly delaying further clinical assessment.

Screening for eligibility

Potential participants in QUIDS2 were identified after the initial assessment on presentation to maternity services and provided with information about the study at this stage. Informed consent was provided before speculum examination. Samples were collected at routine speculum examination.

Study assessments and data collection

Study assessments were similar to those of the QUIDS study (see Appendix 8, Table 43), apart from sample collection. Samples for preterm birth tests were taken using specimen collection kits as per the manufacturer’s instructions. The fFN swab was taken from the posterior fornix, as per the QUIDS study. In 17 out of 19 QUIDS2 centres the fFN sample was run at a near-bedside Rapid fFN 10Q analyser, as per the QUIDS study, which revealed a qualitative fFN result (positive/negative/invalid based on a 50 ng/ml threshold) for clinicians to base clinical decision-making on, in accordance with local protocols. In the two QUIDS2-only centres, routine clinical analysers were used and quantitative fFN results revealed to clinicians and women.

Actim Partus samples were collected using a sterile polyester swab from the cervix during a speculum examination. 17 The swab was left in the cervix for 10–15 seconds to absorb the secretions and then placed into the provided specimen extraction solution and swirled vigorously for 10 seconds. The swab was then pressed against the wall of the tube to remove any remaining liquid from the swab before being discarded. 17 Samples were stored appropriately, as per the manufacturer’s instructions,17 until tested by the research team/nominated site staff not involved in direct clinical care of the woman. For testing, the yellow dip area of the dipstick was placed into the extracted sample and held until the liquid was seen to enter the result area. 17 The dipstick was then removed and placed on a horizontal surface.

PartoSure tests were taken using the PartoSure test kit, as per the manufacturer’s instructions, using a sterile flocked vaginal swab. 18 It was recommended that the swab was taken after removal of the speculum, with the swab inserted into the vagina (no more than 5–7 cm) and withdrawn after 30 seconds. 18 The sample was then placed into the provided solvent vial and rinsed by rotating for 30 seconds before being discarded. 18 Samples were again stored appropriately until testing by the research team/nominated site staff not involved in direct clinical care of the participant. For testing, the test strip was inserted into the sample and held there until either two lines were present or 5 minutes had elapsed. 18

For both Actim Partus and PartoSure, results were reported as positive, negative or invalid at 5 minutes, with the presence of two lines (test line and control) indicating a positive result, however strong the line was. A negative result was shown by only one line (the control line) and an invalid result was recorded if no lines were present or the sample line only (i.e. no control line) was present. QUIDS2 Actim Partus and PartoSure collection kits were stored in clinical areas and test strips were kept remotely and accessed only by the research team/nominated staff who were doing the testing of the samples, to help avoid disclosure of the results.

All other data were collected in same manner as in the QUIDS study.

Sample size calculation

From the outset, we acknowledged that there was uncertainty in the recruitment rate and event rate in this add-on study to the QUIDS study, and recruitment was constrained by the duration of the QUIDS study (with QUIDS2 commencing only in the final 12 months of the QUIDS study period). At the time of commencement an observed event rate of 3.5% was seen in the QUIDS study. We aimed to recruit approximately 500 women in the prospective cohort study and have 10–25 events (preterm labour within 7 days of test). If the true diagnostic test accuracy sensitivity of a test was 95%, the 95% CI around this would be expected to be ± 10.8%. (i.e. 84.2% to 100%) (see Appendix 16, Table 50).

Statistical analysis

A statistical analysis plan was drawn up and agreed prior to database lock and analysis. Definitions used were the same as those used in the QUIDS study.