Evaluation of Health in Pregnancy grants in Scotland: a natural experiment using routine data

Article history

The research reported in this issue of the journal was funded by the PHR programme as project number 12/3070/02. The contractual start date was in March 2014. The final report began editorial review in October 2015 and was accepted for publication in April 2016. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The PHR editors and production house have tried to ensure the accuracy of the authors’ report and would like to thank the reviewers for their constructive comments on the final report document. However, they do not accept liability for damages or losses arising from material published in this report.

Declared competing interests of authors

The Social and Public Health Sciences Unit is core funded by the Medical Research Council (MRC) and the Scottish Government Chief Scientist Office (CSO). Neither funder had any input into the design, analysis, interpretation of results or conclusions drawn. Ruth Dundas declares grants from the MRC, CSO and National Institute for Health Research (NIHR) during the conduct of the study. Alastair Leyland declares grants from MRC, CSO and NIHR during the conduct of the study.

Copyright statement

© Queen’s Printer and Controller of HMSO 2017. This work was produced by Leyland et al. under the terms of a commissioning contract issued by the Secretary of State for Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK.

Background

Pregnancy and the perinatal period are critical stages for the development and improvement of population health. Deaths in infancy have considerable impact on population life expectancy, and the long-term consequences of problems arising during pregnancy, including very preterm birth, track through childhood and into adult life.

Babies of lower birthweight, whether as a result of preterm birth or of an intra-uterine growth restriction, which can lead to a neonate being small for gestational age (SGA), have a higher risk of adverse perinatal outcomes. Low birthweight is an important factor in neonatal deaths1 and poorer development. 2 Although mean birthweight has been increasing in the UK, inequalities in birthweight have increased. The UK has an incidence of low birthweight (< 2500 g) and very low birthweight (< 1500 g) in the top third compared with other western European countries. 3 The proportion of preterm births (< 37 weeks) is also high in the UK compared with other western European countries.

Low birthweight also has an impact on adult disease, being associated with a higher risk of coronary heart disease (CHD) death and diabetes mellitus. The developmental model of the origins of chronic disease proposes the causal influence of undernutrition in utero on CHD and stroke in adult life,4 and longer-term consequences have been demonstrated for many diseases. 5–14 The association between maternal nutrition and birth outcome is complex and is influenced by many biological, socioeconomic and demographic factors, which vary widely in different populations. 15,16 However, it has been reported that favourable prenatal nutrition associated with adequate prenatal care can have a positive impact on birth outcomes and morbidity in adult life. 17,18 In addition to poor maternal nutrition being associated with adverse birth outcomes, a review of maternal nutrition and birth outcomes identified improving maternal nutrition as contributing to the prevention of adverse birth outcomes in lower social class groups. 15 An improvement in fetal nutrition may, therefore, have far-reaching consequences in terms of the prevention of disease.

Low birthweight is not the only adverse perinatal outcome associated with poor infant outcomes and poorer long-term health. Across Europe, approximately 75% of neonatal deaths and 60% of all infant deaths occur among those births classified as preterm (births before 37 weeks of gestation), despite preterm births constituting only a small proportion of all births; the preterm birth rate ranges between 5.2% and 10.4% across countries, and is 7.0% in the UK. 19 A low Apgar score20 – a commonly used indicator of health in the immediate neonatal period – has been shown to be associated with increased neonatal and infant mortality21 and additional educational support needs in later life. 22

Considerable attention has been focused on the causal determinants of low birthweight in order to identify potentially modifiable factors. A substantial proportion of low birthweight is attributable to the mother’s cultural and socioeconomic circumstances, such as socioeconomic status (SES), harmful behaviours (smoking and excessive alcohol consumption) and poor nutrition during pregnancy. 14,23–25 Any social impact may vary with maternal factors such as age and smoking status. Smoking during pregnancy reduces birthweight by 162–377 g, depending on daily consumption (there is a greater reduction for heavy smokers) and the trimester in which exposure occurs (a greater reduction is seen during the third trimester). 24,26,27

A number of interventions have been developed to improve pregnancy outcomes,28 but evidence of the success of any interventions is, in most cases, still limited. An analysis of the potential to reduce preterm birth rates in high-income countries focused solely on five interventions for which there were high levels of evidence: (1) smoking cessation; (2) progesterone; (3) cervical cerclage; (4) decrease in non-medically indicated caesarean delivery and induction; and (5) a limit on multiple embryo transfer in assisted reproductive technology. 29

Although there is uncertainty over how to intervene to improve birth outcomes, many outcomes show strong patterning by measures of socioeconomic position. For example, in Scotland between 1995 and 2000, the odds of preterm birth were 56% higher in social class V (unskilled manual workers) than in social class I (professional workers; prevalence 5.7% vs. 3.7%). 30 Comparing these two groups, the odds were 107% higher for low birthweight (5.8% vs. 3.8%) and 128% higher for SGA (i.e. babies falling on or below the fifth centile of the appropriate age and sex distribution in Scotland; 5.1% vs. 3.1%). A systematic review and meta-analysis of social inequality and infant health in the UK found an odds ratio (OR) for low birthweight of 1.79 in the lowest compared with the highest social class. 31 It is, therefore, no surprise that many attempts to improve outcomes have concentrated on the most disadvantaged populations. For example, the Healthy Baby programme in Manitoba, Canada, which was designed to promote prenatal and perinatal health, comprised a targeted income supplement for low-income women and educational and supportive groups for all mothers up to the infant’s first birthday. Evaluation of this complex programme found that receipt of the prenatal benefit was associated with a reduction in incidence of both low birthweight babies and preterm births. 32

A recent systematic review of the effectiveness of antenatal care programmes for socially disadvantaged and vulnerable women noted that antenatal care was generally thought to be an effective method of improving pregnancy and birth outcomes, although the authors noted that many antenatal care practices had not been subjected to rigorous evaluation. 33 If receiving antenatal care is beneficial, then it is reasonable to think that earlier uptake of antenatal care should be better than later uptake, as this will allow more time for the beneficial effects of behaviour change to be realised.

The intervention

The intervention being evaluated is the Health in Pregnancy (HiP) grant. This was a universal, unconditional, cash transfer of £190 for women in Great Britain and Northern Ireland reaching 25 weeks of pregnancy if they had sought health advice from a doctor or midwife. The grant was introduced for women with a due date on or after 6 April 2009 and subsequently withdrawn for women reaching the 25th week of pregnancy on or after 1 January 2011. 34 The payment was not taxable and was disregarded for the purpose of means-tested benefits. Claim forms were available from the doctor or midwife; the health professional completed their section of the form, leaving the claimant to complete her part and return the form.

The HiP grant was designed to provide additional financial support in the last months of pregnancy towards a healthy lifestyle (e.g. in terms of diet), and it was suggested that the link to the requirement for pregnant women to seek health advice from a health professional may have provided an incentive for expectant mothers to seek the recommended health advice at the appropriate time.

The grant was paid by Her Majesty’s Revenue and Customs (HMRC) on receipt of a claim form partly completed by the midwife or doctor. Payment was made directly into a bank account, with a telephone helpline available to provide support through the claims process, including options for payment in the event of difficulties opening a bank account. Take-up of the grant was said to be at about the same level as for child benefit (92%; HMRC, 2011, personal communication). Advice was offered as normal by doctors and midwives. Payment was made and administration undertaken by HMRC. The cost of the grant was met by the Treasury.

Rationale for the study

The HiP grant represented an attempt to influence behaviour – appropriate and timely receipt of antenatal care and advice – by means of a relatively modest, universally applied cash transfer. The evaluation of the effect of such a payment may inform other policy aiming to change behaviour. A study evaluating payments to influence health behaviour found that financial incentives were effective in increasing infrequent behaviours such as attending clinic appointments, particularly in low-income groups, and recommended payments as being more effective than information and less restrictive than legislation. 35 In general, financial incentives have been shown to be a successful means of promoting behaviour change in general populations. 36 More recently, financial incentives have been shown to be effective in smoking cessation among pregnant women. 37 Among the general population, however, there is a suggestion of some mistrust of financial incentives as a means of promoting healthy behaviour. 38

The evaluation was restricted to Scotland based on the use of routinely collected data for the following reasons. First, data are available at a national level for the approximately 56,000 deliveries per year. Second, the coverage, completeness and quality of the data are considered to be very high. Third, the concentration of deprivation within parts of Scotland is unique within the UK. For example, 6 of the 10 most deprived electoral constituencies in the UK are in Scotland,39 and using a UK-wide Cartstairs index Scotland’s population is over-represented in the bottom five deciles compared with England. 40 If the programme was more beneficial to disadvantaged women, then the higher proportion of such women in Scotland than in the rest of the UK should make it easier to detect such a subgroup effect. Fourth, data on smoking at booking (i.e. the first antenatal appointment with a health-care professional) have been routinely recorded in Scotland for a number of years. This is not yet the case in England and Wales. Maternal discharge forms in Scotland can be linked to National Records of Scotland (NRS) civil registration data, which provide an estimate of completeness and also contribute further information such as social class.

In addition to the differences in terms of their deprivation profiles, England and Scotland differ markedly in terms of the ethnic composition of the two countries. In the 2011 census, 14% of the population of England and Wales described themselves as belonging to a non-white ethnic group41 compared with just 4% of the population of Scotland,42 and birthweight has been shown to vary markedly between the different ethnic groups within England and Wales. 43 Despite such differences, it is reasonable to believe that the results of this study will be generalisable both to the rest of the UK and internationally. If the HiP grant proved beneficial in Scotland, then there is every reason to believe that a similar impact on outcomes could be achieved elsewhere, and certainly in countries with similar health systems and comparable circumstances. Likewise, if the intervention was found to have been more effective for specific subgroups, then it might be expected that similar subgroups would show greater benefits in other settings.

The nature of the HiP grant meant that there were potential risks and harms to the population. The HiP grant was in the form of cash, which was given to the pregnant women with no constraint on what the money was used for or spent on. However, economic theory would suggest that cash transfers are more efficient than ‘vouchers’ or subsidies which try to target the expenditure in the direction of ‘appropriate expenditure’. This is because vouchers, for example, free up disposable income if they displace planned expenditure.

Socioeconomic position and inequalities

This study focuses on socioeconomic position not only through examination of the causes of ill health – behavioural and biological factors, and the extent to which the socioeconomic patterning of these leads to health inequalities – but also through the causes of the causes. 44 Income is regarded as one marker of social position that may impact on, enhance or constrain health behaviours, and this in turn may be influenced by the wider policy context. A cash transfer such as this amounts to a change in material circumstances, and to the poorest this may be a substantial and enabling change.

The use of Scottish routine data sources means that socioeconomic position and inequalities can be assessed at both the individual and the area levels. The availability of both area-based deprivation and individual social class data enable the identification of the relative importance of context and composition and any interaction between individual- and area-level inequalities. This may, in turn, lead to the targeting of any cash transfers to particular groups of areas or individuals. With the large numbers of data available, we have the power to examine the impact on different subgroups including the most deprived socioeconomic groups. Subgroup analyses are conducted for those groups seen as having the greatest potential to benefit from the payments, such as those living in the most deprived areas, those in the lowest social classes, lone mothers, primiparous women and teen mothers. These groups have been identified as having more adverse birth outcomes than the general population. 30

Marital status is an important subgroup, as single mothers have consistently been shown to have poorer birth outcomes. 45 We are particularly interested in single mothers and social class. The father’s social class is used when he is included on the birth certificate, and the mother’s social class is used if no father is present. Social class for lone mothers is therefore an amalgamation of socioeconomic position and lone parenthood.

Although the routinely collected data on ethnicity are incomplete and of dubious quality, ethnicity remains important for birthweight and other neonatal outcomes. 46 We shall, therefore, within the constraints of the data, undertake all analyses on the subgroup of women from a (non-white) minority ethnic background.

These subgroup analyses may lead to conclusions regarding whether or not the reintroduction of a cash transfer benefit universally or targeted at certain groups would help to reduce inequalities in birthweight and other adverse perinatal events.

Research objectives

The objective of this work is to evaluate the clinical effectiveness and cost-effectiveness of the HiP grants in Scotland. We do this by assessing the difference in birthweight between babies born to those mothers who were eligible for the HiP grant, and babies born either before the HiP grants were introduced (women delivering on or after 1 January 2004 and with a due date before 6 April 2009) or after it was withdrawn (women reaching the 25th week of pregnancy on or after 1 January 2011 and delivering on or before 31 December 2013). Specific questions that we address are outlined below.

• Did the HiP grant result in a change in birthweight (our primary outcome measure) or any of a number of secondary outcomes grouped as measures of maternal behaviour (gestation at booking, booking before 25 weeks and maternal smoking during this pregnancy), measures of size [very low birthweight (< 1500 g), low birthweight (< 2500 g), high birthweight (> 4000 g), crown-to-heel length and head circumference], measures of stage [gestational age at delivery, preterm (< 37 weeks), very preterm (< 32 weeks), weight for dates [standardised, SGA and large for gestational age (LGA)], and birth outcomes (elective caesarean section, emergency caesarean section, stillbirths, neonatal deaths and 5-minute Apgar score)?

• Were there differential impacts of the intervention for particular subgroups defined by socioeconomic (both area deprivation and individual occupational social class), demographic (marital status, age, ethnicity) or obstetric (parity, maternal obesity, maternal diabetes mellitus) factors?

• Were the HiP grants cost-effective? What were their total aggregate health and cost consequences? How did cost-effectiveness vary across important subgroups identified as having differential outcomes?

Outcomes and causal pathway

The primary outcome measure was birthweight. This is influenced by maternal nutrition, and one of the intentions of the HiP grant was to improve this. There was also a range of secondary outcomes, which may influence and be influenced by birthweight. Having a wide range of secondary outcomes is included in the guidelines for evaluating natural experiments. 47 The secondary outcomes for this evaluation were maternal behaviour measures (gestation at booking, booking before 25 weeks and maternal smoking during this pregnancy), measures of size [very low birthweight (< 1500 g), low birthweight (< 2500 g), high birthweight (> 4000 g), crown-to-heel length and head circumference], measures of stage [gestational age at delivery, preterm (< 37 weeks), very preterm (< 32 weeks), weight for dates (standardised, SGA and LGA)], and birth outcomes (elective caesarean section, emergency caesarean section, stillbirths, neonatal deaths and 5-minute Apgar score).

If maternal nutrition is seen as a key behaviour that could be influenced both through the antenatal advice offered and through the money provided through the HiP grant, then we would expect to see changes only in the outcomes that we were studying if there was evidence of a link between maternal nutrition and birthweight and other outcomes. To be eligible for the HiP grant, women had to seek advice before the 25th week of their pregnancy. Although there was nothing to stop women booking earlier in their pregnancy, we might expect the group most impacted by the grant to be women who would normally have booked later but were incentivised to bring forward their booking to shortly before 25 weeks. For the intervention to work for such women, we would therefore need evidence that nutrition in the third trimester could affect birthweight (as our primary outcome).

A recent review of dietary intake during pregnancy based on epidemiological studies and randomised controlled trials suggested that there was evidence that maternal nutrition during the third trimester could have an impact on birth outcomes. 48 Adequate long-chain omega-3 polyunsaturated fatty acids are crucial during the third trimester for fetal growth and the development of the nervous system,49 and a low leucocyte zinc concentration in the third trimester is associated with low birthweight. 50 Furthermore, a meta-analysis indicated that maternal haemoglobin levels in the third trimester were associated with birthweight, suggesting that iron is beneficial at this stage. 51

An additional outcome of interest was maternal smoking. Maternal smoking is recorded at booking and during pregnancy, and is known to be a modifiable risk factor with a large impact on birthweight. 52–54 We wanted to examine the effect of the health advice given when receiving the HiP grant. This may have had an impact on smoking during pregnancy rates, over and above that of the smoke-free legislation. The smoking ban in public places was introduced in Scotland in March 2006. 55

In the protocol we stated that models would be adjusted for maternal smoking. During the project, following discussion of the causal pathway with the project steering group, we refined the logic model. We decided that maternal smoking was something that could be affected by the antenatal care received as part of the HiP intervention and, therefore, that models should not be adjusted for maternal smoking. If the health advice given as part of the intervention reduced smoking, and models were then adjusted for smoking, any effect of the intervention would be undetectable.

Research design

Natural experiment using interrupted time series analysis

The HiP intervention was evaluated as a natural experiment using interrupted time series analysis. 56,57 It was not possible to use a randomised or pseudo-randomised design for a number of reasons. The HiP grant was available to all pregnant women over 25 weeks’ gestation so there was no control group. The grant was introduced in April 2009 and withdrawn in January 2011 and was universal, so there was no random allocation to the population. The interrupted time series approach allowed us to compare an intervention group that received the HiP grant both with a comparison group of pregnant women who delivered before the HiP grant was introduced and with a post-intervention group of women who delivered after the HiP grant was withdrawn.

Outcomes before the introduction of the HiP grant in Scotland and immediately after its withdrawal were compared with those during the period for which HiP existed. This interrupted time series approach allowed adjustment for seasonality and temporal trends as well as for the sociodemographic and obstetric characteristics of the mother. This adjustment for seasonality and temporal trends was important, as birthweight (our primary outcome) is known to vary by month of birth (babies born in March, April and May are heavier than those born in June, July and August) and, in addition, mean birthweight has been increasing in Scotland. 58

The Medical Research Council (MRC) has issued guidelines regarding the use of natural experiments to evaluate population health interventions when exposure to the intervention has not been manipulated by the researchers. 47 The guidance advocates a number of designs including regression discontinuity designs such as interrupted time series.

Interrupted time series can detect a discontinuity in the intercept and a discontinuity in the slope of estimates during the intervention period compared with before the intervention period. 59 The interrupted time series design can describe the effect of the intervention on three dimensions: (1) the intercept and slope; (2) the permanence of the effect; and (3) the immediacy of the effect. In terms of the HiP grant, permanence was assumed during the period in which the HiP grant was in place, such that the effect of HiP did not dilute over time, as everyone during that period was given the health advice and the £190 cash intervention. Once HiP was removed, the assumption was that any effect stopped. Sensitivity analyses were carried out on nulliparous women to determine any carryover effect on those who had never received advice or cash (see Strengthening the inference).

In order to conduct an analysis using interrupted time series, a well-defined interruption in the time series is required, and it is recommended that there are at least eight data points in the time series before the interruption and eight data points after the interruption. Other weaknesses include the inability to determine whether or not any effects on the intercept or slope were attributable to something other than the intervention which affected the outcome and which happened at the same time as the intervention; if using routinely collected data, robust evaluation may be hampered by any changes to the data collection method or coding of items, and a change in the composition of the group experiencing the intervention compared with those who do not experience the intervention may make comparison difficult.

The HiP grant was introduced for all pregnant women over 25 weeks’ gestation from a fixed date, meaning that there is a well-defined interruption in the time series of births in Scotland. Individual births were used and the pre-intervention time period ran from 1 January 2004 to 30 March 2009. Our data included month of birth, meaning that there were 63 time points (months) before the interruption; the intervention period ran from 14 April 2009 to 7 April 2011 (25 time points) and the post-intervention period ran from 23 April 2011 to 31 December 2013 (33 time points). Each of these is markedly greater than the minimum recommendation of eight time points but these figures do not reflect the full power of the study, which is achieved by using 525,400 individual observations (births; Figure 1). The routine data used here have not been subject to any changes in the data collection method or coding, other than better (more complete) recording of each item. There have been changes in the composition of mothers who gave birth during the intervention period compared with those who gave birth pre and post intervention, but maternal social and obstetric factors were adjusted for in the analyses. However, one of the potential weaknesses of the interrupted time series design, namely the inability to control for external events that happen at the same time, is much harder to overcome. Sensitivity analyses were carried out moving the HiP grant window to determine any changes in the effect of HiP on birthweight and other secondary outcomes (see Strengthening the inference).

FIGURE 1.

Flow chart showing observations received from the Information and Services Division, removal of outliers and final sample size for analyses. BMI, body mass index; IQR, interquartile range.

Study population including definitions of subgroups

The population under study was all births in Scotland between 1 January 2004 and 31 December 2013. This was split into three distinct periods: the pre-intervention period, the intervention period during which the HiP grant was in place and the post-intervention period. The grant was introduced for women with a due date on or after 6 April 2009 and subsequently withdrawn for women reaching the 25th week of pregnancy on or after 1 January 2011. 34 Therefore, the HiP grant was effectively in place for all births with a due date of delivery during the period 6 April 2009 to 15 April 2011.

In order to assign births to the correct study period, the estimated due date of delivery was calculated from the gestational age at delivery and date of birth. Gestational age at delivery was recorded in weeks, so there was a small loss of precision around the estimated due date of delivery. A washout period of 2 weeks around the start was chosen to ensure that births were assigned to the correct group. Births 1 week before the date on which the intervention was introduced and 1 week after the intervention start date were excluded. A similar washout period of 1 week before the end date and 1 week after the end date was used. The births occurring in the pre- and post-washout periods were excluded from the analyses. Therefore, the dates for the three study periods were: pre-intervention period, 1 January 2004 to 30 March 2009; intervention period, 14 April 2009 to 7 April 2011; post-intervention period, 23 April 2011 to 31 December 2013 (Table 1).

This evaluation used pre-existing routinely collected health data. There was no recruitment, assessment or follow-up of individuals to the study, as the information is already available in routinely collected data. The Scottish maternity and neonatal database, a comprehensive record linkage system, was used. 60 Probabilistic linkage procedures are used to add a unique identifier to all data sets to ensure that all records relating to an individual can be linked as required. It facilitates the linkage of a number of records from the system of Scottish Morbidity Records (SMR), including the mother’s obstetric records (SMR02) and the baby’s birth and neonatal information from Scottish Birth Records (SBR). 61 Further links to the Stillbirth and Infant Death Survey and the NRS birth, stillbirth and infant death records can be carried out. 60 All births in Scotland are required to be registered at NRS within 4 weeks, and 98% of these births are included in the Scottish maternity and neonatal database. This link to NRS civil registration data provides an estimate of completeness and contributes further information such as social class and marital status.

Socioeconomic position and inequalities were measured at both the individual and area level. The availability of both area-based deprivation and individual social class allowed for the identification of the relative importance of context and composition. Area deprivation was measured using the Scottish Index of Multiple Deprivation (SIMD) 2009. 62 The SIMD combines information across six domains: (1) income; (2) employment; (3) health; (4) education; (5) housing; and (6) geographical access. It provides a comprehensive picture of material deprivation in small areas within Scotland. The index ranks 6505 areas from the most deprived to the least deprived and measures the degree of deprivation of an area relative to that of other areas. The areas employed by the SIMD are data zones; each of the 6505 zones has a mean population of 780 people. The reason for employing small-area geography at this scale is to permit identification of relatively small pockets of deprivation. The health domain includes an indicator of the proportion of live singleton births of low birthweight. Outcomes for this project are birthweight and low birthweight; therefore, it was not appropriate to use the health domain or the composite index, which includes the health domain. The income domain was used to assess inequalities at the area level. The income domain contains six indicators relating to receipt of means-tested benefits and tax credits.

Individual socioeconomic position was measured using data from the birth registrations at NRS. NRS collects occupation information for both fathers and mothers for births registered to married couples and jointly registered by unmarried couples. For sole registered births only the mother’s occupation is recorded. National Statistics Socioeconomic Classification (NS-SEC) was used to measure individual socioeconomic position. The father’s social class is used when the father is included on the birth certificate and the mother’s social class is used if no father is present.

The HiP grant was a universal benefit. We wanted to examine the impact of HiP on subgroups, as this may lead to the targeting of any cash transfers to particular groups of areas or individuals. With the large numbers of data available, there was power to examine the impact on different subgroups, including the most deprived socioeconomic groups. Subgroup analyses were conducted for those groups seen as having the greatest potential to benefit from the payments. The subgroups that we analysed were those groups previously identified as having more adverse birth outcomes, specifically those living in the most deprived areas (SIMD = 1), mothers who had never worked, working-class mothers (three-class collapsed version of NS-SEC63), lone mothers, teen mothers (aged < 20 years), severely obese mothers [body mass index (BMI) of ≥ 35 kg/m²], and mothers who had diabetes mellitus. We carried out further analyses on all SIMD quintiles to look for evidence of a threshold effect of the HiP grant.

Marital status was an important subgroup, as single mothers have consistently been shown to have poorer birth outcomes. 45 We were particularly interested in the relationship between single mothers and social class. Social class for lone mothers is an amalgamation of socioeconomic position and lone parenthood.

An increase in birthweight, although desirable at a population level, may not be a beneficial outcome if a baby is already at risk of being LGA. Separate subgroup analyses were conducted for women seen to be at high risk of delivering a LGA baby, namely women with a BMI of ≥ 35 kg/m² and women with diabetes mellitus.

A further key subgroup for birth outcomes was the ethnicity of the mother. In Scotland the routinely collected data on ethnicity are incomplete and of dubious quality (56.5% missing over the period 2004–13). We carried out analyses of the primary and secondary outcomes on the subgroup of women from a minority ethnic background. In practice, this was all non-white mothers who had their ethnicity recorded on the birth record data. It was not possible to examine further distinctions within this non-white ethnic group.

We hoped that these subgroup analyses would inform a discussion regarding the reintroduction of a cash transfer benefit universally or its potential to be targeted at certain groups, thus reducing inequalities in birthweight and other adverse perinatal events.

Data

Numbers of observations used in analyses

The data set from ISD contained all singleton births in Scotland with a date of birth between 1 January 2004 and 31 December 2013. For this analysis, only births between 24 and 44 weeks’ gestation were included. Figure 1 details how observations were removed. The initial population size was 543,676 births (24–44 weeks’ gestation). There were exclusions attributable to impossible-to-resolve gestational age at booking and gestational age at delivery, implausible values and outliers. Births with an expected date of delivery before 1 January 2004 and after 31 December 2013, and births during the washout period around the intervention introduction and withdrawal dates were also removed. Overall, 18,276 (3.4%) cases were removed from the data set sent by ISD. The final sample available for analyses was 525,400.

For the subgroup analyses the size of the data sets are shown in Tables 2 and 3. The sample sizes are shown for the complete cases and also the sizes for the imputed data sets. There were a range of values for five subgroups and one strengthening the inference group. These were mothers with diabetes mellitus, severely obese mothers, lone mothers, mothers in working-class households, mothers in never-worked households and nulliparous mothers. This was due to missing values for these variables and, therefore, they were imputed, sometimes as being in the category concerned and sometimes not. The sample sizes range from just over 8000 in the diabetes mellitus subgroup to > 180,000 mothers in working-class households. There were over 240,000 births in the nulliparous group.

Statistical analysis

It is possible that harm may have occurred as a result of the cash transfer. The £190 was given as money to the pregnant women with no restriction on how it should be spent, and we do not know what the money was spent on. The aim was to examine how the intervention group differed from the comparison groups; birthweight could have reduced or increased. We carried out two-sided hypothesis tests to ensure that potentially harmful effects could be detected.

The main statistical design used was interrupted time series. 59 Interrupted time series can detect whether or not the intervention has an effect over and above the underlying temporal trend; it detects whether or not the effect size (slope) changes after the intervention has been introduced. The use of interrupted time series overcomes other biases such as autocorrelation of repeated measurements (measurements taken close together are related), seasonal effects (birthweight varies according to month of birth), the duration of the intervention (pre-intervention, intervention and post-intervention comparisons) and random variation of the measurement (birthweight).

Outcomes were measured on individual births, which were nested within mothers, with mothers themselves clustered within data zones. Multilevel models were used to determine whether or not the outcomes changed during the intervention period in which the HiP grants were in effect. Multilevel linear regression was used for continuous outcomes (birthweight, gestation at booking, gestational age at delivery, weight-for-dates). Multilevel logistic regression was used when the outcome was dichotomous [booking before 25 weeks, maternal smoking during pregnancy, stillbirths, neonatal deaths, very low birthweight (< 1500 g), low birthweight (< 2500 g) and high birthweight (> 4000 g), preterm (< 36 weeks), very preterm (< 32 weeks), SGA and LGA, elective caesarean section, emergency caesarean section and 5-minute Apgar score of < 7]. Two-level models with births nested within data zones were fitted for the following outcomes: very low birthweight, very preterm, stillbirth and neonatal deaths (Table 4), as the three-level model, which included a level for mothers nested within data zones, did not converge owing to the sparseness of the data (few, and often just one, births per woman).

All analyses were adjusted for temporal trends and seasonal variations in outcomes as well as for maternal age, sex of baby, gestational age at delivery, parity, mode of delivery, marital status, maternal height, maternal BMI, maternal diabetes mellitus, social class and area deprivation. Table 4 details the exact covariates included in the model for birthweight and all secondary outcomes. (Note that not all covariates were included in each model.) In the protocol, it was stated that adjustments would also be made for maternal smoking. However, this was not done owing to the intention of the intervention to have a potential effect on maternal behaviour such as maternal smoking (see Outcomes and causal pathway). In order to comply with the protocol, results are presented for both adjusting and not adjusting for maternal smoking. Temporal trends were assessed by the inclusion of time (year and month) terms in the model, and seasonal variations were assessed by the inclusion of month as a fixed effect. Departures from a linear trend were assessed using quadratic terms in addition to linear terms for maternal height, maternal BMI, gestational age and birthweight when used as a covariate and categories for maternal age. Quadratic terms were assessed in the models for each outcome and tested for significance using the complete cases. The best fit for each outcome was chosen and that model was used for all other models for that outcome. Table 4 details which models had quadratic terms included for maternal height, maternal BMI, gestational age at delivery and birthweight.

The simplest model for the intervention effect included a dummy variable ‘intervention period’ with three categories (pre intervention, intervention and post intervention), with adjustment for relevant factors (see Table 4). To assess whether or not the temporal trend was constant across the three different periods, an interaction between the intervention period and year was fitted for each outcome.

The modelling strategy was to fit a series of increasingly complex models for each outcome. The first model was a null model containing the outcome and a constant. Models with two or three levels were fitted according to whether or not the models converged (Table 4). Then, a model including the relevant covariates was fitted and, finally, a model including all relevant covariates and the interaction between period and year was fitted. If the interaction was significant, then that model was chosen; if the interaction was not significant, then the model without the interaction was chosen. This strategy was followed using the complete cases and once the final model (including interaction or not) was selected, that model was used to fit all subsequent models for that outcome, including the subgroups and moving window analyses.

In order to make the results for period interpretable when the interaction was significant, two separate models were fitted.

Economic analysis

The objective of the economic analysis was to assess the total aggregate health and cost consequences attributable to the HiP grant during its years of implementation (April 2009 to April 2011) from the perspective of the NHS. The pre-intervention years (January 2004 to March 2009) served as the comparison.

Health and cost consequences were mediated through either a birthweight or a gestational age at birth perspective, in addition to smoking during pregnancy. Birthweight was categorised as very low (< 1500 g), low (1500–2499 g) or normal (> 2500 g). Gestational age at birth was categorised as very preterm (< 33 weeks), preterm (33–36 weeks) or normal (> 36 weeks).

All costs are presented in 2015 Great British pounds using the Hospital and Community Health Services index73 to adjust, where necessary. No discounting was used.

Model overview

Figure 2 presents the conceptual analysis approach. We presented the health and cost change during the intervention period compared with during the pre-intervention period. The model is based on the incremental numbers of cases (of preterm, low birthweight or poor maternal outcomes) attributable to the HiP grant. This can be explained in five steps:

1. estimate the baseline cases per annum during the pre-intervention period, apply to an intervention duration

2. apply the adjusted effect of the intervention

3. estimate the intervention period cases

4. estimate the total incremental cases during the entire intervention period

5. estimate the cost impact.

FIGURE 2.

Conceptual analysis approach.

Strengthening the inference

As the allocation of the intervention was not randomised and the intervention and control periods occurred at different time points, pregnant women in the intervention and control groups could have been subjected to different exposures. In particular, the timing of the introduction and withdrawal of the HiP grant overlapped with other policies and interventions that may have had an impact on birthweight. Here, we consider a number of factors that could have affected birth outcomes during the intervention period that may not be attributable to the HiP grant.

Healthy Start is a means-tested voucher scheme for pregnant women and women with children under 4 years of age. 77 If the women are in receipt of certain benefits then they are eligible for free vitamins and vouchers to be spent on liquid and formula milk and fruit and vegetables. This scheme replaced the means-tested parts of the Welfare Food Scheme in the UK (including Scotland) in 2006 and is still currently in place.

We conducted sensitivity analyses to try to ascertain whether or not any observed effects could be attributed to the HiP grant as opposed to Healthy Start or other such interventions. The basis for these sensitivity analyses was the timing of the HiP intervention. This timing – of both introduction and withdrawal – was well defined and fixed, and using the interrupted time series approach enabled us to estimate any effects associated with this intervention window and ascribe those to HiP grants. The HiP intervention window was defined by two parameters, namely the start date and the duration. We repeatedly varied these parameters – effectively changing the start and end dates of the HiP intervention window – to examine the extent to which the effects that we found were dependent on the dates. If the strongest ‘intervention’ effect coincided with the dates during which the HiP grant was in place, then we could be more confident that the effect was associated with the HiP grants rather than another intervention with a slightly different timeline. For the strengthening the inference sensitivity analysis, we therefore varied both the start date and duration; five different start dates and three different durations were chosen, giving 15 different potential ‘intervention’ windows (note that one of these windows was the actual HiP window). We expected to find a dilution of any effects of the HiP grant when the intervention window was moved.

Table 8 shows the start and end dates for each of the moving window or potential ‘intervention’ periods considered. In order to mimic the main analyses, there was a 2-week washout period applied to all dates so the actual date used to calculate the end of the pre-intervention period was 1 week before the start dates shown in Table 8; the start of the intervention period was 1 week after the start dates, the end of the intervention period is 1 week before the end dates, and the start of the post-intervention period was 1 week after the end dates.

A further piece of legislation that may have affected birthweight was the introduction of the smoking ban in public places in Scotland in March 2006. This may have had an effect on the level of smoking in pregnant women and, therefore, may have affected the rate of change of birthweight of babies born after this time. We carried out a further analysis using 1 January 2007 to 1 April 2009 as the pre-intervention HiP grant period. This time period was chosen to ensure that pregnancies in the pre-intervention period were all after the smoking ban had come into effect.

The effect of the HiP grant on birthweight and other secondary outcomes might have had a carryover effect after the withdrawal of the programme. In other words, post intervention, the trend in birthweight might not return to the same rate as pre intervention. This contamination could be attributable to women who gave birth during the intervention subsequently having a birth post intervention but still heeding the health advice given during their first pregnancy. An additional analysis using only the subgroup of primiparous women was carried out to avoid such contamination.

In 2009 there was an outbreak of swine flu (influenza virus A/H1N1pdm09) in Scotland (and the UK). Pregnant women were adversely affected by this virus, resulting in poorer perinatal outcomes. 78 There were two outbreak periods, July 2009 and October–November 2009. To try to take this into account, a further analyses removing births from 1 July 2009 to 30 November 2009 in the intervention period was carried out.

Models fitted and interpretation

Changes from the protocol

Description of outcomes and covariates

Summary statistics for all outcomes by year are presented in Tables 10–13. In the main analysis, the mean birthweight for live singleton births was 3418 g. Of live singleton births, 0.63% were low birthweight, 5.65% were preterm and 4.89% were SGA babies (Table 10). Stillbirths and neonatal deaths represented 0.37% and 0.07%, respectively, of the study population (Table 12). Maternal- and family-level as well as area-level variables are described by year and over the study period in Table 14 before and after multiple imputation.

Results for the main analysis

For the main analysis, we reported the results for the primary and secondary outcomes using the imputed data, both with and without adjusting the models for gestational age at booking and maternal smoking during pregnancy. The results for the complete cases are presented in Appendices 2 and 3. We have more confidence in the results obtained from the imputed data, as it is easy to find examples for which the complete cases are not a random sample of all births. For example, in 2004 3.64% of women had a BMI at booking recorded as being ≥ 35 kg/m². With BMI missing for 54.51% of births, this meant that the observed prevalence of severe obesity was 8.00%, somewhat larger than the average imputed prevalence across the 30 imputed data sets of 7.01%. We have included the results for the complete-case analyses for completeness. Similarly, we believe that we were mistaken to suggest in our protocol that we would adjust for gestational age at booking and maternal smoking during pregnancy for the reasons given in Chapter 2, Changes from the protocol. We favour the unadjusted results, but, again, present the adjusted results for completeness as a per-protocol analysis.

Results of the subgroup analysis

For the subgroup analysis, we report the results for the primary and secondary outcomes using the imputed data, unadjusted for gestational age at booking and maternal smoking during pregnancy.

Results of sensitivity analysis

Summary of results for the main analysis and moving windows

For the primary outcome (birthweight) and five secondary outcomes (gestational age at booking; booking before 25 weeks; preterm delivery; very preterm delivery; and neonatal deaths), the main analysis and moving windows results are summarised in Figures 3–8.

FIGURE 3.

Coefficient and 95% CI for birthweight during the HiP grant period and moving windows.

FIGURE 4.

Coefficient and 95% CI for gestational age at booking during the HiP grant period and moving windows.

FIGURE 5.

Odds ratio and 95% CI for booking before 25 weeks during the HiP grant period and moving windows.

FIGURE 6.

Odds ratio and 95% CI for preterm delivery during the HiP grant period and moving windows.

FIGURE 7.

Odds ratio and 95% CI for very preterm delivery during the HiP grant period and moving windows.

FIGURE 8.

Odds ratio and 95% CI for neonatal death during the HiP grant period and moving windows.

Economic analysis results

The HiP intervention period was not associated with any meaningful differences in birthweight outcomes (very low birthweight: –83, 95% CI –223 to 80; low birthweight: 199, 95% CI –189 to 592; normal birthweight: –116, 95% CI –471 to 238), although it trended towards a reduction in births of very low birthweight. The HiP intervention period also led to negligible differences in gestational age outcomes (very preterm: 90, 95% CI –20 to 204; preterm: 28, 95% CI –303 to 355; term: –118, 95% CI –427 to 191). Finally, it also led to negligible differences in maternal outcomes. These results consequently meant that there were negligible differences in index hospitalisation costs from a birthweight approach (–£0.2M, 95% CI –£1.0M to £0.6M); a preterm approach (£0.6M, 95% CI –£0.1M to £1.4M); and a maternal outcomes approach (£0.2M, 95% CI –£0.1M to £0.5M). These figures were in contrast to a HiP grant cost of £20.4M (Table 26).

Principal findings

There is no evidence that the HiP grant affected birthweight. The analysis in which we have the most confidence, namely that relating to the analysis of 30 imputed data sets, indicates that singleton births during the eligible period (births to mothers with a due date of delivery between 14 April 2009 and 7 April 2011) had a mean birthweight that was 2.6 g lower than that of babies born during the pre-intervention period, from 1 January 2004 to 30 March 2009 (95% CI –6.9 to 1.7 g). This change in mean birthweight was trivial as well as not significant and reflected a like-for-like comparison through the inclusion of important maternal, obstetric and sociodemographic confounders, as well as the seasonality in birthweight and the secular trend of increasing birthweight.

The results of the complete-case analysis suggest that the HiP grant was associated with a significant decrease in birthweight of 9.0 g (95% CI 3.7 to 14.3 g). The difference between these two results reflects the fact that the complete cases represent a biased sample of the 532,968 singleton births that were available for analysis.

In retrospect, the fact that we found no effect of the HiP grant on birthweight is perhaps not surprising, given that the known modifiable determinants of low birthweight are largely limited to maternal smoking and some specific aspects of nutrition. 14,23–25,48 To this extent, the choice of birthweight as the primary outcome measure of this study was somewhat optimistic.

Our findings for mean birthweight were supported by other measures of size and weight-for-dates. There was no significant effect on the odds of low birthweight, very low birthweight or high birthweight, suggesting that no part of the birthweight distribution was affected. Similarly, there was no significant effect on the continuous measure of weight-for-dates (z-score) or on the odds of being either SGA or LGA.

There was evidence of a significant effect on head circumference (the mean head circumference of babies associated with the intervention was 0.04 cm smaller than for babies pre intervention; 95% CI 0.03 to 0.05 cm) and crown-to-heel length (the mean length of babies in the intervention was 0.07 cm longer than babies pre intervention, 95% CI 0.04 to 0.10 cm). However, these changes of < 1 mm are too small to be of any clinical relevance and reflect the power available to the study through the use of a large routine data set.

We do see a significant and positive impact on the date of booking. Mean gestational age at booking during the intervention period was 0.35 weeks lower than pre intervention (95% CI 0.29 to 0.41 weeks). This is a small effect, equivalent to a population shift of about 2.5 days, which in itself might be considered unimportant. More important, however, is the increase in the odds of booking before 25 weeks; the HiP grant was associated with a 10% increase in the odds of booking before 25 weeks (OR 1.10, 95% CI 1.02 to 1.18), suggesting that very late bookers were likely to seek help earlier. This effect can be seen in Table 13; the crude proportion failing to book before 25 weeks [3.5% over the 10-year study period based on the imputed data (96.5% booked before 25 weeks)] decreased from 3.9% in 2006–8 to 2.8% in 2009–11. A 1.1% absolute increase in the proportion of women seeking antenatal care earlier in their pregnancy represents about 600 women each year. Women who book late in their pregnancy tend to be more disadvantaged than those who book earlier; across the 10-year study period, those booking at 25 weeks or later were more likely to be from a household in which the head of the household had never worked (24% compared with 15% booking before 25 weeks), to be a lone parent (12% vs. 5%) and to have been under the age of 20 years at the time of birth (12% vs. 6%).

The only measure of maternal health behaviour – smoking during pregnancy – was unchanged by the intervention. It is worth noting, however, that the odds of smoking during pregnancy showed strong declines during the study period, with a reduction of 8% per year between 2004 and 2011. Crude (imputed) rates of maternal smoking fell from 25.8% in 2004 to 18.8% in 2013.

The intervention was not associated with a significant effect on elective caesarean sections but it was associated with an increase in emergency caesarean section (OR 1.07, 95% CI 1.03 to 1.10). On its own, it is unclear if an increase in the emergency caesarean section rate is a good or a bad thing. Even if there is no adverse health effect for mother or infant, surgical intervention represents an opportunity cost. However, emergency as opposed to elective caesarean section should indicate a perceived need to intervene for the safety of the mother or infant, and so an increase in this rate may be indicative of something going wrong.

There were adverse outcomes associated with the intervention. Gestational age at delivery showed a small reduction of 0.04 weeks (about 7 hours; 95% CI 0.02 to 0.06 weeks). This small decrease in mean gestational age is accompanied by no significant effect on preterm births and a small (but again not significant) increase in the odds of very preterm births (OR 1.11, 95% CI 0.98 to 1.25). This increase in very preterm deliveries is likely to be associated with the increase in the emergency caesarean section rate noted above (see Table 5).

The principal adverse finding is a large increase in the odds of neonatal death associated with the intervention period (OR 1.84, 95% CI 1.22 to 2.78). The crude neonatal mortality rate, averaging 0.07% between 2004 and 2013, was higher between 2009 and 2011 (0.08–0.09%) than in the 2 years before (0.06–0.08%) or after (0.03–0.06%). This increase during the intervention period should be considered against a background of falling neonatal mortality, with the odds of neonatal death falling by 9% per year (95% CI 0% to 17%). There was no significant impact on either the stillbirth rate or the 5-minute Apgar score.

With such limited impact on outcomes, the economic analysis showed that the HiP grant had very little economic effect (positive or negative).

Impact on inequalities

Extensive subgroup analyses were undertaken to enable us to determine whether effects were stronger, or indeed whether they just differed, for the most disadvantaged groups. These could potentially be indications of effects on those with most to gain from the intervention, which are not observed when the whole population is analysed (and with adjustment for measures of disadvantage).

There were no significant effects of the HiP grant on the subgroup of women with diabetes mellitus. For all other subgroups, gestational age at booking was significantly reduced during the intervention period; notably, the effect was more pronounced (i.e. the decrease in gestational age at booking associated with the intervention period was larger) for those women with a BMI at booking of ≥ 35 kg/m², women living in the most deprived 20% of areas, lone mothers, women living in low social class households and teen mothers. Conversely, the effect was less pronounced for women living in the more advantaged areas. Of these subgroups, women with a BMI at booking of ≥ 35 kg/m² and women living in low social class households had increased odds of booking before 25 weeks. These odds were significantly increased relative to the pre-intervention period, but were also larger than those found for all women.

Women living in the most deprived 40% of areas and women living in households in which the head of household had never worked showed significant increases in the odds of emergency caesarean section. However, in each case these odds were lower than those seen for the general population.

For women living in the most deprived 20% of areas, the intervention period was associated with a decrease in mean offspring birthweight of 12 g (95% CI 4 to 21 g). Although this difference itself is probably too small to be important, there was a significant decrease in the odds of high birthweight babies (OR 0.93, 95% CI 0.86 to 1.00) and no significant change in the odds of low birthweight or very low birthweight babies.

Among the subgroup of mothers for whom ethnicity was recorded as a group other than white, the intervention period was associated with a decrease in mean gestational age. This group also saw a significant increase in the odds of having a baby that was small for its gestational age (OR 1.30, 95% CI 1.01 to 1.67).

Interpretation

To understand the findings presented here, it is necessary to consider the wider context of the period under study. In particular, we looked for the influence of factors or events that may have impacted on births in Scotland at some stage, whether these be Scotland-wide, UK-wide or more global in nature. Above (see Robustness and generalisability), we discussed the introduction of a HEAT target, which was designed to increase the proportion of women from the most disadvantaged areas who booked before the 12th week of pregnancy and concluded that, although this may have contributed to the decrease in the mean gestational age at booking, it was unlikely to have caused the increase in the proportion of women booking before the 25th week during the intervention period. This was both because of the relationship of the 25-week target to the HiP grant, and because the proportion of women booking before 25 weeks fell significantly following withdrawal of the grant.

We undertook a sensitivity analysis relating to the introduction of the smoking ban in public places in Scotland that happened during the pre-intervention period, which was known to be associated with positive outcomes and behaviour change. However, the results of this sensitivity analysis were unclear, given that the exclusion of the period prior to the introduction of the smoking ban may have resulted in an atypical pre-intervention period during which there were rapid changes in some outcomes. This led to our finding of a significant increase in preterm births compared with the short-term (post-smoking-ban) trend, but no significant increase compared with the longer-term trend.

Two other global events are worth some consideration. The first of these is the influenza A virus A/H1N1pdm09 of swine origin (swine flu), which was declared to be a pandemic in June 2009. This virus was known to affect pregnant women and perinatal outcomes adversely, increasing the odds of preterm delivery and perinatal mortality in the UK. 78,81 The outbreak peaked during the intervention period in July 2009, and again from October to November 2009. It is difficult to know in exactly which stage of pregnancy exposure to the virus had the greatest impact on outcomes. We excluded 5 months of births from July to November 2009, inclusive, as a form of sensitivity analysis to ascertain whether or not excluding the peak months of the pandemic changed any results. Excluding these months left most results unchanged, but did lead to a significant increase in the odds of both preterm (OR 1.07, 95% CI 1.01 to 1.13) and very preterm birth (OR 1.19, 95% CI 1.04 to 1.34). The increased odds of neonatal death remained significant (OR 1.97, 95% CI 1.27 to 3.06). The fact that these effects were larger than those recorded in the main analysis indicates that the months excluded were lower risk than the months included. It may be, therefore, that the months excluded for this sensitivity analysis were not just conservative (in the sense that the peak months of the pandemic only were excluded) but also that the month of delivery was not necessarily the most important factor affecting these outcomes.

Finally, the global financial crisis may have impacted on these results in ways that are difficult to predict. Often considered to have begun with the collapse of the Lehman Brothers bank in September 2008, the crisis resulted in recessions in many countries, including the UK, and the subsequent imposition of austerity measures. There are many plausible mechanisms by which this crisis might have influenced perinatal outcomes. One such route might be through increasing maternal malnutrition caused by economic hardship; certainly, use of food banks has increased in the UK. 82 Alternatively, it may be that increasing job insecurity, associated with high levels of unemployment or precarious employment, has led to high levels of anxiety and psychosocial stress, and thus that these have influenced outcomes. It is not clear how or when any such effects would manifest themselves; although the poorest groups could be hardest hit, it is also plausible that there would be some protection for the worse off (such as the long-term unemployed) through the welfare state, whereas those with jobs experience falling salaries and the stress of job insecurity. Therefore, our sensitivity analyses focused not only on the most disadvantaged groups but also on groups from across the spectrum of social advantage and disadvantage, and considered the population as broken down into quintiles of deprivation. The results of these analyses were not, however, able to determine any effect that may be attributable to the global financial crisis.

Strengths and weaknesses

A strength of the study was the very high quality, completeness and coverage of the linked routine maternity hospital discharges and SBR. 83 Linked data were available for about 98% of births in Scotland,84 with knowledge of all registered births. Such high-quality data are not currently available at a national level for other countries in the UK, which means that it was not possible to extend this evaluation of HiP grants beyond Scotland. To a large extent, the data were complete, with few missing data in most of the key variables considered. This completeness was not universal and data on some variables, such as ethnicity and maternal smoking, were poorly collected for some or all of the study period.

The methods used were appropriate for the analysis of these data. Individual data were used yielding high power for the analyses and enabling data hierarchies of births nested within mothers and of mothers nested within small areas to be taken into account. Multiple imputation was used to account for the fact that the data were not complete. The results of the analysis of the multiply imputed data at times differed from the complete-case analysis, thus suggesting that the data were not missing completely at random but that missing values were patterned in ways that were associated with other (observed) variables. A large number (30) of imputed data sets meant that the variability within the data was preserved through the imputation process.

The study was based on routinely collected data, which meant that there was no information regarding how the money was spent. The HiP grant was an unconditional cash transfer – there was no restriction placed on how the money could be spent – and there is no way of knowing if it was spent in ways that might improve, damage, or have no impact on the health of the mother and fetus.

The study design remains a major limitation. This was essentially an uncontrolled before-and-after study (although unusually there was a period after the withdrawal of the intervention as well as before its introduction to give comparison groups). The interference of changes in the context that occurred during the study period – including, but not necessarily limited to, the introduction of the ban on smoking in public places, the introduction of a HEAT target to increase the proportion of women from the most deprived areas who sought early antenatal advice, the swine flu pandemic and the global financial crisis – may have influenced a number of the outcomes for all women or selected subgroups in ways that are difficult to predict with any certainty. This means that results from this study should be seen as indicative rather than definitive; in the study protocol we acknowledged that this study design meant that it might be difficult to attribute any effects with certainty to the HiP grant and suggested that, in such a case, a randomised controlled trial might be necessary. 85 It would have been possible to have conceived a randomised design for the original grant when it was introduced. Failing this, another design facilitating evaluation such as a stepped wedge design, as recommended by the House of Commons Health Committee in 2009,86 would have satisfied any political requirement to introduce an (untested) intervention across the country. It remains the case that policy interventions are still rarely introduced in such a way as to facilitate rigorous evaluation and we would urge policy-makers to consider the importance of the evaluation of policy and to work with researchers to utilise designs that would facilitate such evaluation, when considering introducing or changing such public health programmes in the future.

Patient and public involvement

This project had no direct patient involvement, as the analysis used anonymised patient records. There would be benefits to the participation of pregnant women in a qualitative component of a future study as a means of understanding whether and how the cash incentive led to behaviour changes including earlier contact with health services. Such research could also explore how the money was spent and whether or not it contributed to a healthier lifestyle. In a retrospective study such as this, evaluating an intervention that has already been withdrawn, the difficulty in and potential bias of recalling such details made this element impractical.

There was little room for the involvement of the public in designing this research project. The intervention had already been designed, delivered and withdrawn. The methodology for the evaluation was determined by the intervention design and best practice in statistics, economics and the evaluation of natural experiments. The project would have benefited from public involvement on the project steering group, particularly from groups focusing on pregnancy and on poverty. Such involvement would have ensured that we were able to allay any fears over the use of confidential data in our analyses and that we reported our results with sufficient sensitivity.

Implications for health care

We found no effect of the HiP grant on birthweight. Given the difficulty that has been found in changing birthweight, this is probably not surprising, especially given the relatively small sum of money that the grant amounted to.

We found an increase in the odds of booking before 25 weeks to be associated with the HiP grant. We believe that it is likely that this is a direct effect of the £190 cash transfer acting as an incentive; seeking antenatal care before the 25th week was a condition of receiving the money and, following withdrawal of the grant, the odds of booking before 25 weeks fell significantly. This is in line with other examples of health behaviour being influenced by cash incentives,36 and suggests that it is possible to ‘nudge’ behaviour, including health-care-seeking behaviour. The care provided was normal care, and so the only implication for health service provision was an increased uptake of services. The group of women targeted by such an intervention – those who do not normally present to antenatal services until the 25th week of pregnancy or later – tend to be more disadvantaged than those who present earlier in the pregnancy, with higher proportions of worklessness, lone parents and teen mothers.

However, we did find significant increases in the likelihood of one adverse outcome, namely neonatal mortality. It does not appear likely that this was associated with the intervention for two reasons. First, there is no clear mechanism by which the intervention could have brought about this adverse result. If it were the case that the £190 cash transfer had been spent in ways that led to higher neonatal mortality, then we would expect the effect to be more pronounced in certain subgroups, namely for poorer women for whom £190 might have represented a larger increase in available cash. This does not appear to have been the case for women living in the most deprived areas or for women living in households in which the head of household was a semiskilled or unskilled manual worker, for example. Second, if the payment of £190 associated with the introduction of the HiP grant caused an increase in neonatal mortality then its removal should have brought about an equivalent decrease. The small decrease we found on removal of the grant was far from significant and left the neonatal mortality rate higher than that seen prior to the introduction of HiP. We feel that the impact of extraneous factors, such as the swine flu pandemic and the global financial crises, are more likely to have been the causes of the increase in neonatal mortality. Unfortunately, however, it is not possible to isolate and thereby distinguish the effects of these global phenomena.

It is unclear whether the increase in emergency caesarean sections that we found was an adverse effect (i.e. an unnecessary and expensive surgical intervention) or whether such emergency interventions brought about improvements in maternal and infant health. Given that we found a significant increase in the rate of emergency caesarean sections in the post-intervention period, not only in relation to the pre-intervention period but also in relation to the period during which the HiP grants were in place, it seems unlikely that the HiP grant could have caused such an increase. It is plausible that this outcome and other related outcomes (such as the odds of preterm or very preterm delivery, which were elevated but not significantly so) were also related to the global context including the swine flu pandemic and the global financial crisis.

The outcomes that were used for the analyses contained in this report were restricted to those that are routinely collected and do not include, for example, any measures of maternal wellbeing or mental health. It is possible that such outcomes could have been affected by the cash transfer, and it is important to note that the absence of evidence of an effect is not the same as evidence of the absence of an effect. It is possible that booking shortly before the deadline – say in the 23rd or 24th week of pregnancy – does not leave much time for the effect of any behavioural changes to be realised. It may be that earlier booking would lead to an increased dose of antenatal care, and that either reinforcement of health messages over a longer antenatal period would lead to behaviour change, or earlier intervention would lead to behaviour change earlier in the pregnancy. Either consequence of earlier booking might improve infant outcomes.

Recommendations for research

It seems likely that the HiP grant was responsible for an increase in the proportion of women booking before 25 weeks, which represents a step in the right direction, but the big question is why this did not translate into an improvement in any of the other outcomes evaluated. We have suggested above (see Implications for health care) that this may in part reflect the threshold of 25 weeks being too late in the pregnancy to modify behaviour in any way that would lead to notable differences in birthweight or other outcomes indicating improved infant health. Our first recommendation for further research is, therefore, to test whether a threshold earlier than 25 weeks would lead to increased exposure to antenatal care and advice (increased frequency of antenatal visits), and whether any accompanying change in health behaviour (such as smoking or diet) occurring earlier in the pregnancy would lead to improvements in maternal and infant health outcomes. We recommend evaluation of the economic benefit that can be achieved through such a modified threshold and any consequent improvement in results.

It should be noted that the disadvantaged nature of the approximately 4% of women who do not book before 25 weeks offers an opportunity and a challenge. If the health-seeking behaviour of this group can be modified then this offers an opportunity to provide services to a group with great potential to benefit and, as a result, potentially to reduce inequalities in pregnancy outcomes. However, linking financial incentives to booking targets may result in an effective penalty for those women who are in greatest need of the money but fail to seek professional advice before the deadline, and if the incentive did not lead to behaviour change in this group then this could lead to an increase in inequalities.

Our other recommendations concern the nature of the intervention, and, although these questions are pertinent to such an intervention to improve pregnancy outcomes, they are relevant to many interventions in different fields.

Our second recommendation is to evaluate whether or not this intervention has greater impact, and higher cost-effectiveness, when delivered as a targeted or as a universal intervention. The HiP grant was universal; all women could claim the same cash transfer regardless of personal circumstances. This approach has many benefits, including an easier application process (no need for the complicated form filling that would be required for a means-tested grant) and a lack of stigma or perceived stigma attached to the grant, both of which should lead to higher uptake. There is clearly an opportunity cost; by targeting the payment to those in the greatest need, the payment could be increased or the money saved could be used for other purposes. An increased payment may not lead directly to an improvement in outcomes, but may provide an added incentive to seek advice before the deadline. Targeting the payments also comes with an attached cost, as there is a need to assess the eligibility of claimants.

Our third recommendation for future research is to test whether or not outcomes differ depending on whether the payment is conditional or unconditional. The cash transfer in the HiP grant was unconditional and could be spent as the recipient wished. This means that women were, in theory, able to spend the money received on cigarettes, alcohol or other products that could damage the fetus. An alternative approach would be to issue vouchers rather than cash with restrictions on what can be bought with those vouchers (as happens, e.g., with the Healthy Start voucher scheme77). In fact, the two approaches are not as distinct as they might at first appear, because spending a conditional payment on goods that would have been bought anyway will free up cash to be spent as the recipient chooses. Their success depends in part on the retailer enforcing any conditions that are attached, and restricting the retailers that can be used or the goods that can be purchased may prove a disincentive, or may create inequalities between communities with differential access to participating retailers. However, the success of conditional payments will also depend on the acceptability of the restriction of goods to those women included in the scheme.

Our fourth and final recommendation is to investigate the relationship between the size of the payment and subsequent outcomes. The cash payment served two purposes: first, as an incentive to earlier health-care booking in pregnancy, and second, as additional financial support. Varying the size of the payment may not only indicate the optimal payment in terms of cost-effectiveness but may also, in combination with changes to the threshold in the first recommendation above, lead to a better understanding of the separate roles of antenatal care and financial support. For a given booking date a dose–response relationship between the size of the payment and outcomes would emphasise the value of the financial support, whereas improved outcomes associated with earlier antenatal booking at the same level of financial support would highlight the importance of the antenatal care itself.

Research answering these questions would determine whether or not a modification of the HiP grant should be considered as an option for policy-makers looking to improve pregnancy outcomes and to address inequalities in health from the beginning of the life course.

Project steering group

Our sincere thanks to the project steering group:

Dr Peter Craig, MRC/CSO Social and Public Health Sciences Unit, University of Glasgow (chairperson).

Dr Mary Hepburn, Consultant Obstetrician and Gynaecologist (retired).

Dr Graham McKenzie, Consultant in Public Health, NHS Lothian.

Mrs Marion McNab, Lead Midwife, NHS Greater Glasgow and Clyde.

Reviewers

We would like to thank the four anonymous reviewers for their thoughtful and helpful comments which have improved our final report.

Contributions of authors

Alastair H Leyland (Associate Director SPHSU, Population Health Statistics) oversaw the project and contributed to all aspects of the design, analysis and interpretation of the results.

Samiratou Ouédraogo (Research Fellow, Epidemiology) led the statistical analysis and contributed to the interpretation of the results.

Julian Nam (Research Fellow, Health Economics) led the economic analysis and contributed to the interpretation of the results.

Lyndal Bond (Principal Research Officer, Social Interventions) contributed to the study design and the interpretation of the results.

Andrew H Briggs (Professor, Health Economics) oversaw the economic analysis and contributed to the study design and the interpretation of the results.

Ron Gray (Senior Clinical Research Fellow, Perinatal Epidemiology) contributed to the study design and the interpretation of the results.

Rachael Wood (Consultant in Public Health Medicine, Public Health) contributed to the interpretation of the results.

Ruth Dundas (Senior Investigator Scientist, Statistics) managed the project, oversaw the statistical analysis and contributed to the study design and the interpretation of the results.

Publications

Dundas R, Ouédraogo S, Bond L, Briggs AH, Chalmers J, Gray R, et al. Evaluation of Health in Pregnancy Grants in Scotland: a protocol for a natural experiment. BMJ Open 2014;4:e006547.

Data sharing statement

The data came from the Scottish maternity and neonatal database held by the ISD at the NHS National Services Scotland. The release of the data for research purposes was subject to agreement (6 May 2014; PAC38A/13) from the Privacy Advisory Committee at NHS National Services Scotland and the Registrar General (www.nhsnss.org/pages/corporate/privacy_advisory_committee.php). Access to the individual data was restricted to the investigators as part of this agreement.

Disclaimers

This report presents independent research funded by the National Institute for Health Research (NIHR). The views and opinions expressed by authors in this publication are those of the authors and do not necessarily reflect those of the NHS, the NIHR, NETSCC, the PHR programme or the Department of Health. If there are verbatim quotations included in this publication the views and opinions expressed by the interviewees are those of the interviewees and do not necessarily reflect those of the authors, those of the NHS, the NIHR, NETSCC, the PHR programme or the Department of Health.