Pathways to specialist community perinatal mental health services: a two-site longitudinal retrospective service evaluation

Article history

The contractual start date for this research was in May 2019. This article began editorial review in October 2023 and was accepted for publication in June 2024. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The Health and Social Care Delivery Research editors and publisher have tried to ensure the accuracy of the authors’ article and would like to thank the reviewers for their constructive comments on the draft document. However, they do not accept liability for damages or losses arising from material published in this article.

Copyright statement

Copyright © 2024 Jovanović et al. This work was produced by Jovanović et al. under the terms of a commissioning contract issued by the Secretary of State for Health and Social Care. This is an Open Access publication distributed under the terms of the Creative Commons Attribution CC BY 4.0 licence, which permits unrestricted use, distribution, reproduction and adaption in any medium and for any purpose provided that it is properly attributed. See: https://creativecommons.org/licenses/by/4.0/. For attribution the title, original author(s), the publication source – NIHR Journals Library, and the DOI of the publication must be cited.

2024 Jovanović et al.

Study design

This is a two-site, longitudinal retrospective service evaluation study. It is reported in line with the Strengthening the Reporting of Observational Studies in Epidemiology checklist for observational studies.

Data collection

Data were collected using the adapted WHO encounter form. 13 The original questionnaire is a standardised schedule for gathering basic sociodemographic, clinical and pathways data for each participant. The questionnaire was adapted to perinatal services by the research team, piloted by an independent researcher and refined based on feedback received from a panel of researchers based at the Queen Mary University of London, and a Lived Experience Advisory Panel assembled for the project. The adaptation ensured that specific aspects of perinatal care pathways were captured, for example ‘encountered services’ included midwifery teams, obstetric services, children’s social services, etc. (see Report Supplementary Material 1). The Lived Experience Advisory Panel met regularly throughout the project and advised on all aspects of the study, including interpretation of findings and dissemination.

Settings and patient eligibility

The study was conducted in community PMHS in Birmingham (Birmingham and Solihull Mental Health NHS Foundation Trust) and London (East London NHS Foundation Trust) during a 6-month period (July–December 2019).

Eligible patients were newly referred to services at least once during this time period. Patients who had received treatment from PMHS in the last 2 years prior to this were excluded because their past experience could have impacted the pathway they took and so biased the results. As mentioned, these services are commissioned to care for women presenting with moderate to severe mental illness. The referrals that did not meet the PMHS threshold for moderate to severe mental illness were declined and these were not included in the study. Only accepted referrals were analysed. Declined referrals were often redirected for support within primary care and third-sector organisations. The list of patients was obtained from information technology services in participating mental health trusts. Clinicians accessed patient electronic records to extract data on patients’ sociodemographics, clinical characteristics, the involvement of children’s social care and pathways to PMHS.

Studied variables

The adapted WHO encounter form gathered information on each clinician/service encountered on the care pathway; the duration of the patient’s journey to the service; the source of referral; reason for referral; and treatment offered. Based on the collected data, the following variables were created: help seeking initiated by patient or by others, the referrer to PMHS [primary care (e.g. family doctor)/secondary care (e.g. specialist medical services)], main reason for referral to PMHS (current deterioration of mental health/review of pre-existing condition); number of clinicians encountered on pathway to PMHS; type of pathway (‘simple’ – contact with one clinician/services before contact with PMHS or ‘complex’ – contacts with two or more clinicians/services before PMHS); treatment offered (monitoring and support; medication; counselling; combined treatment); outcome of first perinatal appointment (accepted under PMHS/discharged); and time (in weeks) between the start of perinatal mental health problems (self-reported or diagnosed) and the first appointment with PMHS. If a patient saw the same clinician or service twice or more in a month, particularly if this was due to rescheduling appointments, it was noted as one contact.

Data analysis

Descriptive statistics were used to report study variables. Mean (M), standard deviation (SD), median (Mdn), range and frequencies were used as appropriate. Ethnicity was initially collected for 18 categories taken from the British census and classified into five groups (White, Mixed, Asian, Black and other), as this facilitates comparison with public sector documents that also recommend the approach. 15,16 Due to small number of patients in ethnic groups ‘Mixed’ (n = 3) and ‘other’ (n = 4), data were reported (see Appendix 1) but they were excluded from quantitative analyses comparing the groups and these analyses were performed on three groups only (Asian, Black and White).

The normality of the distribution of study variables was tested using Shapiro–Wilk test. Comparisons were done using chi-square, independent sample t-tests, Mann–Whitney and Kruskal–Wallis tests as appropriate (see the notes in Tables 1–3). In cases where using a chi-squared test was ill-advised due to small cell counts, Fisher’s exact test for count data was used and compared with Monte Carlo simulated (100,000 replications) chi-squared test which allowed for the calculation of Bonferroni-corrected post hoc tests. Where Kruskal–Wallis test suggested significant differences in rank across groups, post hoc pairwise Wilcox tests with Benjamini and Hochberg (1995) adjustment to control for false discovery rate were used to compare ethnic groups. 18

For testing the association of various predictors with simple care pathway to PMHS, a binary logistic regression was performed. Two sets of predictors were selected based on clinical relevance and results from univariate analyses (i.e. ‘Patient-related variables’ and ‘Pathway-related variables’). They were entered in the regression model at the same time (Table 4). A 5% alpha error was used as the limit of statistical significance for each predictor.

Linear regression models testing the association of various independent variables and the amount of time between symptoms developing and access to PMHS were performed as shown in Table 5. We first established that there was no significant multicollinearity between the predictors [all variance inflation factors (VIFs) were between 1.02 and 1.46], and then entered all the predictors into the model at the same time.

Data analysis was carried out using SPSS [(version 22.0), IBM Corp (2013). IBM SPSS Statistics for Macintosh, Version 22.0. Armonk, NY: IBM Corp) and R (The R Foundation for Statistical Computing, Vienna, Austria, R Core Team R, R: A language and environment for statistical computing, 2013) using base [(version 4.2.2), stats (version 4.2.2), R Core Team (2022). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria], psych [(version 2.2.9), William Revelle (2022). psych: Procedures for Psychological, Psychometric, and Personality Research. Northwestern University, Evanston, IL, R package version 2.2.9, https://CRAN.R-project.org/package=psych Revelle: not a software but an author of psych, DescTools [(version 0.99.47), Signorell A (2022). DescTools: Tools for descriptive statistics. R package version 0.99. 47. https://github.com/AndriSignorell/DescTools/. CRAN. 2022], rcompanion [(version 2.4.18), Mangiafico Salvatore S (2022). rcompanion: Functions to support extension education program evaluation. version 2.4.18 Rutgers Cooperative Extension. New Brunswick, NJ. https://CRAN.R-project.org/package=rcompanion] and chisq.posthoc.test (version 0.1.2, Ebbert D (2019). Chisq. posthoc. test: a post hoc analysis for Pearson’s chi-squared test for count data. R package version 0.1.2) packages.

Participating services and established referral pathways

Please refer to Table 1 for the key characteristics of the services included in the study. Both services included in the study cater to ethnically diverse communities in inner-city areas. The Birmingham service operates at a higher capacity, which reflects the larger number of live births in its catchment area. In 2019, the number of live births in the studied Birmingham catchment area (n = 8000) was nearly double that of the London one (n = 4331). The comparison of the ethnic composition of study participants with the maternity populations in Birmingham and London showed no difference (ps > 0.05; see Table 1). This indicates that, in terms of ethnicity, patients under the care of PMHS in the present study were similar to those in the general maternity population.

In both services, established referral pathways included primary care services (e.g. general practitioners, community midwifes, health visitors and counsellors) and secondary care services (e.g. community mental health teams, inpatient psychiatric wards, crisis resolution and home treatment teams, specialist midwives, obstetricians and emergency services). The crisis resolution and home treatment teams offer assessment and treatment for individuals experiencing a mental health crisis, as an alternative to hospital admission. It is worth noting that specialist midwives are part of primary care pathway for mental health and part of secondary care for maternity services. At the time of the study, self-referrals were not possible. Both services did not accept direct referrals from social care; instead, clinicians advised that the patient is first seen within healthcare services and then referred to PMHS.

Study sample

The study sample is described in Table 2. The study analysed records of 228 patients treated in Birmingham (n = 131, 58%) and London (n = 97, 42%). The majority of patients were pregnant (n = 166, 73%), while others were in the postnatal period (n = 54, 23%) or referred for preconception review (n = 8, 4%). Study participants in Birmingham, compared to London, were statistically significantly younger (p = 0.003), more women were pregnant (p = 0.008), and from Asian ethnicities (p = 0.007). There were few White British women in the Birmingham sample (p < 0.001) and overall Asian women were more likely to be unemployed (p < 0.001).

Pathways to community perinatal mental health services

Detailed information about pathways to PMHS can be found in Table 3. The majority of patients (n = 146, 64%) initiated seeking help for their perinatal mental health problems. The initial contacts were largely within primary care (n = 157, 69%). In more than half of participants (n = 150, 65.8%), the first clinician/service that was contacted made a referral to PMHS. Approximately half of participants (n = 121, 53%) were referred to PMHS by primary care clinicians. The vast majority of participants (n = 195, 86%) were referred because their mental health deteriorated and required intervention from mental health services. Others were referred for reviewing and monitoring of a pre-existing condition (n = 33, 14%). In Birmingham, referrals to PMHS came more often from secondary care (p = 0.001). In London, review and monitoring of pre-existing conditions was a more common reason for referral (p = 0.002). After the first PMHS assessment, patients were offered monitoring and support (56%), medication (22%), counselling (13.6%) and combined treatment (8.3%). Seeing a larger number of carers on the pathway to PMHS was associated with a longer period between the first referral and the first perinatal appointment (r = 0.51, p < 0.001).

Differences in pathways between ethnic groups

The included variables and how they varied across the five ethnic groups can be seen in Tables 2 and 3. The study results revealed that several parameters varied among ethnic groups.

Compared to Asian and Black patients, White patients initiated help seeking more often (p = 0.040) and they were more likely to have complex care pathways (p = 0.046).

Simple and complex pathways to perinatal mental health services

The total number of clinicians/services encountered on the pathway to PMHS ranged from 2 to 5 (Mdn 2). The majority of participants in the study went through so-called ‘simple pathways’ (n = 144, 63.2%). The number of complex care pathways cases was higher in Birmingham than in London (p < 0.001) (see Table 3).

Looking into the factors influencing the likelihood of having a simple care pathway to PHMS, the odds for such pathway were higher for patients in London, while other predictors were not significant when controlling for the shared variance (see Table 4). Regarding ethnicity, there was no evidence of difference when the model controlled for other potentially confounding variables (i.e. patient- and pathway-related variables). Still, the overall model was better at predicting the type of pathway compared to model with no predictors.

The complex pathways are presented in detail in Appendix 2. In Birmingham, the most frequent complex pathway, present in 29% of patients, was to access the PMHS through both community midwife and a specialist midwife. In London, the most frequent complex pathway included crisis services such as accident and emergency (A&E) and crisis resolution and home treatment teams (36%). Six patients encountered five carers on their journey, five in Birmingham and one in London. Five of these women were pregnant and experiencing current deterioration in mental health. One woman was in the postnatal period, also deteriorating. Two of these women were suicidal and they utilised the crisis pathway.

Duration of patient journey to perinatal mental health services

The study also explored duration of time in weeks between the self-reported or diagnosed start of perinatal mental health problems and first appointment with PMHS. Large variation was found and the range was from < 1 week to 276 weeks; the median was 20.4 weeks or 4.5 months. Table 5 shows the linear regression model testing the association between the time between the start of perinatal mental health problems and the first appointment with PMHS based on the service location, the experience of a simple care pathway, ethnicity, age, involvement of social services and main reason for referral. The time until the first perinatal appointment with PMHS in weeks was shorter for patients in the London subsample (p < 0.001), and those who were experiencing current deterioration in mental health (p = 0.016). The results suggest that ethnicity was not a significant predictor of the time to first perinatal appointment. Taken together, around a quarter of the variance in time from experiencing symptoms to being seen by PMHS can be explained by the variables included in the model (R² = 0.24).

Main findings

• Primary care services emerged as the primary point of contact for most patients seeking help in the perinatal period, highlighting the crucial role of general healthcare providers and midwives in recognising perinatal mental health issues. The majority of patients accessed PMHS through primary care and their pathway was simple, that is they saw one service before PMHS.

• Several issues related to access to PMHS were identified. There was a pronounced variation in time between first experiencing symptoms and obtaining help from services, there were significant differences in pathways between the participating services, and approximately one-third of the sample came into PMHS via a so-called ‘complex’ pathway.

• The study did not reveal substantial differences in the variables studied between different ethnic groups, indicating a degree of equality in access to PMHS.

• The study highlights the influence of service location on patient pathways, encouraging further exploration of this phenomenon to better tailor mental health services to patients’ needs.

• The majority of study participants sought help for their mental health problems, thus indicating that opening services to self-referrals could further improve access to PMHS.

Strengths and limitations

The study has several strengths. This is the first ever study to explore care pathways to PMHS in the UK and globally. The study offered unique insights into the type and length of patient journeys to PMHS. The issue of accessibility of PMHS for women from ethnic minority groups had previously not been explored and results from this study can be used when improving access to PMHS. The study adapted the WHO encounter form, which can be used by other services and researchers in the future.

The study also has several limitations. Due to pragmatic reasons, ethnic groups were simplified into merged groups (e.g. Black, Asian, White), which can be problematic because this obscures any within-group differences. For example, within the Asian group, Pakistani and Bangladeshi have higher levels of diagnosed ill health than the White population in England. 19 Furthermore, it has been well recognised that these large ethnic groups are distinguished by their cultural, linguistic, religious and economic variations. Next, the sample size was limited, thus preventing exploration of pathways across more specific ethnic groups. The study samples matched the population of the Birmingham and East London catchment areas in terms of ethnicity, indicating that the results are generalisable to these areas. However, it is important to note that results may not be generalisable to other areas with different ethnic compositions. Data were extracted from available clinical records and not validated with patient’s own accounts of their pathways to care. In future studies, it is advisable to conduct interviews with patients to ensure comprehensive documentation of all interactions with services, as not all of these may be recorded in electronic records. The quality of data depends on the quality of available records. The study focused on accepted referrals and declined referrals were not analysed, although this could have provided additional information about service access. However, the primary reason for declining referrals was not meeting the commissioned services’ threshold, which requires individuals to have moderate to severe mental illness. Next, inconsistencies between records across services could have contributed to identified differences in pathways. The study was conducted before the COVID-19 pandemic, so pathways may be different in the post-COVID-19 period mainly due to poorer access to primary care, further expansion and increased capacity and visibility of PMHS. The first encountered service or professional could have been outside healthcare services (e.g. community group or religious leaders), which was not sufficiently explored. Self-referrals were not available at the time of the study; most services started offering them in 2021–2. This could change pathways, especially since we found that most women initiated contact with services; however, currently, services still do not receive extensive number of self-referrals (e.g. eight patients in the London service during 2022). This study focused only on patients who were referred to PMHS for the first time. It may be that there are entirely different patterns of pathway use for women who are being referred back into PMHS. The studied pathways were categorised into simple versus complex. This can be considered a simplification, as the type of pathway is defined by the size of the patient-service contacts, not their quality.

Interpretation of study findings

As mentioned, patients’ initial contacts with services were primarily within so-called universal services, such as primary care, maternity services and health visiting. Notably, in over half of cases, the first clinician or service contacted for women’s perinatal mental health disorder made the referral to PMHS. This type of pathway, dubbed a ‘simple pathway’, is suggestive of good accessibility, an effective referral system and good responsiveness from services. As mentioned earlier, untreated perinatal mental health disorders can pose substantial risks to both the pregnancy and the infant. Hence, having access to services through a straightforward pathway can offer significant advantages. These findings also provide evidence for effective partnerships within the wider system of care working with women and families, as recommended by the NHS England guidance on pathways in perinatal mental health care. 8 They also confirm that the perinatal period offers a unique window of opportunity to detect and treat mental illness, due to increased frequency of contact with healthcare professionals during regular antenatal and postnatal visits. 2

The study also revealed a few worrying findings in regard to access to PMHS. There was a pronounced variation in time between first experiencing symptoms and obtaining help from services, with median time being 20 weeks. The prolonged duration of patients’ journey to PMHS is a concern, as it may result in substantial risks to both the pregnancy and the infant due to delays in care delivery. It is important to note that women with delayed access to services could have received support from family, friends or community organisations. 20 However, we can argue that adding specialist treatment to existing support could facilitate recovery and certainly be helpful for women with little external support.

This study also showed that White women, compared to Asian and Black, more frequently went through complex pathways to access PMHS. However, in regression models, ethnicity was not a variable that accounted for a significant variance in the type of pathway or time taken to get into PMHS. This can be considered as a positive finding; however, more research into this matter is needed and studies with a larger sample size and with less merging of ethnic groups can provide further insights. Previous research has indicated that services need to make more efforts to address this issue and ensure that women from under-represented groups are able to access any support needed. 21–23

The study also found significant differences in pathways between participating services. In Birmingham, patients were more often referred to PMHS from secondary care services, they were more often experiencing deterioration in mental health, and they more often went through complex pathways. The most frequent complex pathway was characterised by more contacts with midwives and this can be explained by the fact that the Birmingham subsample included more pregnant women. Additionally, inclusion of specialist midwifes, which then made the pathway more complex, was not necessarily a ‘negative’ finding. This may have lowered barriers for midwives and contributed to more pregnant women being referred to PMHS. In London, patients referred to PMHS were predominately coming from primary care, the main reason for referral was reviewing and monitoring of pre-existing condition, and patients more often went through simple pathways. The most frequent complex pathway can be described as crisis pathway and included presentation to A&E and crisis resolution and home treatment teams.

The role of service location and accompanying factors requires further exploration. The variation identified in this study was not related to differences in clinical needs between patients, indicating it is other, possibly spurious factors playing a part in shaping women’s experience of service. In-depth research should explore the factors driving this. Different methods will be needed to obtain a full picture of what is happening: including service-level data, and perhaps ethnographic research to observe and understand how referrals happen (or do not happen) and the decision-making processes involved. These findings and future research should also consider the wider literature and recent reports which point to different groups having different experiences in services, and the role that might play in determining pathways to care. 21,24

Implications for clinical practice and research

The study demonstrates the benefits of investigating patients’ referral pathways, rather than focusing solely on the caseload/access statistics. Services could use the adapted WHO encounter form for regular audits of a sample of patients accessing PMHS as an overall quality measure of the efficiencies of their pathways across services.

Furthermore, healthcare service standards appear to play a crucial role in determining patient pathways to PMHS. Improving service-level standards is essential for enhancing patient outcomes, satisfaction and overall healthcare system performance. To achieve this, a comprehensive and systematic approach is needed, which starts with conducting in-depth research and data analysis to identify gaps and areas of improvement. Future research should explore service-level factors that have the potential to affect the accessibility of services and patient pathways (e.g. partnerships with key stakeholders, allocation of resources, and support for patients regarding transportation and child care). Additionally, research should investigate the outcomes and risks associated with different pathways.

There is a remarkable dearth of research on the association between pathways to and through care and outcomes. This study suggests there is an enormous degree of variation. It should be considered that an assumption has been made in this study that more complex journeys with more specific healthcare clinicians is worse than simple and shorter journeys. Qualitative studies of women experiencing PMHS support the notion that a complex pathway, involving encounters with numerous services and carers, can be disruptive. 9,10 Women often describe feeling abandoned and as if they had to fight for help, which does not make them feel cared for by the services. 9 Future studies could also explore whether simple pathways were actually associated with better health and well-being outcomes (also based on the background variables explored in this study) – in other words, what is the contribution of the pathway above and beyond those variables.

An important implication for care that arises from this study’s findings is the reminder to not assume all women have the same experience. Clinicians and everyone encountered on women’s pathways should realise the importance that each contact has, and that the clinician may not have good understanding of what has come before them on a patient’s journey. It seems beneficial to discuss this point during patient’s first contact with the service.

Equality, diversity and inclusion

This research focused on understanding the inequalities and inequities that persist in perinatal mental health settings. Equality, diversity and inclusion is an integral aspect of this work, as evidenced by our research team’s diverse experiences related to perinatal mental illness, ethnic background, migration history and professional disciplines (clinical psychiatry, psychology and research roles). Additionally, we carefully considered language pertaining to gender and ethnicity. The participant populations were diverse and inclusive in terms of ethnicity and migration status. For this study, data were extracted from the records of patients under PMHS. We were not able to verify how they describe or identify themselves. We also acknowledge that combining ethnic groups is not ideal, as discussed under Limitations. We made efforts to report data for all ethnic groups, including ‘Mixed’ and ‘other’ that were not used in the main analysis due to small numbers.

CRediT contribution statement

Nikolina Jovanović (https://orcid.org/0000-0002-7554-9837): Conceptualisation (lead), Investigation (lead), Methodology (lead), Formal analysis (supporting), Supervision (lead), Writing – original draft (lead), Writing – editing and reviewing (lead).

Zan Lep (https://orcid.org/0000-0003-0130-4543): Formal analysis (lead), Writing – original draft (supporting), Writing – editing and reviewing (supporting).

Jelena Janković (https://orcid.org/0000-0001-5542-0051): Conceptualisation (supporting), Supervision (equal), Writing – editing and reviewing (supporting).

Aysegul Dirik (https://orcid.org/0000-0003-0812-8458): Data curation (lead), Project administration (lead), Supervision (supporting), Writing – editing and reviewing (supporting).

Anna Rees (https://orcid.org/0009-0001-1855-9270): Data curation (equal), Writing – editing and reviewing (supporting).

Maev Conneely (https://orcid.org/0000-0001-8326-4498): Writing – original draft (supporting), Writing – editing and reviewing (supporting).

Ethics statement

The study is part of the National Institute for Health and Care Research-funded (NIHR) project entitled ‘Accessibility and acceptability of perinatal mental health services for women from Ethnic Minority groups (PAAM)’ (17/105/14). This study used service-level data. As there was no contact with patients for the study, Health Research Authority approval was not required, and obtaining consent from individuals was not required. The two participating services provided local approvals.

Information governance statement

East London NHS Foundation Trust (ELFT) is committed to handling all personal information in line with the UK Data Protection Act (2018) and the General Data Protection Regulation (EU GDPR) 2016/679. Under Data Protection legislation ELFT is the Data Processor; the Department for Health and Social Care (DHSC) is the Data Controller, and we process personal data in accordance with their instructions. You can find out more about how we handle personal data, including how to exercise your individual rights and the contact details for DHSC’s Data Protection Officer https://www.nihr.ac.uk/documents/nihr-privacy-policy/12242#how-we-protect-your-personal-data.

Disclosure of interests

Full disclosure of interests: Completed ICMJE forms for all authors, including all related interests, are available in the toolkit on the NIHR Journals Library report publication page at https://doi.org/10.3310/YTRK6337.

Primary conflicts of interest: None.

Data-sharing statement

All data requests should be submitted to the corresponding author for consideration. Access to anonymised data may be granted following review.

Department of Health and Social Care disclaimer

This publication presents independent research commissioned by the National Institute for Health and Care Research (NIHR). The views and opinions expressed by authors in this publication are those of the authors and do not necessarily reflect those of the NHS, the NIHR, MRC, NIHR Coordinating Centre, the Health and Social Care Delivery Research programme or the Department of Health and Social Care.

This article was published based on current knowledge at the time and date of publication. NIHR is committed to being inclusive and will continually monitor best practice and guidance in relation to terminology and language to ensure that we remain relevant to our stakeholders.

Study registration

The study was conducted as part of the larger NIHR-funded research project, which received approval from the Health Research Authority through the Research Ethics Committee (REC reference: 19LO1830, 264632).

Funding

This article presents independent research funded by the National Institute for Health and Care Research (NIHR) Health and Social Care Delivery Research programme as award number 17/105/14.

About this article

The contractual start date for this research was in May 2019. This article began editorial review in October 2023 and was accepted for publication in June 2024. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The Health and Social Care Delivery Research editors and publisher have tried to ensure the accuracy of the authors’ article and would like to thank the reviewers for their constructive comments on the draft document. However, they do not accept liability for damages or losses arising from material published in this article.

Copyright

Copyright © 2024 Jovanović et al. This work was produced by Jovanović et al. under the terms of a commissioning contract issued by the Secretary of State for Health and Social Care. This is an Open Access publication distributed under the terms of the Creative Commons Attribution CC BY 4.0 licence, which permits unrestricted use, distribution, reproduction and adaption in any medium and for any purpose provided that it is properly attributed. See: https://creativecommons.org/licenses/by/4.0/. For attribution the title, original author(s), the publication source – NIHR Journals Library, and the DOI of the publication must be cited.