Early morbidities following paediatric cardiac surgery: a mixed-methods study

Clinicians’ use of language

Being unprepared for certain complications

Families information needs

Overview

This section of our study provided a clear steer that parents consider longer-term events that on the surface may not appear to be directly related to the heart itself, such as difficulties establishing oral feeding after discharge home and parental psychological stresses, to be ‘legitimate morbidities’ linked to paediatric cardiac surgery. As such, these are suitable topics to cover during preparation for paediatric cardiac surgery, ideally backed up by appropriate data concerning incidence and impact. Furthermore, the analysis of parental comments highlighted clinician, parental and situational factors that may influence parental understanding of potential morbidities following paediatric cardiac surgery and, consequently, aspects of communication about morbidities that can be targeted for improvement. These are discussed further as follows.

Preparation

The aim of the first meeting was to identify a shortlist of 15–20 candidate morbidities that would then be considered by the definitions group. Prior to the meeting, the panel was supplied with an extensive list of candidate morbidities identified through:

• a systematic review of the literature – the search terms, a PRISMA diagram and an exhaustive list of morbidities (see Appendices 1 and 2) and the reference list are provided (see Report Supplementary Material 1)

• three facilitated focus groups held in different UK cities, with parents recruited by the CHF

• an online forum for patients and families hosted on the website of the CHF.

The panel was also sent an abridged version of the study protocol, a description of the role of the selection panel and an agenda. Each panellist was asked to identify among or beyond this list of candidate morbidities those they judged most important to monitor routinely according to a deliberately broad working definition of a surgical morbidity as:

Any health or emotional problem that arose as a result of the fact of surgery (whether directly caused by surgery/post-operative care or not).

For the first meeting, panellists were requested not to censor their suggestions on the grounds of the perceived difficulty of definition or measurement and it was stressed that another group would be making these judgements.

We used the nominal group technique,94,95 augmented by a robust voting process to determine group rankings of morbidities (see Appendix 3). The nominal group technique is designed to reduce the influence of perceived power differentials and of dominant personalities on group decision-making, while retaining the benefit of discussion absent from other systematic approaches to group decision-making, such as Delphi. 96 We inferred group preferences from individual rankings, using a method developed by Utley et al. ,93 which is briefly described in Appendix 3.

Structure of the first panel meeting

Each panellist was given the opportunity to speak uninterrupted for 2 minutes on the morbidities they considered important. Each suggestion was entered onto a spreadsheet, which was projected in the meeting room to ensure that there was accurate transcription. The panel was then given the opportunity to add to this initial list if they thought that something important had been missed.

The chairperson (TT) led a process of identifying suggestions that fell outside the working definition above (see Preparation): duplication among suggestions and merging of closely related suggestions. There was then a secret ballot in which panellists were asked individually to rank the resulting list of candidate suggestions in order of descending importance. The voting process and the method for generating group preferences from individual ranking data are described further in Appendix 3.

During a scheduled break, the group preferences were calculated from the individual rankings by CP and MU, who did not have a vote on which morbidities to measure. After the break, the group preferences were fed back to the panel. The chairperson then led a second round of discussion, focusing on the group preferences and giving panellists the opportunity to argue for specific morbidities being given greater importance, and for the group to further consolidate the list of morbidities.

There was a second round of secret ranking followed by feedback of group preferences, prior to a consensus being sought as to the prioritised shortlist of morbidities to be passed to the separate definition panel for an assessment on the feasibility of defining, measuring and monitoring each in routine practice.

Results of the first panel meeting

Of the 15 panellists, three could not attend. At this meeting, 66 morbidity terms were suggested during the round of 2-minute contributions from each panellist (see Appendix 3). In the discussion that followed, seven terms were removed as being irrelevant to our study, related to impacts of post-surgical morbidity that would be measured as part of our empirical study, were too long term in nature to be captured within our study or redundant given other terms suggested.

Of the remaining 59 terms, seven were accepted as candidate morbidities as they were and a further five terms were simply relabelled before being accepted. The remaining 47 terms were mapped onto 16 groups of two to eight terms, with each group considered to relate to a sufficiently similar phenomenon for them to be single candidate morbidities. Members of the panel confirmed that they wanted the term ‘extracorporeal membrane oxygenation (ECMO)/mechanical support’ to feature on its own, as well as being an indicator of a ‘major adverse event’ (MAE). A new term (‘liver injury’) was added as a candidate at this stage, with the agreement of the panel. This gave 29 candidate morbidities.

These candidate morbidities and the group ranking of importance among them in the first round of voting are shown in Table 2. It is worth noting that, in this first round of voting, the group ranked ‘new global permanent neurological impairment’ as the most important morbidity. After that, there was a group of 22 candidate morbidities that could only be separated by applying tie-breaks, with the other six candidate morbidities [including ‘necrotising enterocolitis’ (NEC)] ranked as less important.

In the discussion that followed the first round of voting, the panel merged the two items describing neurological impairment and individual panellists expressed surprise at the low ranking of ‘NEC’ and ‘vascular thrombosis’, leading to a discussion of these particular morbidities and their impact.

The results of the second round of voting are shown in Table 3. Although the panel’s view of the top three morbidities [including the merged item of ‘new permanent neurological impairment (global or focal)] was clear, there was a large group of 21 candidates that could only be separated by applying tie-breaks. This group included ‘NEC’ and ‘vascular thrombosis’. Given the lack of unambiguous group preference among these 21 candidates, the panel decided to request that the separate definition panel considered the 24 candidate morbidities given in bold in Table 3, with the remainder discarded.

Causal mapping

Following the first selection group meeting, it was decided that it might be useful for non-clinical members of the panel to have an accessible summary of any causal relationships among candidate morbidities so that, when choosing the best set of morbidities to monitor, any overlap or redundancy among candidates could be accounted for. To this end, a set of causal mapping exercises was conducted by one of the facilitative team (MU) separately with two of the panellists (IM and HJ). In each exercise, cards representing shortlisted morbidities were placed on a large sheet of paper and arranged left to right, with lines drawn to indicate potential causal relationships. Photographs were taken of these causal maps, which were then converted to diagrams (see Appendix 4).

Preparation

Prior to the second panel meeting, panellists were provided with a pack of materials containing:

• a summary of any estimates of incidence and impact of candidate morbidities from the systematic review

• the judgement of the definition panel on the feasibility of defining, measuring and monitoring routinely each candidate morbidity

• a summary of potential long-term impacts of each candidate morbidity from the definition panel

• a summary of the parent and family focus groups and the online forum

• diagrams generated through the causal mapping exercise provided in Appendix 4

• minutes from the first meeting

• a statement of the purpose of the second meeting and its agenda.

Structure of the second panel meeting

At the beginning of the second meeting the panel were given a brief reminder of the scope of the overall project, the remit of the selection panel and the time scales we were working to.

The panel were tasked with narrowing the list of shortlisted candidates to a selection of 6–10, the incidence and impact of which would be measured in five centres over 18 months. It was explained that the upper limit of 10 morbidities was a result of the sample size required for measuring the impact of distinct morbidities. It was explained that we could measure just the incidence of other morbidities, if possible, from routine data. Panellists were also alerted to the (then) recently launched ‘NHS England consultation on the future of Children’s Heart Services in England’, which highlighted the possibility of future national audit of surgical morbidities.

Panellists were asked to consider the following.

Results of the second panel meeting

Of the 15 panellists, six could not attend. An overview of the results of the first year’s selection process is given in Figure 1. The initial assessment of the definition panel as to the feasibility of defining and monitoring each of the candidate morbidities still in contention after the first selection panel meeting is shown to the right-hand side of Figure 1.

FIGURE 1.

Overview of results as presented to the selection panel. ICU, intensive care unit; JET, junctional ectopic tachycardia; SUI, sudden unforeseen incident. Reproduced from Pagel et al. 90 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/. This figure includes minor additions and formatting changes to the original text.

Reproduced from Pagel et al. 90 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/. This figure includes minor additions and formatting changes to the original text.

Eleven candidate morbidities were considered straightforward to define and monitor in routine practice: (1) ‘unplanned reoperation (URO)/reintervention’; (2) ‘length of intensive care unit (ICU) stay’; (3) ‘MAE (e.g. cardiac arrest, ECMO, serious untoward incident)’; (4) ‘ECMO/mechanical support’; (5) ‘NEC’; (6) ‘prolonged hospital length of stay’; (7) ‘acute kidney injury’; (8) ‘prolonged pleural effusion (PPE)’; (9) ‘vascular thrombosis’; (10) ‘surgical bleeding’; and (11) complete heart block’.

Six candidate morbidities were considered less straightforward: (1) ‘new permanent neurological impairment (global or focal)’; (2) ‘problems feeding’; (3) hospital-acquired infection’; (4) ‘poor communication between clinical team and family’; (5) ‘complications during surgery’; and (6) ‘phrenic nerve injury’.

The remaining seven candidate morbidities were deemed difficult to define and monitor in routine practice: (1) ‘new impaired cognitive function more than a month after surgery’; (2) ‘developmental delay’; (3) ‘low cardiac output’; (4) ‘mental health consequences’; (5) ‘recurrent laryngeal nerve palsy’; (6) ‘questionable clinical team decision and diagnosis’; and (7) ‘level of support at home’.

The output from the causal mapping exercises given (see Appendix 4) highlighted how ‘mental health consequences’ could be a result of several other candidate morbidities, how the majority of candidate morbidities could result in prolonged stay in the ICU or hospital and how ‘low cardiac output’ could become manifested in several of the other candidate morbidities.

Based on these assessments and the selection panel’s own previous assessment of the importance of these morbidities, it was agreed that the following candidate morbidities would be selected without further discussion: ‘new permanent neurological impairment (global or focal)’; ‘URO/reintervention’; ‘problems feeding’; ‘acute kidney injury’; and ‘poor communication between clinical team and family’.

The following candidate morbidities were discarded at this point without further discussion: ‘mental health consequences’; ‘prolonged hospital stay’, ‘questionable clinical team decision and diagnosis’; and ‘level of support at home’.

The remaining 15 candidate morbidities were then discussed, with the panel reminded of the need to select at most five from these 15. Group decisions were made to select ‘ECMO’, ‘MAE’, ‘NEC’, ‘hospital-acquired infection’ and ‘recurrent laryngeal nerve palsy’. Note that the group accepted that ‘MAE’ should not include ECMO, as this had been selected as a distinct morbidity.

Group decisions were made to discard at this stage ‘new impaired cognitive function more than one month after surgery’, ‘developmental delay’, ‘low cardiac output’, ‘PPE’, ‘vascular thrombosis’, ‘surgical bleeding’, ‘complications during surgery’, ‘phrenic nerve injury’ and ‘length of ICU stay’.

The online poll conducted after the second panel meeting identified ‘recurrent laryngeal nerve palsy’, ‘phrenic nerve injury’, ‘complete heart block’ and ‘PPE’ as potential substitute morbidities in the event of the definition panel vetoing any of the selected morbidities in routine practice.

Neurodevelopmental outcomes

At each stage of selection, the morbidity that was ranked of greatest importance by the panel was neurological impairment. This came as little surprise to the study investigators and vindicates the inclusion in our overarching programme of research of a parallel evaluation of the BDA, which it is hoped will provide a tool that can be deployed by nursing staff to identify patients who would benefit from referral to specialist neurological or other developmental services.

Differing priorities between health-care professionals and lay people

We found that opening up the process of choosing the metrics by which services should monitor their performance to include the perspectives of patients and family representatives, which is in line with policy initiatives in England,97 brought challenges. Throughout our work, there was a tension between choosing a ‘clean’ set of ‘clinical’ measures that most closely matched the understanding of ‘surgical morbidity’ among the tertiary clinicians on the panel, and the inclusion of arguably murkier phenomena considered hugely important by families and those working in secondary care. It is fair to say that inclusion on the panel of family representatives and clinicians from outside the tertiary surgical centres brought other issues, such as problems feeding and poor communication between clinical teams and families, to greater prominence than if the panel had consisted solely of tertiary clinicians, or if the study investigators had chosen the morbidities themselves.

Attribution of cause

In particular, those working in surgical centres were more concerned than family representatives and others with the attribution of morbidity to the surgical act, keen to include morbidities that may be related in part to surgical technique (laryngeal nerve palsy and phrenic nerve injury) and degree of success (low cardiac output), and were anxious to avoid the attribution to surgical teams of morbidities that are currently considered to ‘come with the territory’ of CHD and its surgical treatment. Family representatives and others highlighted the value of gathering information on the incidence and impact of key morbidities, even if they were not caused by surgery, not least as some of them may be reducible through interventions at other points in the care pathway.

Benefits and limitations of the methodology we used

We consider that the features of our study design were vital in terms of drawing out and balancing the differing perspectives. The nominal group technique, starting as it does with an opportunity for each panellist to speak without interruption and within an embedded democratic process, is specifically designed to minimise the influence of perceived power differentials and dominant personalities within a group. This was reinforced by the use of a secret ballot process to determine group preferences, allowing panellists to record their disagreement with the positions stated by others without that being openly declared. The voting tool used distinguishes between unambiguous group preferences and those that rely on tie-breaking. This acceptance and presentation of lack of consensus helped to focus discussion on where it would be most valuable to the task of selecting a group of morbidities and divert unnecessary debate focused on achieving a false consensus through attrition. Nonetheless, we recognise that the final selection of morbidities inevitably reflects the composition of the panel. In choosing panel members we wanted to keep the numbers down to a manageable number for an in-person group; we took the view that a group of > 14–18 participants would be too many for a constructive discussion to be feasible. Moreover, we also wanted a balance between project partners and independent panel members from non-study sites. We were disappointed that despite our best efforts we were unable to recruit a GP to the panel.

In addition, our choice to separate the process of identifying which morbidities are most important from the process of assessing the feasibility of defining and monitoring morbidities prevented all parties from self-censoring and clinical panellists from unconsciously using or claiming privileged knowledge of measurement processes to strengthen the case for morbidities that they wanted to include.

Firm and expert chairing was essential to maintaining this discipline. Having a chairperson conversant with the clinical area but also experienced in working with multistakeholder groups, including patient and family representatives, was also key. Although it meant that the chairperson brought their own clinical perspective to the table, the panel benefited from the chairperson’s ability to distinguish between the wheat and chaff of clinical discussions and summarise for non-clinical participants. It is also questionable whether or not a chairperson that was not an accomplished surgeon and clinical researcher would have held the respect of all parties through the process.

Separating the processes of judging the importance and the feasibility of routinely monitoring morbidities did, however, risk some of the subtlety of discussions slipping through the gaps between two panels of people. Although the preparation of detailed summaries of panel meetings and the presence of the same facilitating team (CP and MU) at all meetings reduced this risk, we acknowledge that the defined morbidities that will be monitored do not correspond exactly in all cases to the phenomena deemed important by the selection panel.

Consideration of pre-procedural factors

A major difficulty when contemplating the monitoring of morbidity following paediatric cardiac surgery is achieving a distinction between the morbidity that was present in the patient before the operation, compared with new morbidities that arose after surgery. It must be acknowledged that preoperative events, such as existing congenital diagnoses and patient condition, are inextricably linked to the postoperative journey6,9 and, indeed, both pre and postoperative events matter for the patient. Preoperative events may also potentially be subject to quality control, for example the collapse of a neonate from late diagnosis of heart disease leads to higher rates of multiple organ failure,109 and may be averted by antenatal diagnosis and prospective management of the circulation. 110 Nonetheless, our focus is on early outcomes after paediatric cardiac surgery, not the entire care pathway. Hence, the definitions are designed to delineate postoperative events as clearly as possible. The delineation of new neurological morbidity in a postoperative patient may be challenging because of the inherent difficulties of assessing (in particular) small infants that may be critically ill. Prospective serial evaluation including pre and postoperative scans and detailed neurodevelopmental follow-up is the ideal. However, this is not feasible within a UK NHS context, where cranial scans may only be undertaken based on clinical indicators of suspected neurological injury, hence our definition is pragmatic by necessity, although we hope that in the future it will be supplemented by enhanced methods of assessment.

Post-procedural timing

Conventionally, the time horizon linked to surgical complications has been taken as 30 days following the operation. 108 For mortality outcomes, registries, such as STS in the USA, view the relevant time horizon as within the same operative hospitalisation or 30 days, whichever is longer. 111 For the majority of the morbidity definitions the time limit of within 30 days and/or within the same hospitalisation was applied (see Box 1) based on what was considered most appropriate for the individual morbidity event. Certain morbidities, particularly those defined by the use of a technology (e.g. renal support, ECLS), are likely to occur only within a hospitalisation, whereas others may occur at any time point over an operated child’s lifespan (reoperation, endocarditis, feeding problems); hence, a time limit was placed accordingly in order to enhance the feasibility of postoperative audit, despite this time limit in some cases appearing arbitrary. It was noted that deep surgical site infection or mediastinitis, although always linked to cardiac surgery, may arise after discharge home and later than 30 days post surgery, so the timeline was extended for this morbidity.

Consistency and complexity of definitions

There are inherent practical difficulties with prospective audit of complex outcome measures; this is one reason for the historic focus on mortality as an outcome, as this is much easier to measure than morbidity. For some of the morbidities, a treatment indicating the presence of morbidity was considered the better option, rather than basing the diagnosis on clinical findings. This applies to the postoperative morbidities of renal failure, diaphragm paralysis and feeding problems, for which postoperative renal support, the need for diaphragm plication and technology-assisted feeding at discharge were selected as the most objective definitions available. A concern with using a treatment, rather than a diagnosis, as a measure of morbidity is that treating centres may initiate therapy at differing thresholds. During the course of our study, additional data items will be collected to explore the potential for such variation. To illustrate this further with examples: practice patterns in respect of technology-assisted feeding in cardiac babies vary widely between geographic regions and diagnostic groups, and it is acknowledged that the audit of feeding problems at discharge rather than over time in outpatients may not capture the full picture. 112 For the case of ECLS, there is an inextricable link between the severe condition of patients requiring this therapy and the burden of the treatment itself,10,11,102 and, therefore, this is reasonably widely accepted as a major morbidity after paediatric cardiac surgery by all stakeholder groups. 1 Moreover, considering the example of renal failure, given the complex inter-relationship between the patient’s preoperative condition (which may incorporate renal dysfunction28), their age (especially very young neonates, as is common32), their body mass index (which may be low in CHD), their postoperative condition and their measures of renal function, a definition involving a specified measure of renal function was considered to be impractical to define for routine use. Of note, it proved infeasible for the panel to agree a clear and usable definition of low cardiac output syndrome for use in routine audit.

Face validity

The BDA tool was designed by a multidisciplinary group, including paediatric neurologists, developmental experts, paediatricians, psychologists, nurses and a statistical expert. The BDA development group consulted the published literature on long-term outcomes of children with cardiac disease and critical illness in order to identify the optimal domains to incorporate within the measure, these being gross motor skills,14,36,38,125,126 fine motor skills,14,36,127,128 daily living skills,8,127,129 receptive and expressive communication,14,36,38,128,129 socialisation,130,131 and behaviour and coping skills. 8,119,132,133

Within each age band of the BDA, individual items were selected by the expert panel based on review of the following measures: Mullen,53 Battelle Developmental Inventory,134 Battelle Developmental Inventory Screen,135 Denver Developmental Screening Test II,136 Bayley-III Screener Test,137 Bayley-III Scales of Infant and Toddler Development, Bayley Infant Neurodevelopmental Screen,51 Ages and Stages,124 Developmental Neuropsychological Assessment,138 Wechsler Individual Achievement Test,139 Wechsler Abbreviated Scale of Intelligence (WASI-II),140 Wechsler Intelligence Scale for Children,141 Children’s Memory Scale,142 Behaviour Rating Inventory of Executive Function,143 Behavioural Assessment of the Dysexecutive Syndrome144 and Parents’ Evaluations of Developmental Status. 145

Preliminary testing of the BDA during the development phase is reported separately,54 with key excerpts presented as follows.

Internal reliability

The pre-validation data on internal reliability of the BDA were collected for 138 children. Of note, certain poorly performing items (α < 0.6) were revised in the next iteration of the BDA (version January 2014), which was used in the validation study.

Construct validity

The pre-validation data on construct validity of the BDA related to 17 children in a known group (Down’s syndrome, n = 12; other genetic syndromes, n = 5) who had significantly lower BDA scores [median 10; interquartile range (IQR) 8–15] than their age-matched counterparts (median 16, IQR 5–50, z = 3.08; p = 0.002), with an effect size of 0.53 (equating to a medium effect size). There were at least two matched pairs in each of the five age bands; hence, individual numbers in each age band were too low for valid statistical comparison.

Inter-rater reliability

The pre-validation inter-rater reliability data for the BDA based on 74 children were excellent. In terms of inter-rater concordance for children scoring as a ‘red’ in each age band, there was perfect agreement on both the BDA gross motor and the BDA cognitive score scales for children in each of the age bands 1–4 (κ = 1; p < 0.0.001).

Acceptability and feasibility

• The BDA took up to 10 minutes to complete and score, unless there was a requirement to use an interpreter.

• It was feasible to undertake the BDA in a ward or clinic setting, in terms of timing, space and integration with other ward or clinic work.

• A range of clinicians, in particular those based in community paediatric settings, were in favour of the traffic lights scoring system.

• Parents responded favourably to the BDA, reporting that the content was relevant to their child and that they found it acceptable and useful for their child to be assessed with it, and nurses commented that completing the BDA was a good ‘ice-breaker’ for children at pre-admission clinics and that it helped to build rapport with the child and family.

• The importance of training with explicit instructions for each item was emphasised by clinicians using the BDA.

Study population

The study setting was at the three tertiary children’s cardiac centres based in London and the time period of the study was January 2014 to July 2015. By approaching all children between birth and 17 years of age with heart disease attending outpatient or inpatient settings, excluding those who were clinically unwell within the ICU, a representative convenience sample of 200 patients with heart disease within each age band was recruited. Once the target of 200 participants was reached in any age band, recruitment to that age band ceased.

Study procedures

Written informed consent was obtained from parents.

Children were assessed in a quiet room by a small team of trained psychology assistants under the supervision of a single senior psychological researcher (JW) and a medical lead (AH).

Elements of the validation

The internal consistency of the BDA was assessed for items within each domain, between totals for domains and across the entire measure based on Cronbach’s alpha.

The internal reliability of the BDA was evaluated in terms of inter-rater agreement when two raters simultaneously and independently performed and scored the BDA. The study team consisted of at least three raters throughout the study duration, and the inter-rater performance assessment was undertaken whenever two raters could be scheduled to be free at the same time. Again, recruitment ceased as soon as the required number of patients was recruited.

External measures used for validation of the BDA in children aged < 5 years were Mullen53 and the Ages and Stages,124 details of which are presented in Table 5.

In children aged < 5 years the concurrent validity of BDA scores was assessed against Mullen scores. In children aged between 5 and 17 years the concurrent validity of BDA scores was assessed against WASI-II scores.

Evaluation of construct validity was based on detection of known abnormalities and, to that end, study participants were defined as falling within a ‘known group’ when the child had been diagnosed to have a condition linked to neurodevelopmental problems (a congenital syndrome,150 an acquired condition, such as stroke,151 or a previously diagnosed developmental delay based on a specialist clinical assessment, even if the cause was not stated), and when the Mullen result was amber or red.

Construct validity was further assessed by calculating sensitivity and specificity of the BDA for detection of neurodevelopmental abnormalities against both of the external measures.

Analysis criteria for measures

For the evaluation of agreement of BDA scores between two independent raters and with external measures:

• Each of the age bands was analysed individually.

• Raw scores were used in order to remove within-measure age standardisation, as this standardisation is undertaken differently within the BDA and the Mullen/WASI-II.

• Gross motor comparisons were available for only children aged < 33 months, reflecting the protocol for the Mullen, for which this domain is only assessed in younger children aged less than this cut-off.

• For children aged < 33 months, cognitive and gross motor scores were analysed separately, as this reflects the validated scoring protocol for the Mullen, which contains a cognitive summary score covering the domains of visual reception, fine motor, receptive language and expressive language, combined and a fifth individual domain of gross motor function separated.

Thresholds for validity

Interim analysis

The interim analysis involving the first 100 patients in each of the six age bands is shown in Table 6. As can be seen, the threshold for go was not met in the oldest age band; hence the BDA was not considered valid in this age group and further validation work stopped at this point. The younger five age bands involving children aged from 1 month to 5 years passed the interim analysis and the validation work continued as planned.

Final analysis

The case mix of 982 consented participants in the final validity study is shown in Table 7. The age distribution of the sample is skewed towards younger babies because the width of the five age bands narrows as age falls [median age across all age bands is 11.5 months (IQR 5 months–2.6 years)].

Internal reliability

The internal reliability of the BDA, expressed as Cronbach’s alpha, is shown in Table 8. This was high between BDA total scores and between all items, but low in selected domains of the BDA, particularly within the youngest two age bands representing children under 8 months old.

Inter-rater reliability

A total of 160 children participated in the evaluation of inter-rater reliability of the BDA (Table 9). Correlations were very high for all age bands, thus passing the pre-set threshold for inter-rater validity.

Agreement between the brief developmental assessment and the Mullen

Of the 981 participants, 21 did not complete one or more domains of the Mullen and thus a total of 960 children participated in the evaluation of concurrent validity of BDA against the Mullen (see Table 9). For age bands 2–5, the pre-set thresholds were met with the exception of gross motor in age band 2; however, in age band 1 the BDA displayed a much weaker correlation with the Mullen and therefore did not pass the pre-set threshold for validity.

Developmental outcomes

The developmental outcomes of participants based on the BDA, the Mullen and the Ages and Stages are presented in Table 10 and summarised as follows.

Of 960 children completing both the BDA and the Mullen, for BDA a total of 364 (38%) children had a green result, 361 (38%) children had an amber result and 235 (24%) children had a red result.

For Mullen, and considering both Mullen cognitive composite scores and (when applicable based on age) gross motor scores, 639 (67%) children had a green result, 178 (18%) children had an amber result and 143 (15%) children had a red result.

Data were missing for at least one Ages and Stages domain in 149 children (15%), and all of these children were excluded from validity analyses involving the Ages and Stages. Of 832 children completing the Ages and Stages, only 238 (29%) had a green result, whereas 213 (26%) had an amber result and 381 (46%) had a red result.

Construct validity

Of the 960 participants who undertook the Mullen, 227 (24%) had a condition linked to developmental delay, of whom 153 (67%) also had a red or amber Mullen result, thus meeting the criteria for a ‘known group’ with which to evaluate construct validity. Of these, 141 (92%) were also detected based on a red or amber BDA result, thus passing the pre-set threshold of 80%.

Surprisingly, 74 (33%) children with a condition linked to developmental delay had a Mullen result of green. Of these 74 children, 40 (54%) were under the age of 8 months and therefore based on young age, developmental delay may not yet be evident. Furthermore, although 17 (23%) children had a defined genetic condition such as Down’s syndrome, 13 (18%) had perioperative neurological events of unknown significance and the remaining 44 (59%) had a range of congenital abnormalities in which development incorporates a range of outcomes, including normality.

Moreover, there were 168 children with a Mullen result of red or amber, representing 18% of the study cohort that were not in a known group (i.e. they had no known genetic or acquired condition linked to developmental problems and no known diagnosis of developmental delay either stated by the parents or written anywhere in their medical records that were based at the tertiary hospital). The charts of these patients were reviewed (see Discussion and next steps) and this finding may relate to the underdetection of true abnormalities in the study population.

As might be expected, given that child development assessments in general are more reliable in older children, the sensitivity and specificity of the BDA against the external measures improved with increasing age, with the best performance in age band 5 and the poorest performance in age band 1, which did not meet the criteria for validity. As expected (given that the Ages and Stages is based on parental report only, whereas the Mullen is an objective-validated developmental test) the BDA performed better against the Mullen than the Ages and Stages. Considering all of the age bands 2–5 combined:

• The test measure of BDA red or amber has excellent sensitivity against the Mullen and good sensitivity against the Ages and Stages, but moderate to low specificity for both external measures.

• The test measure of BDA red has variable sensitivity but high specificity for both external measures. When considered, based on American Association of Pediatrics standards for the performance of a developmental screening tool (which state the sensitivity and specificity of a developmental screening tool should fall between 70% and 80%),152 the BDA outcome of red against an outcome of Mullen red is compliant.

Positive and negative predictive values, as well as comparisons between the Ages and Stages and the Mullen, are presented for information purposes in Table 11.

Summary of validation

The primary aim to validate the BDA as an early recognition tool for childhood developmental delay was achieved within a population with heart disease between the ages of 4 months and 5 years. Previous researchers have presented sensitivity and specificity across a range of thresholds, as a method to judge the performance of a new test against validated measures, and have used this approach to select the optimal threshold to trigger an abnormal result,154,155 as has been undertaken in this study. These analyses support the use of BDA results of both amber and red as thresholds to trigger further evaluation of a child, although after reassessment a proportion of such children may turn out not to have developmental delay. The protocol for such reassessment is currently being delineated within a Delphi survey and goes beyond the scope of the current study. The Delphi survey entails a series of questions to a large group of health professionals from a range of settings and backgrounds, which seeks to achieve a consensus as to the referral and reassessment pathway for children who have either an amber or a red BDA test result picked up by the cardiac team at the tertiary centre.

Limitations of the validation

Our positive evaluation of BDA validity and reliability relates only to tests at a single time point and the constructs of test–retest validity and responsiveness over time could not be assessed within the scope of this study. Both concepts are challenging to test within a rapidly developing population of very young children with a significant health problem, such as CHD, and a further dedicated study will be required to explore these, in particular repeated testing over time.

We note that the BDA has been developed as an early recognition tool for child development and it is not intended to replace full, formal neurodevelopment evaluation. It is our hope and intent that children flagged up by the BDA, when it is used with them by the cardiac team, will be speedily referred to and assessed by a neurodevelopmental clinic with a more detailed formal evaluation, using gold-standard neurodevelopmental tests.

A motivation underpinning our study was a hypothesis that the processes in place to assess the neurodevelopment of children with CHD require improvement within the UK, and children with CHD and developmental delay may be under-recognised. Indeed, 168 children with red or amber Mullen results were not in a known group, and a chart review, undertaken by one of three clinicians, revealed concerns from the parents about the child’s development and/or other risk factors for abnormal development, such as a history of cardiac arrest or mechanical circulatory support,10,156 in the majority. A review of the services that children were under and what actions might need to be taken to meet their needs is under way, and goes beyond the scope of the validation study. Health-care professionals and parents have commented, anecdotally, that children with CHD, such as those under the age of 5 years recruited to this study, are in general undergoing treatments for their heart, including surgery, and this represents the main focus of contacts with health professionals, including both those at the cardiac centres and also in the community such as health visitors. This may be represent a reason for these 168 children with red or amber Mullen results not already being identified as in a known group.

The BDA was developed for use with children who have heart disease, and has not been used or validated with healthy children. There is the potential that the BDA might be useful within other groups of children who, for medical reasons, are at greater risk of neurodevelopmental problems, such as survivors of other types of critical illness. However, in order to take this forward, further testing and research would be required.

Comment on external measures

Parents who were concerned about their child’s development preferred to watch the entirety of the testing, and were less likely than parents who had no concerns about their child to complete the Ages and Stages while their child was being assessed. This is supported by a comparison of the Mullen cognitive results between children with missing Ages and Stages (29% Mullen cognitive results red or amber) and those who completed Ages and Stages (19% Mullen cognitive results red or amber). The overall proportion of red results for the Ages and Stages (46%) was very high, and perhaps proportion of Ages and Stages results that were red would have been even higher, had the missing 15.2% been included.

This is not a cohort study with longitudinal follow-up, thus limiting interpretation. However, as displayed in Table 11, developmental delay based on Mullen results was detected least frequently in the youngest babies, in contrast with developmental delays based on Ages and Stages results, which were detected most frequently in the youngest babies. Medical ill health in children with CHD is more prevalent in infancy, as this is a period when interventions are commonly undertaken. These observations support a hypothesis that, in children with CHD, the Ages and Stages may be picking up a range of issues (including developmental delay), but also general ill health. This further emphasises the importance of an initial evaluation for signs of developmental delay that incorporates direct observation of children with CHD, such as the BDA provides. Furthermore, this emphasises the recognised importance of periodic assessment of neurodevelopment over time in children with heart disease.

Recruitment and data collection

All children aged < 17 years undergoing cardiac surgery in each of the five participating centres were prospectively monitored for the presence of the morbidities selected90 in Chapter 4 (and defined98 in Chapter 5) by the clinical teams liaising with the dedicated research nurse and the consultant surgeon or intensivist at each site. The original study protocol set out the incidence study timeline as 18 months at all sites, but this timeline was amended slightly for practical reasons and actual recruitment dates were as follows:

• GOSH: 1 October 2015 to 30 June 2017 (21 months)

• EVE: 1 November 2015 to 30 June 2017 (20 months)

• BIRM: 1 October 2015 to 30 June 2017 (21 months)

• BRHC: 1 October 2015 to 30 June 2017 (21 months)

• GLA: 15 October 2015 to 30 June 2017, with a break December 2015 to January 2016 (18 months).

Incidence data on each of the defined morbidities were collected in line with the protocols stated in Chapter 7. 98 Alongside the prospectively collected data on morbidities, we collected selected key fields from nationally mandated audit data for the NCHDA24 and the Paediatric Intensive Care Audit Network (PICANet). 158 Data collected were drawn from copies of the audit data held within each study site and were pseudoanonymised before being provided to the research team: all names, numbers, dates and places were removed. For analysis, the diagnoses and procedures were grouped (see Case mix risk factor variables), thus further reducing the chance of de-identification. The advantage of harnessing national audit data for this study was that each field is clearly and consistently defined, it is mandatory to record every cardiac procedure and NCHDA data, overall, are externally validated. Recruitment by centre is shown in Appendix 6.

Inclusion and exclusion criteria

The age inclusion range was from birth to 17 years.

Cardiac surgery in this instance, for the purposes of both the incidence and the impact studies, was defined as any open, closed or hybrid operation involving the heart or circulation that is categorised as a cardiac surgery procedure and subjected to mandatory audit by the NCHDA,24 apart from the stated exclusions, which were:

• premature babies undergoing persistent ductus arteriosus ligation, as these are cared for by neonatal teams over a prolonged period, and experience a different profile of risks and outcomes to patients with CHD

• cardiothoracic transplant procedures and tracheal surgeries, as these procedures are undertaken at only one of the five study centres and have a very distinct risk profile and set of morbidities, therefore these were considered to be outside the current scope.

Isolated interventional cardiology procedures were not included, as these go beyond the scope of the study topic.

Case mix risk factor variables

Given the known heterogeneity within the study population,159,160 we prespecified important clinical risk factors in the protocol based on previous documented empiric links with mortality, as well as clinical expert viewpoints, including patient age, weight, cardiac diagnosis, operation type, bypass time and comorbidities. As a preparatory step prior to statistical analysis, we collapsed the categories for cardiac diagnosis and cardiac procedure as follows in Cardiac diagnosis category and Specific procedure category, in order to remove the smaller categories and to help with clinical interpretation. For the category of comorbidity we used existing broad categories as described in Comorbidity. Note, in all of the risk factor groupings we referenced previous empiric research related to paediatric cardiac surgery mortality, described as follows.

Primary outcome measures

The main outcome measure for our analysis was the occurrence of the selected and defined morbidities. 90,159 These morbidities occur as single events in isolation and can also occur as combinations, which we refer to as ‘multiple morbidity’. During the design of the study we noted the occurrence of multiple morbidities as being a particularly adverse outcome for patients and, hence, a priori we hypothesised that outcomes in these this group would be especially poor. Furthermore, we noted that the special case of ECMO or ECLS, which, based on experience of the study team and within the scope of the literature review (see Report Supplementary Material 1 for complete reference list), often occurs in conjunction with other morbidities and has a particularly poor outcome in terms of mortality and length of stay. 165–169 Therefore, a rule was created such that the occurrence of ECMO or ECLS was counted as a single standalone morbidity whether it occurred as a single morbidity or in combination with other morbidities.

Secondary outcome measures

During the course of the study the independent Steering Committee recommended that the mortality of patients in the incidence study should be assessed at the time point of 6 months after the operation, and therefore this outcome was included in the analysis. The patients’ outcome at 6 months was provided at the end of the study and was rechecked at the study sites in March 2018, using a combination of hospital records and NCHDA data.

During the course of the study the selection panel recommended that an assessment of length of hospital stay should be included in the analysis; therefore, this outcome was described within the incidence analysis and is reviewed in further detail in the impact analysis. The length of stay was defined as the number of days between the operation that led to the child entering the study and the date of discharge from the specialist cardiac centre. Two data sources (study database and NCHDA) were cross-checked to ascertain the accuracy of these data.

Data checking and validation

Morbidity cases were prospectively diagnosed based on the assessment of the dedicated study nurse and the local principal investigator who was either a consultant paediatric cardiac surgeon or a consultant paediatric intensivist, working with the clinical team caring for the child. Every child meeting inclusion and not meeting exclusion criteria was reviewed regularly over the course of their hospital stay at the treating centre, to ascertain whether or not the criteria for one of the morbidities were met. A case record form was completed for every patient whether or not morbidity was diagnosed and eventually signed off by the dedicated study nurse and the local principal investigator. Additional data checks to ensure accuracy of study data and complete case ascertainment for incident morbidities were undertaken as follows:

• A monthly telephone conference call involving at least one person from all sites was held, in which any grey cases were discussed and final case ascertainment was agreed.

• A check of a 3-month sample of data from each study site, relating to data collected on 443 patients between 1 January 2016 and 31 March 2016, was undertaken. Every case from this 3-month time period was cross-checked against the local case record collected for the NCHDA to compare the level of completeness for ECLS, URO, renal support or dialysis (Renal), PPE and part of MAE, which by that time point were common to both data sets. The mandatory data collection for the NCHDA is carried out within each centre independent of data collection for our study. Note that all of the data used for the study from the NCHDA were collected at the end of prospective morbidity data collection. There was excellent agreement between the two sources, with nine morbidities present in the study data set but not in the NCHDA, and zero morbidity found the NCHDA but not in the study data set.

• A final reconciliation of morbidities was undertaken at the end of the study, in which any cases where there was partial data completion for the morbidities were re-reviewed locally by the dedicated research nurse and a senior local clinician.

Note on sample size and morbidity groups

In the original study protocol we anticipated that between 3000 and 3300 surgical patients would participate in the analysis across the five sites. We expected that this would have been a sufficient sample to estimate accurately the incidence of each of the morbidities. For example, an observed incidence of 3% would have CI 2.4% to 3.6%. In fact, as we demonstrate later in Results, for five of the single morbidities in isolation the incidence was < 1.5%.

Therefore, we used three approaches to group morbidity outcome:

1. Two categories – any morbidity compared with no morbidity.

2. Four categories – single morbidity (excludes ECLS or ECMO), ECLS or ECMO, multiple morbidity and no morbidity. This outcome enables the discrimination of risk factors for the particularly adverse outcomes of ECLS and multiple morbidities.

3. Eleven categories – each of the nine individual single morbidity groups as defined, multiple morbidity and no morbidity. The individual morbidities were ECLS, ANE, URO, feeding morbidity (Feed), MAE, PPE, post-surgical infection (SSI), Renal and NEC.

Data analysis

Risk factors are presented with frequency (%) for categorical risk factors and mean (SD) or median (IQR), as appropriate, based on the distribution of the data for continuous risk factors.

The incidence of each of the selected morbidities was estimated with 95% CIs, both for a single event of one of the defined morbidities and in combination with other morbidity. Multilevel logistic regression analysis was used to explore the role of preoperative, patient-level case-mix factors on morbidity outcome 1 (any morbidity vs. no defined morbidity), accounting for multiple procedures within patients.

The case mix risk factors considered were sex; age band (neonate, infant child); calculated weight-for-age z-score;170 cardiac diagnosis category; functionally univentricular heart; specific procedure type category, operation type (bypass, non-bypass or hybrid); bypass time; acquired comorbidity; congenital comorbidity, excluding Down syndrome; Down syndrome; additional cardiac risk factors; prematurity; and severity of illness indicator. 162

For the four-category morbidity outcome 2, multinomial regression was used with robust standard errors to adjust for clustering within patients. For both outcomes we investigated whether or not the inclusion of site as a random factor was important and found that results were almost identical to those without site; therefore, we present results from the model without site.

Univariate models were fitted and the estimated effects are presented along with 95% CIs. All factors significant on univariate analysis (p < 0.1) were considered in the multivariable models. Data completeness was excellent for most of the risk factors, although there were some missing data for weight: we state the number of missing values when relevant in results. We used multiple imputation by chained equations to account for missing data and the imputation model included all of the risk factors considered in the univariate analysis, which we assumed to include all predictors of missingness. The final models were derived by fitting a regression model for all significant predictors and estimates were combined using Rubin’s rules. 171 Model performance for the final multivariable models was assessed using the c-statistic (area under the receiver operator curve) and Hosmer–Lemeshow statistic. All analyses were performed in Stata^® 14 (StataCorp LP, College Station, TX, USA).

We did not consider it appropriate to undertake multinomial regression for the 11-factor outcome, given the low rates of < 1.5% for five individual morbidities, and present descriptive data for these individually.

Descriptive data

The flow chart depicting inclusion and exclusion is shown in Figure 3. Of note, reoperation within 30 days was an outcome of the study and hence these are not reported in the headline total number of procedures.

FIGURE 3.

Morbidity incidence flow chart. a, Records matched on unique procedure identification.

Morbidities

After the exclusions, as shown, there were 3090 procedures in total from 2861 patients. A total of 213 patients had two surgical procedures and 16 patients had three surgical procedures. Of the 3090 procedures:

• 2415 (78.2%) resulted in none of the nine defined morbidities

• 675 (21.8%) resulted in at least one morbidity

• 478 (15.5%) resulted in a single morbidity, including those with ECLS

• 197 (6.4%) resulted in a multiple morbidity, excluding ECLS.

Mortality

There were 16 patients discharged alive for whom we were unable to recheck the life status at 6 months. Given instances of multiple procedure-based admissions in the same patient, we report mortality rate at 6 months based on the outcome after the latest procedure. In all cases of mortality this was a second procedure (there were no mortalities after more than two procedures). Among 2861 unique patients there were 92 patients (who between them underwent 105 procedures) who died within 6 months, giving a mortality of 3.2%.

Looking at the latest procedure for each of the 2861 unique patients:

• 21 out of 2294 (0.9%) patients without one of the defined morbidities died

• 71 out of 567 (12.5%) patients with one or more of the defined morbidities died.

For those patients with at least one defined morbidity:

• 17 out of 355 (4.8%) patients with a single morbidity died (the 17 deaths with a single morbidity comprised one ANE, four URO, one Feed, two Renal, five MAE, three NEC and one PPE)

• 27 out of 54 (50%) patients with ECLS died

• 27 out of 158 (17.1%) patients with multiple morbidity died.

After cross-checking the length of stay data from the two stated sources we found that values agreed, except for three admissions which were set to missing. Overall, the length of stay was missing for nine patients. Length of stay in hospital is shown in Table 12 and Figure 4 by morbidity type and by morbidity group.

FIGURE 4.

Length of stay by morbidity groups and individual morbidities. Box plot shows the median and IQR: 25th (Q1) to 75th centiles (Q3). The upper and lower bounds are Q3 + 1.5 × IQR and Q1 – 1.5 × IQR as defined by Tukey. 172 Multi, multimorbidities.

Incidence of morbidities

The most common morbidity was PPE (6.5% as any occurrence), then Feed (6.0% as any occurrence) and then URO (5.2% as any occurrence). Considering single morbidities, only four morbidities occurred in isolation at a rate of > 1.5% (this was a lower threshold prespecified in our protocol, below which a morbidity might be considered rare or uncommon); these were PPE, Feed, URO and ECLS. However, if considering morbidity rates based on any occurrence, meaning as a single morbidity or in combination with other morbidities, all of the nine selected had a rate of > 1.5%, the least common being ANE, with a rate of 2.1% (any occurrence).

The incidence of morbidity is shown in Table 13 and Figure 5, both when in combination with other morbidities or as a single event.

FIGURE 5.

Incidence of morbidities by procedure, with 95% CIs. Multi, multimorbidities. Reproduced from Brown et al. 157 © 2019 Published by Elsevier Inc. on behalf of The American Association for Thoracic Surgery. This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY-NC-ND 4.0) license. See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

Reproduced from Brown et al. 157 © 2019 Published by Elsevier Inc. on behalf of The American Association for Thoracic Surgery. This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY-NC-ND 4.0) license. See:https://creativecommons.org/licenses/by-nc-nd/4.0/. The text below includes minor additions and formatting changes to the original text.

Multiple morbidities

There were 197 (6.4%) procedures that resulted in a multiple morbidity, excluding ECLS. Of these 197 procedures, 76 (39%) involved a feeding problem, 73 (37%) had URO, 72 (37%) had PPE, 67 (34%) involved a MAE, 66 (34%) involved renal support, 49 (25%) involved SSI, 34 (17%) involved ANE and 33 (17%) involved NEC.

For the 197 multiple morbidity cases, 140 involved two morbidities, 39 involved three morbidities, 17 involved four morbidities and one involved five morbidities.

Extracorporeal life support morbidities

Among the 62 (2%) procedures in which there was postoperative ECLS, only two involved, ECLS only and no other morbidities. Among these 62 ECLS morbidities there were 37 (60%) with Renal, 33 (53%) with MAE, 29 (47%) with URO, 19 (31%) with PPE, 16 (29%) with ANE, 10 (16%) with NEC, 9 (15%) with SSI and 9 (15%) with Feed.

Risk factors for any morbidity

Table 14 includes the frequency (%) for categorical risk factors and mean (SD) or median (IQR), as appropriate, based on the distribution of the data for continuous risk factors. The weight was missing for 87 patients and the weight for age was missing or not valid, based on being an extreme outlying value of > 5 SD from the normative mean, for 186 patients (multiple imputation was used).

A summary table of risk factors by morbidities is presented in Appendix 8.

The univariate analysis showed that all risk factors were statistically significant for any morbidity, except for sex, prematurity and Down’s syndrome. Multivariable analysis showed that all of the factors that were significant on the univariate analysis remained significantly, independently associated with outcome, apart from low weight.

Age was an important risk factor for the occurrence of any morbidity after adjustment for other factors: compared with children aged ≥ 1 year, neonates had an increased chance of any morbidity, with an odds ratio (OR) of 5.26 (95% CI 3.90 to 7.09) and, similarly, infants had an increased risk OR of 1.61 (95% CI 1.26 to 2.05). Cardiac diagnosis group was the next most influential factor in the multivariable analysis, with more complex conditions carrying a higher risk for any morbidity (adjusted OR > 2 for diagnoses A, B and D; see Table 14 for full data), followed by a prolonged bypass time in excess of 90 minutes (adjusted OR 2.8, 95% CI 1.67 to 3.12). A palliative or staged procedure increased the chance of a morbidity arising (OR 1.6), as did the presence of a functionally univentricular heart (adjusted OR 1.6, 95% CI 1.07 to 2.24) (these two factors clearly have some overlap) or a severity of illness factor (adjusted OR 1.5, 95% CI 1.16 to 2.00) (which includes pre-procedure mechanical ventilation or shock). A child being underweight or having acquired comorbidity, congenital comorbidity and additional cardiac risk factors was not so influential for occurrence of any morbidity.

The area under the receiver operator curve for the final multiple logistic regression model for any morbidity was 0.77 (95% CI 0.75 to 0.79; Hosmer–Lemeshow goodness of fit p = 0.13).

Recruitment

Morbidity cases and matching of pairs

Cases of morbidity have been defined in Chapter 5 and include postoperative occurrence (either single or in combination) of ANE, URO, feeding morbidity (Feed), renal support (Renal), MAE, treatment with ECLS, NEC, SSI and PPE.

We aimed to pair each recruited morbidity case to the next available morbidity-free patient from within the same centre who had the following three criteria:

1. Age (matched within 3 months for children < 1 year, within 1 year in children < 5 years and within 2 years in children > 5 years).

2. Single or double ventricle status. 173

3. Surgical procedure type matched by the broad Risk Adjusted classification for Congenital Heart Surgery (RACHS-1) category2 or where there is a choice by the finer Central Cardiac Audit Database procedure classification. 174

In the first 6 months of the study, patients were matched based on all three of these criteria; however, from month 7 onwards the third matching criterion was removed on the advice of the Steering Committee, because of difficulties finding the necessary three-factor matches.

Our approach to children with comorbid conditions was to recruit them as either cases or controls, dependent on the presence of morbidity.

Controls were recruited postoperatively and close to patient discharge in most instances, so as to be sure that they were morbidity free. In a handful of cases in which patients were recruited earlier than this and a morbidity was diagnosed between recruitment and discharge, the patient status was reassigned as a case.

Data collected

Study patient data collection included demographics and clinical information regarding the procedure that was undertaken and its linked hospital admission that were entered in the bespoke study database. In addition, we obtained extracts of nationally mandated audit data from for the NCHDA24 on all procedures undertaken over 6 months of follow-up and the PICANet158 on all admissions over 6 months of follow-up.

Based on the case mix factors collected, we utilised the following variables in the impact data set in our analyses: sex, age, age band (neonate, infant child), weight at primary operation, calculated weight-for-age z-score, cardiac diagnosis category (A most complex to E least complex), functionally univentricular heart, specific procedure type category (palliative/staged, corrective/reparative or ambiguous), operation type (bypass, non-bypass or hybrid), bypass time, acquired comorbidity, congenital comorbidity excluding Down syndrome, Down syndrome, additional cardiac risk factors, prematurity and severity of illness indicator. (For details and rationale for clinical variable categorisations see Brown et al. 164 and Cole et al. ,170 and Chapter 7.)

We also utilised information on family income (five categories), ethnic group (categorised as white/non-white) and study site (five categories).

Measurement of impact

We assessed the impact of the defined perioperative morbidity events over a 6-month follow-up period, using three primary outcome measures over 6 months from the date of enrolment in the study, which was the date of the first index paediatric cardiac surgical procedure:

1. quality of life and psychological burden on children and parents using age-specific measures

2. days at home by 6 months, as an additional measure of disruption to family life

3. NHS costs, including further interventions and hospitalisations, and costs borne by families.

The first two impact outcomes listed are discussed in the current chapter and the third is discussed in Chapter 9.

Follow-up processes

The research nurses at each site followed up each patient to collect data four times over a 6-month period: (1) at (first) discharge from hospital, (2) at 6 weeks, (3) at 3 months and (4) at 6 months after the primary cardiac procedure. If a patient remained in hospital for a prolonged stay after the procedure, the follow-up data were collected whenever possible, although if the child was very sick and required ICU care or the parents were distressed, this was infeasible. In all cases we aimed to undertake follow-up contacts within 2 weeks of the specified time point; if this was not achieved, the data were treated as missing.

Follow-up data were collected from patient families, with the exception of communication data (see Chapter 10), face to face, by telephone or electronically, based on family preference or convenience. Our protocol stated that postal completion should not be used because of the poor response rates. The measures completed by families are listed in Table 17.

Primary outcome: quality of life

Health-related quality of life for cases and controls was assessed using the Pediatric Quality of Life Inventory (PedsQL) 4.0 core scales, which are generic measures for children aged 0–18 years. 175–177 For patients aged < 5 years there are parent-proxy versions only; for those aged > 5 years there are self-completed versions and parent-proxy versions. Normative data exist for all forms of the PedsQL and the measures have been widely used with healthy and ill children, including those with heart disease. 175–177 Although we prefer to collect self-reported data when possible, the majority of patients undergoing surgery were aged < 5 years and in most cases the parent elected to undertake the questionnaire on their child’s behalf, therefore all of our PedsQL data were by parent proxy.

Proxy reporting of a child’s quality of life can be influenced by parental mental health and therefore we measured this using the Patient Health Questionnaire-4 items (PHQ-4),180 to explore this with parents.

A child’s illness and subsequent treatment can also have a broader impact on the family and this was assessed with the PedsQL family impact module. 181 The PedsQL family impact data from our study, although collected, have yet to be analysed and these data are not covered in our report.

These questionnaires typically ask respondents to consider the past 1 month. As we did not want to reflect the early postoperative hospital experience (although we recognise that unfortunately a small number of children do remain in hospital for longer periods), we administered questionnaires at 6 weeks and 6 months post procedure (> 1 month after discharge for most).

Primary outcome: days at home by 6 months

Another objective measure of the impact of morbidity is the number of hospital-free days a child experiences within 6 months of the primary procedure. We collected length of stay data over the 6 months of follow-up from NCHDA,24 which contains hospital (ward) discharge dates, and the PICANet, which contains ICU admission dates. 158 The research nurses further collated all relevant data about hospital stays in secondary as well as specialist centres, accident and emergency department visits and outpatient appointments, and recorded these in the study database. The survival status of all patients was rechecked with sites after the end of the study. Patients who died during the hospitalisation of their surgery scored zero on this scale, to reflect that this is the worst outcome.

Original protocol statement on sample size

We note previous research that stated that a clinically relevant difference in quality of life between pairs corresponds to a mean difference of at least 0.5 SDs. 182 To detect such a difference for PedsQL responses at 5% significance with 80% power requires a minimum of 32 matched pairs. Allowing for a 10% loss to follow-up rate, we aimed to recruit 36 matched pairs for each patient with morbidity. We anticipated that in the incidence study there would be 3000–3300 patients and, therefore, we considered that we would have 80% power to detect a significant effect for any morbidity with a prevalence of at least 1.5%. We noted that based on an analysis of 1 year of cardiac surgery cases from GOSH,6 several major morbidities are anticipated to have lone incidence rates of 1–3% and a multiple morbidity rate of 4–7%. Based on this we estimated that there would be sufficient cases from which to recruit 6–10 sets of 36 matched pairs and around 120 matched pairs for multiple morbidities (a total of 672–960 children, depending on the number of morbidities included).

Statistical analysis

The number of patients and morbidities in the impact cohort

As we have outlined in Chapter 7, we included 3090 procedure episodes (675 with morbidities) in the incidence study. These occurred in 2861 individual patients who had cardiac surgery across the five participating sites. Nested within the incidence study we recruited patients to the impact study, although of note, in Glasgow, we recruited to the impact study for only one-third of the study time because of local logistic problems at that site.

The recorded recruitment rate from eligible participants across all centres was 60% (based on locally held screening logs), and the total number recruited into the impact study was 666 patients, 340 (51%) of whom had at least one morbidity.

Although we recruited 118 patients (18%) with multiple morbidities, which was within the anticipated range, the numbers in many of the single morbidity categories (shown in Table 18) were lower than expected, such that only the most prevalent morbidities, PPE (50 patients) and Feed (45 patients), exceeded the value of 32 cases set out in the sample size calculation as required to detect a 0.5 SD difference in the quality-of-life outcome. The number of patients with ECLS (27 patients), URO (26 patients), Renal (24 patients) and MAE (22 patients) were all close to the target number; however, the number of patients with NEC (11 patients), SSI (11 patients) and ANE (six patients) was much lower owing to the rarity of these morbidities in the base population.

We note that although the total number of patients recruited to both the incidence study and the impact study fell within the projected range stated in the original protocol, there was a lower than expected occurrence of single complications in the study population, leading to the smaller numbers in the categories that related to the rarest morbidities. Given the small number of patients with individual single morbidities, we present the outcome analyses of the impact of morbidity on the quality-of-life outcome by the same broader morbidity groups that we outlined in Chapter 7:

• Two categories: any morbidity compared with no morbidity.

• Four categories: single morbidity (excludes ECLS or ECMO), ECLS or ECMO, multiple morbidity and no morbidity. This outcome enables the discrimination of risk factors for the particularly adverse outcomes of ECLS and multiple morbidities.

For the days at home at 6 months and health economic analyses, which include the deceased patients, we present the analyses of morbidity impact by the 11-category outcome, which incorporates each individual morbidity, multiple morbidity and no morbidity.

Case mix in the impact cohort

The case mix in the impact study cohort is presented in Table 19 by presence or absence of morbidity.

Deceased patients

Twenty patients who died before they could be approached for consent were included based on a truncated pseudoanonymised data set. Of these 20 patients, 19 patients had a morbidity, consisting of three MAEs, eight ECLSs and eight multiple morbidities.

Over the 6-month follow-up period, there were 39 deaths among patients in the impact study cohort. Of these, four had no morbidities and the other 35 had morbidities (one Renal, three MAE, 12 ECLS, two NEC and 17 multiple morbidities).

The PedsQL outcome data

The PedsQL outcome by four-level morbidity groups and individual morbidities

Table 21 shows the case mix-adjusted PedsQL scores categorised by the four morbidity groups and here it is possible to see the more significant effects of ECLS and multiple morbidity on the quality-of-life outcomes; both groups are associated with large differences in quality of life for physical scores, with moderate differences in psychosocial scores at 6 weeks. Although these scores have improved by 6 months, a clinically important reduction on physical quality of life persists at 6 months in the ECLS and multiple morbidities.

TABLE 21 - PedsQL outcome by the four-level morbidity groupings at 6 weeks and 6 months

Multi, multimorbidities.

Model adjusted for all significant risk factors in incidence data (all models have been adjusted for these covariates) and includes multiple imputation for missing values.

Time point, PedsQL score	No morbidity	Single morbidity	ECLS morbidity	Multiple morbidities	Difference:a single vs. none (95% CI), p-value	Difference:a ECLS vs. none (95% CI), p-value	Difference:a multi vs. none (95% CI), p-value
6 weeks
Number	242	145	13	78
Physical score, mean (SD)	79.0 (16.2)	72.4 (21.1)	50.4 (23.2)	66.1 (20.9)	–5.8 (–9.6 to –2.0), < 0.01	–20.9 (–31.9 to –9.9), < 0.001	–11.9 (–16.6 to –7.2), < 0.001
Psychosocial score, mean (SD)	79.4 (14.7)	76.8 (17.3)	65.6 (24.1)	73.5 (20.9)	–1.4 (–5.1 to 2.2), 0.43	–4.9 (–15.0 to 5.2), 0.34	–4.9 (–9.3 to –0.4), 0.03
Total score, mean (SD)	79.3 (13.8)	74.5 (18.3)	59.4 (22.0)	69.6 (19.3)	–3.4 (–6.8 to 0.02), 0.05	–11.9 (–21.2 to –2.7), 0.01	–8.0 (–12.1 to –4.0), < 0.001
6 months
Number	208	125	10	61
Physical score, mean (SD)	82.3 (16.6)	79.2 (16.8)	68.4 (28.0)	72.6 (19.6)	–2.3 (–6.0 to 1.5), 0.24	–12.0 (–25.3 to –1.3), 0.08	–7.3 (–12.1 to –2.5), < 0.01
Psychosocial score, mean (SD)	78.0 (14.7)	76.9 (14.1)	78.9 (18.8)	74.9 (17.4)	–0.3 (–3.1 to 2.6), 0.86	1.7 (–10.4 to 10.0), 0.97	–2.2 (–6.4 to 1.9), 0.29
Total score, mean (SD)	79.8 (13.9)	78.1 (14.0)	74.6 (21.4)	74.0 (15.8)	–1.1 (–3.9 to 1.7), 0.44	–5.2 (–15.7 to 5.4), 0.34	–4.4 (–8.2 to –0.5), 0.03

Although, as stated, we did not undertake statistical analyses of quality-of-life outcomes by individual morbidity, the values for these measures are shown in Figure 6, in which we can see that ECLS, PPE and multiple morbidity have the lowest scores at 6 weeks, particularly involving physical domains. By 6 months, quality-of-life scores are much improved for all, although physical scores in the ECLS group remain impaired.

FIGURE 6.

PedsQL total scores for each morbidity at 6 weeks and 6 months. Multi, multimorbidities.

The PHQ-4 outcome

The PHQ-4 results in Tables 23 and 24 are presented based on the collapsed score based on two levels of either anxiety or depression. Table 23 shows that presence of any morbidity increased the level of anxiety and depression at 6 weeks, and there is still an effect at 6 months, but it is smaller and not statistically significant.

The analysis of PHQ-4 results based on the four-level morbidity grouping (Table 24) enables us to see that the impact in terms of anxiety and depression is more profound for parents of children with ECLS and multiple morbidities, as compared with other single morbidities. Although this has improved by 6 months, there is still some suggestion that scores remain higher, especially for depression.

Days at home by 6 months

One of the advantages of this outcome measure was that we were able to include 662 patients in the analysis and only four were dropped because of missing data that were related to hospital stays. This measure (defined in Methods) included the 39 deceased patients, the majority of whom died within their primary hospitalisation and scored zero for the measure.

Based on the analysis using quantile regression to predict the median days at home over 6 months, adjusting for all of the stated covariates and accounting for clustering within matched pairs we found that presence of any morbidity reduced the median days at home by 13 days (95% CI 10.2 to 15.8 days; p < 0.001). Considering the four-category outcome, a single morbidity compared with no morbidity reduced days at home by 7.2 days (95% CI 5.1 to 9.3 days; p < 0.001), ECLS had a very significant impact in comparison to no morbidity, with a median reduction in days at home of 114.7 days (95% CI 76.4 to 153.1 days; p < 0.001), and there was a moderate reduction for multiple morbidity compared with single morbidity of 21.9 days (95% CI 15.6 to 28.1 days; p < 0.001).

The relationship between the presence of each of the individual morbidities and the outcome of days at home by 6 months is shown in Table 25, in which we see that most morbidities, apart from renal support, had a statistically significant impact. The distribution of the days at home by each morbidity is shown in Figure 7.

FIGURE 7.

Days at home by 6 months, by individual morbidities. Multi, multimorbidities.

Patients

The sample for this analysis comprised the 666 patients recruited into the impact study described in Chapter 8.

Resource use and costs

Health-related quality of life and quality-adjusted life-years

Statistical analysis

Descriptive statistics

The demographics and case mix characteristics of the impact study cohort, as well as the morbidities among the impact study cohort, are described in full in Chapter 8.

Summary statistics for the outcome measures are in Table 27. Descriptive data on the outcome measures used in economic modelling are shown in full in Appendix 14. In terms of our primary outcome measures, data on days in ICU during index hospitalisation (outcome measure 1 in Table 26) and cost of days in ICU during index hospitalisation (outcome measure 10) were available for 645 patients (97% of the sample), and the mean (SD) and median (IQR) values were 9.7 (SD 15.7) days and 5 (IQR 3–9) days and £20,058 (SD £33,257) and £9800 (IQR £5475–18,656), respectively. Total hospital days (outcome measure 9) were available for 645 patients (97%), with mean (SD) and median (IQR) values of 23.8 (SD 29.5) days and 13 (IQR 8–22) days. Total cost of hospital stays during 6 months’ follow-up (outcome measure 23) was available for 593 patients (89%), with mean (SD) and median (IQR) values of £39,916 (SD £47,437) and £23,028 (IQR £15,117–40,439). The distribution of both these cost variables was positively skewed (Figure 8).

FIGURE 8.

Distribution of total cost of hospitalisation during 6 months’ follow-up.

Comparing mean and median values between the outcome measures in Table 27, the variables that contributed most to the total societal cost are hospital costs, more specifically the cost of days in the ICU and days on the ward during the index hospitalisation.

Costs are in 2016/17 UK pounds sterling. The ‘Missing’ column (see Table 27) shows the amount of missing data for each variable included in the analysis across the 666 patients in the sample (627 in the data for survivors only). Values may not sum across variables due to different numbers of observations.

Note that several of the non-derived resource use/cost measures (number of prescriptions during 6 months’ follow-up, cost of primary care contacts during 6 months’ follow-up, family costs incurred during 6 months’ follow-up, days off work during 6 months’ follow-up) had in excess of 50% missing data, sometimes in excess of 90%. In addition, based on the [mean (median)] values of the cost per patient of prescriptions during 6 months’ follow-up [£257 (£224)], primary care contacts during 6 months’ follow-up [£448 (£248)] and family costs incurred during 6 months’ follow-up [£826 (£255)], each of these cost components is likely to make a small contribution to the overall societal costs of morbidities {e.g. balanced against a total cost of hospital stays [£39,916 (£23,028)]}. Given the high degree of missingness for these variables, plus the small contribution that they make to total societal costs, we elected not to analyse them further in regression analyses.

Mean and median QALYs at 6 months (all patients) were 0.327 (SD 0.152) and 0.399 (IQR 0.310–0.430), respectively, in the 194 patients for whom these data were available (29%). The distribution was bimodal, with values clustered around 0 and the maximum achievable value (0.442; Figure 9). For survivors only, mean and median QALYs at 6 months were 0.397 (SD 0.051) and 0.410 (IQR 0.370–0.436), respectively.

FIGURE 9.

Distribution of QALYs at 6 months (all patients).

The imputed variables of the outcome measures had broadly similar descriptive statistics to the original versions with missing data (see Appendix 15).

Regression results

Regression results for our primary outcome measures are shown in Tables 28–30. Regression results for the full list of economic outcomes are reported in Appendix 16. Each table presents the association with the morbidity variables for each outcome measure. Controlling for the covariates, on average, URO, Feed and ECLS each resulted in significantly more ICU days during the index stay (on average 3, 4 and 23 more days, respectively). These findings also translated into higher ICU costs during the index admission (which were a function of both the number of days in ICU and the level of care at each of those days) and those with multimorbidities incurred around £24,071 higher ICU costs. Total ICU days and costs for the 6-month follow-up period were higher for ECLS patients (25 days and £54,155) than for patients with multimorbidities (15 days and £26,389).

Patients with ECLS had the highest additional hospital days and costs (around 33 more days and £71,051 higher costs); other morbidities with significantly higher days or costs were URO, Feed, Renal, NEC and PPE. Patients with multimorbidities also incurred significantly higher hospital days and hospitalisation costs compared with patients with no morbidity (25 days and £41,484 higher, respectively).

Extracorporeal life support across all patients, and ECLS and multimorbidities for survivors only, were associated with significantly lower QALYs over the 6-month period than patients with no morbidity (see Table 4). Across all patients, the QALY decrement associated with ECLS was –0.095; among survivors only, it was –0.051. Among survivors only, multimorbidity produced a QALY decrement of –0.028. None of the other single morbidities was associated with a significant reduction in QALYs compared with patients with no morbidity and conditional on the covariates. Utility scores are presented in full in Appendix 15.

The trends shown with respect to our primary outcomes were borne out with the other outcomes included in the regression analysis (see Appendix 16). We report the marginal effects for each morbidity and whether or not the coefficient was significantly different from zero. Note that the marginal effects do not necessarily sum across the resource use and cost measures because we used two different regression models (GLM and OLS where necessary) and the GLM model is non-linear. For nearly every resource use and cost outcome measure, multimorbidities were associated with significantly higher values than that for patients with no morbidity. Individual morbidities that were commonly associated with higher resource use and costs were URO, ECLS, PPE and Feed. Where it had a statistically significant effect, ECLS was generally the morbidity category that had the highest resource use or cost impact, usually with a larger effect than multimorbidities. ECLS and multimorbidities were associated with utility decrements at 6 weeks (ECLS) and 3 months (multimorbidities) compared with patients with no morbidity, reflecting their impact on QALYs shown above.

Selection of the questions about communication

As part of the broader project, a panel of clinicians and parent representatives selected morbidities to measure,90 which were then defined by a separate panel of professionals. 98 The panels included a substudy to measure communication between parents and the clinical team. The definition panel recommended using a subset of questions from the Picker Survey,107 as those questions were developed using focus groups and formally validated. The Picker Institute assisted us in identifying of a shortlist of six questions to ask parents about communication, as described in Brown et al. ,98 and issued the research team with a licence to allow our study to use these questions for patients recruited to a 6-month follow-up substudy as part of the larger project.

The final questions were:

• Question 1: did new members of staff treating your child introduce themselves?

• Question 2: were you encouraged to be involved in decisions about your child’s care and treatment?

• Question 3: were you told different things by different people, which left you feeling confused?

• Question 4: were the different members of staff caring for and treating your child aware of their medical history?

• Question 5: before the operation or procedure, did a member of staff explain to you what would be done during the operation or procedure?

• Question 6: did a member of staff tell you what to do or who to talk to if you were worried about your child when you got home?

Population and administering the survey

Our study prospectively monitored all children aged ≤ 16 years who had cardiac surgery within the five participating hospitals between 1 October 2015 and 30 June 2017 for the presence of nine selected important postoperative morbidities. Families of children with these morbidities and case-matched controls without morbidity were then approached for consent to participate in a 6-month follow-up programme. All families who consented to participate were given the communication questionnaire either at discharge or shortly after discharge, along with a stamped addressed envelope. Parents were asked to complete and return it in a de-identified format within 6 weeks of their child’s operation. For staff resource reasons, only four of the sites participated in the communication survey.

Parents were assured that neither the clinical team caring for their child nor the research team would be able to see their responses. All responses were entered by a separate data entry clerk who only had access to the child’s anonymised study identification. Access to the communication survey data entry screen was password protected to prevent clinician or research team access.

Statistical analysis

For each question we transformed responses into binary responses, for which yes was defined as the ‘always positive’ response (Figure 10).

FIGURE 10.

Proportion of positive responses to each of the six questions for the national Picker Survey107 (black dots), control patients (green dots) and case patients (blue dots) from our study. Exact 95% CIs are shown for the proportions measured in our study. Q, question.

The proportion of positive responses for each question was compared with the corresponding overall proportion reported in the Picker Survey. 107 We performed two-sided binomial tests for difference at the 5% level for all control patients (no morbidity) and separately for case patients (with at least one morbidity) to the national Picker Survey proportion. To adjust for multiple hypothesis testing, we applied a Bonferroni correction201 and required p < 0.05/12 = 0.004 for significance at the 5% level.

We explored whether or not different factors affected the proportion of positive responses using multiple logistic regression. The candidate explanatory variables were hospital site, case versus control, black or minority ethnicity, log age at procedure, and log length of stay. We used the natural logarithm of length of stay because the distributions were highly skewed (skew > 2, kurtosis > 12). Additionally, the impact of age on outcome is highly non-linear. 162

Data were analysed using Stata software.

Comparison with the national inpatient survey

The proportion of positive responses to each of the six questions from parents of patients in the study is shown in Figure 10, along with the corresponding proportion from the national Picker Survey. There were no significant differences in the proportion of positive responses compared with the Picker Survey for questions 1, 5 and 6. Compared with the Picker Survey, parents of children with at least one morbidity were less likely to say that involvement in decision-making was always encouraged (question 2, p < 0.001), whereas parents of children with none of the measured morbidities were more likely to say that professionals were aware of their child’s medical history (question 4, p = 0.002). All parents in our study were less likely to say they were never told different things by different people (question 3, p < 0.001), with parents of case patients reporting lower rates than parents of controls (31% vs. 44%, 68% nationally).

Multivariable analysis of explanatory factors on responses

None of the explanatory factors was associated with responses to questions 5 and 6. Hospital site affected responses to questions 1–4, with site A having consistently higher proportions of positive responses than the other sites. Parents of younger children were more likely to report positively that they were involved in decision-making (p = 0.045) and on staff always being aware of their child’s medical history (p = 0.041). However, parents of children who stayed longer in hospital were less likely to report that they were involved in decision-making (p = 0.013) and less likely to say that they were ‘never told different things by different people’ (p = 0.036). Parents of children who experienced at least one morbidity were less likely to say that staff were aware of their child’s medical history (p = 0.019). Black or minority ethnicity status was not significantly associated with responses to any of the questions.

We illustrate the differences in responses by each factor for questions 1–4 in Figure 11. Note that although the regressions were run using continuous log age and log length of stay, we give results in illustrative categories for easier interpretation. We only show proportions when there was a significant association of that factor with responses for that question under a multivariate analysis.

FIGURE 11.

Association of each of the explanatory factors with the first four questions. (a) Site; (b) illustrative age bands; (c) illustrative length of stay bands; and (d) case vs. control. Q, question. Reproduced from Pagel et al. 194 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/. This figure includes minor additions and formatting changes to the original.

Reproduced from Pagel et al. 194 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/. This figure includes minor additions and formatting changes to the original.

None of the factors was associated with responses to questions 5 and 6. We only show proportions for questions when there was a significant association between the factor and responses in a multivariate analysis. Proportions are shown for categorical age and length of stay bands for ease of interpretation but the analysis was carried out using continuous variables.

Initial design of icons and data summaries

As an initial step, we designed a set of icons intended to represent the complications in data summaries. For complications affecting specific sites in the body (brain, kidney, bowel, pleural space, surgical wound) we adapted widely used icons of that site. For events (URO, MAE) and interventions (ECLS) we aimed to convey the essential characteristics of the complication. For Feed, we initially used safety iconography of a red circle with a bar across to indicate ‘nil by mouth’.

We then constructed basic data displays incorporating the icons to present hypothetical data on the counts and proportion of cases having each complication (in isolation, in combination with at least one of the other selected complications and in total). Displays were also constructed to display combinations of complication at the level of individual patients.

At this point we visited several of the surgical centres involved in the study to discuss with data managers and available clinicians if and how they envisaged routine monitoring of the measured complications being incorporated into their quality assurance processes. During each meeting we discussed local initiatives and practice concerning the monitoring and feedback of early morbidities or complications; recognition of the icons developed; the team’s responses to our proposed data summaries; and ideas for other data summaries that would be found useful.

Feedback from these discussions and from presentations to the Study Steering Group informed the redesign of icons, where necessary, and informed the functional specification of the prototype software tool developed.

Prototype software tool development

Based on previous experience of developing an outcome monitoring tool for UK centres performing paediatric cardiac surgery and on the software tools currently used by sites to collate, analyse and present data, we decided to build a prototype tool within Microsoft Office, specifically an Excel^® spreadsheet (Microsoft Corporation, Redmond, WA, USA) application that could be used to generate a PowerPoint^® presentation file (Microsoft Corporation, Redmond, WA, USA) containing graphical data summaries.

Icons

The final set of icons developed for use in graphical summaries of complication data are shown in Figure 12. Feedback from clinicians and data managers at the participating sites indicated that the icons developed were, generally, readily associated with the complications that they were designed to represent.

FIGURE 12.

Final set of morbidity icons. Reproduced from Grieco et al. 202 © Cambridge University Press 2019. This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/.

Reproduced from Grieco et al. 202 © Cambridge University Press 2019. This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/.

Changes made to the initial designs to incorporate feedback were:

• replacing the ‘nil by mouth’ icon based on the international prohibition sign of a barred red circle with a drawing of an infant with a nasogastric tube (feedback was that the prohibition sign could be interpreted as the clinical team saying the child was not allowed to feed, rather than the child having difficulty feeding)

• redrawing the bloodbag component of the icon for MAE to avoid it looking like a syringe

• recolouring the patient depicted in the icon representing ECLS to reflect feedback that the clinical experience (and indeed intent) was of children on ECLS being notably pink.

The proposed summary displays of complications data were welcomed by data managers and clinicians, albeit with feedback and suggestions for improvement and development. Clinicians stressed the importance of expressing the incidence of complications as a percentage of operations performed, as well as in terms of absolute counts. It was requested that, once the data from the ongoing study were collated, there was some form of benchmarking to put local complication data in the context of data from multiple sites, acknowledging that such benchmarking would not, initially, take account of case mix differences between sites.

Concerning the incidence of different combinations of complications, there was interest in exploring the sequence of complications in individual patients. Although we recognised the clinical motivation for this request and created some ‘mock-ups’ of how such displays would look (Figure 13), reflection on how the morbidities were defined led us to conclude that the sequence in which complications were recorded would not reliably reflect the sequence of clinical events, undermining the usefulness of sequence information.

FIGURE 13.

Example of representation of all possible sequences of complications (not based on actual data). Reproduced from Grieco et al. 202 © Cambridge University Press 2019. This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/.

Reproduced from Grieco et al. 202 © Cambridge University Press 2019. This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/.

There was a degree of interest in building subgroup analyses into the prototype displays, with a particular focus on the potential for monitoring complication rates among specific complex diagnoses (e.g. those associated with a functionally univentricular heart) and patient groups (e.g. neonates). Although we understood the value of routine monitoring of complication data to allow for subgroup analysis in time, we took the view that different subgroups might be of interest at different times at different centres, depending on local quality improvement initiatives for example, and so should not be hard-wired into the prototype tool.

Our initial ideas for presenting changing rates of morbidity over time were viewed as being complicated, which led us to incorporate, instead, time series displays employing the same formalisms as used in routine monitoring of mortality in UK centres. The following sections summarise the structure and content of the prototype tool designed incorporating this set of feedback and suggestions.

Structure of automated presentation and example slides/charts

Figure 14 depicts the structure of the presentation automatically created using our tool, as well as the flow of navigation enabled by the action buttons.

FIGURE 14.

Structure of output presentation.

Development of the time series approach, building on widely used and accepted mortality variable life-adjusted display charts

Variable life-adjusted display charts are simple graphs providing an intuitive representation of the occurrence of a given clinical outcome over time, measured against a baseline risk. They were originally proposed as a way to indicate whether a surgeon’s outcomes were better or worse than might be expected based on the case mix of their practice (difference between expected ad actual cumulative mortality). 208

We adapted VLAD charts to measure the occurrence of a given morbidity over time being above or below a given benchmark (or baseline risk b). In the chart, every procedure is plotted from left to right (in temporal order) on a horizontal axis:

• If the procedure is not associated with the morbidity, the line moves up by an amount equal to the risk of occurrence of that morbidity (i.e. b).

• If the procedure is associated with the morbidity, the line moves down by an amount equal to the chance of that morbidity not occurring (i.e. 1 – b).

In the example of VLAD chart shown in Figure 15, ANE morbidity occurs less frequently than in the compared benchmark (baseline incidence) and its incidence seems to globally decrease over time.

FIGURE 15.

Example of VLAD-style chart.

Excel tool

We developed a tool for the automatic creation of a document containing a structured set of the graphical summaries described above. The tool was developed by embedding visual basic for applications programming code into an Excel spreadsheet storing source data about morbidities. Source data consist of a list of procedures (one procedure per row in the spreadsheet) along with procedure date, current life status and diagnosed morbidities (i.e. yes/no for each of the morbidities considered in this study). Benchmark risks for each morbidity can also be specified (default values are provided) as an input to generate temporal summary graphs. A ‘Run’ button enables the automatic creation of a PowerPoint presentation as a separate file, incorporating action buttons to facilitate navigation.

From the home page, users can access a set of slides summarising the number of morbidities reported in the source data. Icons are reported in a circular layout and labelled with numbers or percentages (the user can switch between the two types of visualisation), representing morbidity occurrences. Navigation buttons also allow to switch between summary considering procedures with (1) any number of morbidities; (2) exactly one morbidity; or (3) with at least two comorbidities. Information on deaths and on procedures without any complication is also reported.

A button ‘Temporal summary’ gives access to a slide reporting VLAD charts for all morbidities, for which users can have a quick overview of the temporal trend of each morbidity across a time window covering the source data.

Finally, from the ‘Procedures with multiple morbidities’ slide, users can click on any morbidity icon to access a slide summarising how many times the morbidity co-occurred with each of the other morbidities across all procedures. Examples of the slides in the output are included in Appendix 17.

Responses and feedback on materials

Participants preferred option 1 to the others, commenting that option 3 might encourage people to look for explanations. One parent commented that ‘It looks like you’re employing a bad surgeon on those particular children’.

The participants suggested starting with option 1, but then providing the information on the separate morbidities as additional information.

There was an important discussion led by participants and the PPI co-applicant around normalising morbidities. Participants preferred wording that said ‘18 out of 100 children experience at least one problem’, as they found it reassuring to know that it was not unique to their child. One parent commented:

. . . it’s nice to know that, you know, this isn’t just a one-off, and, ‘Oh my gosh, my baby’s got this infection, and no one knows what it is, and no one knows what to do with it’. If it’s already identified, and the hospital already have experience in making it better, then that’s comforting information, almost. It makes it less scary, in a way.

We also asked participants about the language of ‘morbidity’ versus ‘complication’ versus ‘problem’ and, after some discussion, the word ‘complication’ was preferred.

We went on to show similar diagrams for children with different categories of risk: older children with the least complex heart conditions (incidence ≁ 3%) and young babies with the most complex conditions (incidence ≈ 33%). Participants thought that it was important information, but preferred the presentation without the detail (option 4 preferred to option 5 in Figure 18), as the full information looked too overwhelming. Again, normalisation was an important theme.

FIGURE 18.

Incidence options for young babies with serious heart defects.

I think it would be useful to know not necessarily the gory details, but . . . it’s almost saying, ‘We’ll cross each bridge as it comes’, but there is a chance, even if it’s effectively a 33% chance, that something could happen, then you don’t feel so alone.

We also asked participants about how and whether or not to include information about increased risk of death for different morbidities within the displays. Somewhat to our surprise, the feedback was strongly against including mortality information on the grounds that it was a qualitatively different outcome to morbidity and it was already something that was discussed separately during consent and planning for the surgery. Some comments about mortality versus morbidity were:

It’s almost at a different, kind of, level of information. It’s a different level of anxiety.

It’s incurable. I mean, if a child dies, that’s it. To me, this is mostly, ‘We can cure this, and we have a plan for it’.

It’s not that you’re not told. When you sign that surgery information . . .

The final piece we asked parents about was representing the potential impact of experience of a morbidity in terms of length of stay in hospital, as our preliminary data showed that children who experienced morbidity were significantly more likely to stay longer in hospital. Figure 19 shows one of the examples we showed people, with the key to reading the graphic at the top. We chose to show the IQR and no point estimate (like a mean or a median) to communicate how variable length of stay could be.

FIGURE 19.

Examples of how morbidity can affect length of stay. Green icons indicate none of the nine morbidities and yellow icons indicate at least one morbidity.

The participants thought that the information was very useful and that they would have liked to have seen this before their child was admitted for surgery:

So, to know what was going to possibly happen in the weeks leading after the birth was important to me, even down to, like, how much I pack, and what arrangements I make, you know, for the cat, and things like that . . .

. . . if there are other siblings and things, so that families can look and think, ‘Hopefully I’ll be out in 16 days, but what happens if we’re not? How do we cope with . . . ,’ ‘Looking at the maths for 20+ days, what do we do, childcare or whatever?’ It enables you to be . . . , that whole thing about being prepared for it.

We discussed providing more detailed information for each of the individual morbidities, as there was a wide variation in length of stay depending on the morbidity experienced. Participants felt that information on all of the morbidities would be too overwhelming, especially as some of them were very rare. The consensus was that if a morbidity was experienced, then more specific information from this study about that morbidity could be shared, similar to the existing information already shared with parents about ECLS.

Participants thought that the range was useful, to normalise the situation both if a child stays longer than the initially anticipated time period and if the child stays for less time.

Another important insight that emerged from the discussion was the benefit in having more information to enable discussion and support with other parents going through similar situations on the ICU:

I think, to get a whole better understanding . . . , also, in the expressing room, it was such a little support network to each other. So, to have a bit more of an understanding, to be able to talk to each other, with better education, I suppose, would have been good.

Similarly, participants thought that it would be useful to have this information in writing to share with family, employers or schools, so that they knew better what to expect and that heart surgery was not a simple ‘quick fix’.

We discussed how and whether or not to add in further information about morbidities once the impact follow-up data had been analysed. Participants thought that it might be useful if your child experienced one of the morbidities, but that information would have to be accompanied with advice on what to do next or signposting to resources that would help. In a similar vein, we were told that we needed to ensure that there was sufficient narrative to accompany the graphics in any leaflet, to ensure that there was context and trustworthy information about where these data had come from and what they meant.

The key points that we took away from the first workshop and used to prepare a draft information leaflet were:

• the importance of normalising morbidities and emphasising that hospitals are used to dealing with them

• layering information

• not combining morbidity with mortality information.

Responses and feedback

Participants were largely positive about the text, the explanation of the morbidities and the graphical displays and found the displays intuitive to interpret. However, we received valuable feedback on improvements, which included:

• Fewer graphics for incidence – parents felt that only one or two examples additional to the overall incidence were needed, otherwise the sheer number was overwhelming.

• Participants thought that the length of stay information would be useful for everyone, whereas some parents might not want to see the incidence data, and so they suggested putting this information before the incidence data.

• We were told that navigating the document was difficult and we should add a clearer overview of the content to the beginning.

• Parents asked if we could signpost readers to other resources and charities that would be helpful.

• Parents suggested adding a sentence about plans for your child changing during the hospital stay, to prepare families better for potential uncertainty.

Additionally, when asked about adding impact data once they were, available, participants suggested focusing on practical information (e.g. time out of work, time in hospital) rather than clinical information. We were also told that parents might not read the information immediately, but that having a leaflet printout to hand might be invaluable later.

Perspectives of parents and health professionals

Our research study sought to integrate the viewpoints of parents and carers via the use of focus groups, an online discussion forum run by a user group, by engaging with parent advisors (LA and BT) during the course of the project about the protocol and the analysis of study data, and finally by undertaking workshops with parents to co-develop information resources.

Reflecting on these activities and noting the sensitivity of the topic, we suggest that although there is much common ground between parents, carers and health-care professionals, all of whom want to see the best outcome for the child, the range of focus for parents and carers differs somewhat to that of clinicians. This leads to some divergence between the two groups about the fundamental issues of what the important morbidities linked to paediatric cardiac surgery are: clinicians tended to prioritise clearly clinical issues related to the heart (use of ECLS and reoperation), whereas parents placed greater emphasis on holistic outcomes for their child (feeding, child development and communication). During the consent process and the postoperative periods, the highly skilled practitioners involved in paediatric cardiac surgery understandably have a particularly intense level of attention on the conduct and success of the procedure itself. Although parents and carers obviously share this priority, given their role as primary carers they wish to know more about what their child will be doing afterwards in the medium and longer term: clearly both areas are very important.

Information resources

We used study data collected during the course of the morbidity project to co-develop parent and carer information resources. To date, this has been focused predominantly on showing what the morbidities are and how their incidence and the length of stay may vary based on the complexity of the child’s condition. For example, a newborn baby with a very complex heart defect has a one in three chance of experiencing a complication, whereas an older child with a less complex heart defect has a 1 in 20 chance of a complication. Parents emphasised consistently that the default position should be to offer the information during the consent process, even though not every parent may wish to accept and read it immediately. They told us that it helps parents to know first that, they are not alone in facing a complication; second, that clinical teams have seen complications before and know how to deal with them; and third, that it is better as a parent to ‘be prepared’. Furthermore, they indicated that information about impact, such as nearly all children who experience a complication and recover, will have a similar quality of life to children who did not experience a complication by the 6-month mark, was very useful to know. We believe that the developed information resources represent a major step forwards; however, they would be optimally backed up by an evaluation study and a plan for their dissemination into general use.

Morbidities tell us more about outcome than mortality

The main measure used to evaluate outcome of paediatric cardiac surgery remains 30-day survival, which is very high (above 98% in the UK for 2016/17) and is now not discriminatory enough as an outcome measure for teams that wish to explore new routes to improve patient care.

We have identified nine measures of morbidity: ANE, URO, Feed, Renal, MAEs, ECLS, NEC, SSI and PPE. Younger children and those with the most complex CHD were at greatest risk of getting morbidity, as were children with prolonged cardiopulmonary bypass times. Patients with any morbidity present had mortality at 6 months of 12.5% and a median perioperative length of stay of 24 days. We note that the length of stay and mortality outcomes within the population of patients who did not experience any morbidities were excellent: their survival at 6 months post operation was 99.1% and their median length of stay was 8 days, and this leads us to believe that the morbidities we selected do capture most of the complication-related adverse outcomes for this context.

Patients with any morbidity scored, on average, 5.2 points lower on their total quality-of-life score at 6 weeks, but this difference had narrowed to only 2 points less by 6 months. At the 6 months mark, most types of morbidity had comparable quality-of-life scores to patients without morbidity, except for patients who had been treated with ECLS. Most of the individual morbidities were linked to a greater number of days in hospital and higher hospital costs over 6 months, but ECLS patients had the fewest days at home at 6 months (median 43 days out of 183) and highest costs (£71,051 higher costs), followed by patients with multimorbidities (147 days and £41,484 higher costs).

How can morbidity monitoring be taken forward?

There is evidence that stakeholders are interested in taking forward the use of morbidity measures for quality improvement, given that selected clinical teams around the country have begun to share other quality markers, such as accidental extubation rates and hospital-acquired infection rates locally. We note the success of the multicentre collaborative (PC⁴) reporting morbidity outcomes from a North American setting, albeit focused on clinician-selected metrics. Therefore, looking forward, there is the potential for use of our study findings at a local and at a national level. It is particularly important to consider future research that has the potential to reduce the rate of morbidity or to ameliorate its effect on patients.

Neurological and developmental surveillance and follow-up

We note that in the selection work, neurodevelopmental morbidities came up as the most important morbidity affecting children with CHD across all stakeholders, both clinicians and lay people. Nonetheless, the definition that was arrived at for the morbidity of ANE, which captured major early acute neurological complications after surgery, had a rate of 0.5% as a single event and 2.1% in combination with other complications, in the incidence study. This low ANE rate may be reviewed in the context of the BDA validation study in which we assessed a representative sample of children with CHD using validated neurodevelopmental tests (against which the BDA was compared). In the sample of 400 children aged between 15 months and 5 years (selecting an age band where developmental tests are more reliable rather than a very young age band where they are less reliable), we found that the Mullen result was more than 2 SDs below the normative mean in between 17% and 21% (depending on exact inclusion criteria). The difference between the rate of ANE and the rate of very low scores in children with CHD when assessed in a different context is stark and suggests that ANE represents the ‘tip of the iceberg’.

Since undertaking the BDA validation study, we have completed a Delphi survey in three rounds to establish consensus as to the process for evaluation of suspected neurodevelopmental abnormalities in children with CHD. We have also undertaken an assessment of the services that children with CHD were under within the validation sample, asking the research question ‘how many children with low scores on gold standard testing were under appropriate child development services?’ These secondary follow-on projects are currently being analysed for publication. We suggest that there is a pressing need to take this topic forward, with changes to the follow-up pathway for neurodevelopment in children who have CHD. We suggest that further research is needed to explore at what time points in the patient pathway should the BDA be used, what actions should be taken when the BDA is amber or red and whether or not use of the BDA is effective.