The management of Otitis Media with Effusion in children with cleft palate (mOMEnt): a feasibility study and economic evaluation

2.1 Does your cleft service have dedicated audiology input based at your centre?

The majority of centres (9/10) completing this question indicated that they had access to a named health-care professional who undertook age-appropriate hearing testing in children with CP.

2.4 How often do children with cleft palate receive routine audiological assessment at your cleft centre? If assessment varies by age please give frequency of routine audiological assessment and age ranges

Details received would suggest that although testing regimes vary, most children undergo at least four hearing assessments by the age of 5 years (in addition to Universal Newborn Hearing Screening). Children may be seen more regularly, depending on clinical need.

2.6 At primary cleft palate repair, how is the decision made to insert ventilation tubes or not, and who is involved in the decision-making process?

The responses received indicate that it is not standard practice in the centres surveyed to sanction the insertion of VTs at primary cleft repair.

2.8 After what period of time would a conductive hearing loss (> 25–30 dBHL) trigger ‘active’ intervention (referral for/decision to insert ventilation tubes or prescribe hearing aids) at your centre?

Most centres would need evidence of persistence of OME over at least a 3-month period to recommend HAs or VTs. The response to this question would suggest adherence to the recommendations contained in NICE CG6011 regarding a 3-month period of ‘watchful waiting’ prior to making a decision to recommend an intervention for OME.

2.9 Please describe the decision-making process to provide hearing aids or to insert/refer to ear, nose and throat for consideration of ventilation tubes as the first-line treatment for persistent otitis media with effusion. Please include any involvement of parents and/or the child

The responses indicated that patient choice is an important factor in the decision-making process, again adhering to NICE CG60,11 which recommends that ‘treatment and care should take into account children’s needs and preferences together with those of their parents or carers’.

2.5 What tests do you routinely use to diagnose and guide the subsequent management of otitis media with effusion?

All centres had access to a comprehensive set of audiological tests that would enable accurate assessment of frequency-specific hearing thresholds from approximately 6 months of age (visual reinforced audiometry 6 months to 3 years) through to adulthood (play audiometry 3–5 years, pure tone audiometry 4–5 years+). Tests were also available in certain centres that would enable threshold assessment in children < 6 months of age (auditory brainstem response) and assessment of other aspects of hearing, including the perception of speech.

This would indicate that all of the centres replying to the clinician survey were in a position to participate in a subsequent study to determine the most effective treatment for OME in children with CP.

2.13 What is your view on the optimum age for inserting ventilation tubes?

There was a spread of ages given in the responses received, with several centres indicating that clinical need was the more important determinant (Table 3).

The results for this question should be interpreted cautiously, as the wording of some answers suggested that the question asked for a minimum age for inserting VTs, as opposed to an optimum age. Although there was variance in practice, the majority of centres considered that the decision to insert VTs was either influenced by clinical need and not age (6/13 centres), or 1–4 years (4/13 centres). Only 3 out of 13 centres stated that the optimum age to insert VTs was under 1 year or over 5 years. It is likely that a subsequent study would include children (> 1 year old) of nursery, pre-school and school age. Therefore, with respect to age at VT insertion, the majority of centres would not be required to agree to a significant change in clinical practice.

Parents’ willingness to enter their child into a trial comparing ventilation tubes and hearing aids

In nine parent interviews, participants stated that they would allow their child to be part of the trial, whereas in 19 the answer was negative, and in eight, participants were unsure what they would do. Table 9 groups interviewees based on their response to taking part in a trial (‘no’, ‘unsure’, ‘yes’) and summarises key factors influencing their decision-making as recorded in interview transcripts. Patterns which emerged on how individuals responded are shown in Table 10.

Most interviewees were reticent about entering their child into the proposed trial, although they recognised the need to advance scientific knowledge. Some of their reluctance stemmed from concerns about not being able to choose treatment and a risk of their child not being allocated to the most effective arm. Furthermore, several parents had pre-existing views about the benefits or drawbacks of VTs or HAs, as suggested in the following interview extracts:

Urm, possibly not, just because if she then fell in the grommets group, she would have to have an operation and I wouldn’t want her to go under general again just for grommets. So probably not.

P7 (m)

I’ve had the results and I’ve witnessed it and he was a changed child. He could hear perfectly well. I mean he went for his hearing test after his grommets and he was passing them with flying colours . . . So no I wouldn’t be happy with that and I wouldn’t want a hearing aid because it’s there, it’s on view, children will poke at it and say ‘what’s that in your ear?’ and it’s the sheer embarrassment for a child, I would say no, absolutely no way.

P8 (m)

Hence, interviewees saying ‘no’ or ‘unsure’ did not see the two treatments as sufficiently equivalent to accept randomisation and did not appear to feel that risks associated with one intervention were warranted. Some parents expressed fears that there could be social consequences of HAs, including the potential for bullying:

Just the stigma, the stigma with hearing aids isn’t it. It’s, you don’t want anything that anyone can say to, to the chances of your child being bullied, being picked on and you know yourself it’s like kids and it’s just one more that they can, he’s not a very confident child, you know you worry that he, that’s why we were concerned whether his speech, would that cause bullying and stuff like that. It’s just a stigma isn’t it really, it’s like anything.

P16 (m)

Others had strong views about physical risks that they associated with VTs (e.g. causing damage to the ear or infections):

I think she was about 1 when she had her first set of grommets that they said would help sort out the glue ear and that unfortunately made things worse . . . I think the hearing aids have actually helped more than the grommets ever did because we haven’t had an ear infection for over a year . . . They [VTs] never worked. She’d be ill, she’d be off school. We’d get phone calls saying she wasn’t in but it wasn’t our fault.

P24 (m)

Participants were divided in terms of their favoured treatment, including parents of children in the watchful waiting group (Table 11). Hence, even when interviewees had no personal experience of VTs or HAs, they could still hold strong opinions about these approaches, informed by conversations with friends or relatives, media outlets or social norms. Parents with children who had received VTs only tended to prefer this approach. Just one person in this group expressed a preference for HAs; P29 did not think it was fair to put her child under anaesthetic when the VTs kept falling out. Most parents with a child who had experienced HAs only described being encouraged to try them by a health-care professional. They expressed a preference for these devices in part because they eliminated the need for anaesthetic, which they worried could be required on several occasions if VTs fell out. These parents talked about their own struggles witnessing their son or daughter being anaesthetised. They also mentioned their child’s difficulties with surgery:

. . . he had a bad reaction . . . he was being sick . . . it was just horrible . . . I didn’t want to put him through another operation no matter how big or small.

P33 (m)

. . . we’ve had a lot of battles with her going for surgery . . . she goes to the play specialist a couple of months before surgery . . . cannulas . . . they are the major issue with her.

P34 (m)

It was notable that though interviewees with children who wore HAs said they had been worried about teasing, these fears had not been realised; most stated that their child coped better wearing HAs than they had expected. As for those whose children had tried both treatments, HAs tended to be preferred as a result of poor previous experiences with VTs (e.g. falling out, repeated insertions, ear infections that parents attributed to VTs). Of those receiving both, most children had received VTs followed by HAs; just C13 and C18 had HAs first. Only P25 in this group of parents preferred VTs because, unlike HAs, they allowed for a constant improvement in hearing while in place:

So sort of when you go to bed they’re not in and when you get up on a morning they’re not in so obviously you’ve got that lull of whereas it’s like putting contact lenses in, you can constantly see rather than put your glasses on you know that type of thing, that was the only thing in my head to compare it to . . . the grommets give you a more rounded hearing cause it’s always there as opposed to just when they’re in.

P25 (m)

Views on the presentation of information about trial participation

Interviewees stated that prior to deciding whether or not to allow their child to be part of the proposed trial, they would like to talk to a researcher about it and wanted written information which they could take home and reflect on with family members. They also suggested that information should be provided to children if they were old enough. Participants stated that the information they would like to help make a decision included:

• General That neither treatment would make the child’s situation worse, potential benefits and drawbacks of each treatment and what might happen if either did not work.

• HAs What this would involve for parents and how often they would have to take their child to get new moulds fitted.

• VTs What might go wrong, how many sets would be inserted, the chances of them falling out and any after effects.

Some parents were clear that how the idea of randomisation was presented could affect their willingness to contemplate their child’s involvement:

I don’t know if kind of like, I know it’s, to me kind of like roll of a dice sounds a bit like a board game . . . you’re thinking about kind of like your child’s kind of like welfare and to think of a dice, you’re thinking I don’t know whether or not I like the idea of that, whereas if you’ve got kind of like a computer generated list . . . then that’s fine.

P5 (m)

This is consistent with previous research, which has noted that explaining randomisation in terms of pulling names out of a hat or coin tossing may influence willingness to be part of a trial, leaving individuals feeling as if the approach is haphazard. 29

Conditions for agreeing to participation

This section moves on from the description of beliefs, experiences and knowledge to consider how individuals could be clustered based on factors influencing their willingness to allow their child to be part of a trial comparing VTs and HAs. By reflecting on participants’ responses, we grouped them according to key factors shaping their decision-making:

• ‘protecting’: not wanting to put their child at undue risk of harm (physical or psychosocial)

• ‘fixing’: believing that one treatment was more appropriate and/or convenient

• ‘following’: being persuaded by the views of professionals

• ‘helping’: wanting to advance knowledge and assist patients in the future.

The response of those characterised as ‘protecting’ was influenced mainly by previous experience, either direct or vicarious. These individuals were concerned about perceived physical or social risks associated with either VTs or HAs. Some had witnessed their child having several sets of VTs and some believed that these had caused permanent damage inside the ear. Others were reluctant to agree to the possibility of HAs, seeing these devices as an additional burden on top of scars from surgery and speech difficulties:

Just because urm I think I had so many other things, so many other problems as well then to sort of . . . be picked out for grommets or hearing aids and think it’s going back to the hearing aids issue for us, I think we would say no.

P36 (m)

Whereas parents classed as ‘protecting’ rejected either HAs or VTs, those defined as ‘fixing’ articulated a preference for one of these treatments and a sense of knowing what was best for their son or daughter. Some valued the opportunity to capitalise on their child undergoing an anaesthetic for a palatal closure to have VTs inserted at the same time. Others felt that HAs were preferable because they could avoid the need for repeated VT insertions.

Data from parents whose response was shaped predominantly by a wish to protect or fix implied that accepting uncertainty about which option is most effective (clinical equipoise) was a necessary but not sufficient condition. Agreement to trial participation could also require what we refer to as ‘parental equipoise’. This relates to the wider impact a treatment may have on everyday life. For example, those defined as ‘fixing’ felt that one approach to managing OME was more suitable in terms of its bearing on their child’s psychosocial well-being. This could be expressed as a preference for the expedient solution of inserting VTs while the child was anaesthetised for another procedure. Alternatively, HAs could be preferred as an acceptable solution to hearing loss that avoided the need for surgery. Those classed as ‘protecting’ were not in parental equipoise because they had significant beliefs about potential adverse consequences of either VTs or HAs.

If parents expressed concerns or preferences for VTs or HAs, they were unable to agree to participation. When these were not overriding factors influencing decision-making, some individuals agreed to their child participating in a trial because they trusted practitioners. We have described this as ‘following’. Alternatively, those we designated as ‘helping’ talked about being motivated mainly by a wish to progress knowledge and assist others in a similar situation. These interviewees did not voice strong views about VTs or HAs and, in that sense, appeared to accept there was sufficient clinical and parental equipoise to allow their child to take part in a trial. In their narratives they reflected on the widespread benefits of participation for future generations of patients.

Figure 2 summarises conditions associated with agreeing to be in the proposed trial, which were derived from interview data. It highlights that those saying ‘no’ or ‘unsure’ expressed views associated with ‘protecting’ or ‘fixing’ which were not articulated by those saying ‘yes’, who were willing to follow a practitioner’s suggestion or to help advance knowledge. It also underlines differing moral drivers shaping people’s decisions. Those characterised as ‘protecting’ exhibited the socially expected role of safeguarding their child. Likewise, interviewees described as ‘fixing’ demonstrated parental authority by suggesting a need to improve their child’s circumstances in what they felt was the most efficient way possible. ‘Following’ could result from a sense of loyalty to staff involved in patient care, if they are the people asking parents to take part. ‘Helping’ suggested a wish to assist others in a similar position in the future.

FIGURE 2.

Conditions associated with agreeing to be part of a trial.

These different conditions associated with decision-making (‘protecting’, ‘fixing’, ‘following’, ‘helping’) highlight possible barriers to and enablers of recruitment. They suggest that allaying concerns around a perceived need to protect children from risks, and/or addressing beliefs that one treatment has particular benefits making it more suitable for the child, might be essential prior to recruitment. If parents can be reassured about the safety of interventions proposed, and that there is no definite advantage to either, they may be more likely to move on to ‘following’ or ‘helping’ and, therefore, saying ‘yes’ to randomisation. Further research could test how these drivers of decision-making relate to recruitment, which could allow for individualised approaches to trial presentation based on whether a parent’s view, at recruitment, was most reflective of ‘protecting’, ‘fixing’, ‘following’ or ‘helping’.

Nature of the two interventions being compared

We have described how most interviewees were resistant to the idea of entering their child into the proposed trial because they did not regard it as a fair comparison. As noted by P20:

I don’t think I’d be too happy . . . because they’re two extremes, you’ve got the grommets, which is an operation, or the hearing aid, which isn’t an operation and to me it’s, I’d be, you’d be scared to be put in the grommets group . . .

P20 (m)

To test the influence this perceived lack of equivalence had on parents’ response to our question about trial participation, after 10 interviews we explored participants’ views of their child being involved in a study of a hypothetical medication for OME. As shown in Table 12, they appeared much more receptive to this treatment route. Those whose initial response was characterised by a drive to protect agreed if convinced that the medication was safe, as did parents classed as ‘fixing’, if they saw it as an attractive solution to the child’s hearing problem. In general, participants described medication as a treatment that was easier to accept than surgery or HAs. They talked about taking medication being a ‘normal’ activity compared with the possibility of social stigma associated with HAs or the invasive nature of VTs. In addition, this option was seen as being more in parents’ control, with any side effects rectified by stopping the medication:

. . . you have to try new things because we’re never gonna learn are we and we’re never gonna progress, we’ve got to try new things and it’s a medicine. If he was ill from it, it would either come out of his bottom or through his mouth and I’d stop giving it him.

P37 (m)

The main hurdle that parents saw with this type of treatment was how the medication tasted, which might put the child off taking it.

Only one person agreeing to a child’s participation in a trial comparing VTs and HAs would not allow her son to be part of a study examining medication for OME. Her rationale related to a concern about potential side effects, and in that sense ‘protecting’ came to dominate her decision-making:

. . . the grommets will go in his ear but they come out after so long and . . . they’re not gonna damage him, you know, there’s nothing major that that’s gonna do to his body but obviously him taking a medication, I don’t know whether he could be allergic to something that’s in the medication . . . I wouldn’t really want to put him through that if there was no real, ’cause obviously they couldn’t promise me that something wasn’t gonna happen to [child].

P12 (m)

Two parents who said ‘no’ to the trial of VTs and HAs also said ‘no’ to a study of a hypothetical medication for OME. Their reasons related to a concern about side effects and not wanting to try a new treatment route for this condition.

Discussion: strand 1

Interviewees diverged in their opinions of VTs and HAs, with some favouring the former while others preferred the latter. They often held strong opinions about treatment, but also reflected the uncertainty that surrounds best practice in managing OME in those with CP. Our data show that recruitment could be difficult in research comparing VTs and HAs, a common obstacle with trials30 and a key reason for their failure. 31 Only one-quarter of those interviewed said that they would enter their child into such a study, although parents’ explanations for their responses are potentially more useful than this figure because they identify concerns that, if addressed, might facilitate a higher rate of recruitment.

Other authors have talked about ‘individual’ or ‘patient’ as opposed to ‘collective’ equipoise. 32 We used the term ‘parental equipoise’ to highlight that interviewees thought more broadly than a treatment’s clinical effectiveness, considering its impact on a child’s and the family’s psychosocial well-being. Most participants were not in parental equipoise, a condition that is necessary, we suggest, before agreeing to enter a child into a trial comparing VTs and HAs. Individuals were distributed across a continuum that included phases of decision-making shaped primarily by a need to protect, fix, follow or help. Data suggested that a lack of parental equipoise would need to be addressed if it were not to be a significant hurdle to trial recruitment.

Parents whose responses we have described as ‘protecting’ were not in equipoise because of the risks they associated with treatments. Some were particularly concerned to avoid their child having general anaesthetics. Negative views about VTs could also be based on personal or vicarious experience of multiple insertions, ear infections and/or concerns about long-term damage inside the ear. Therefore, HAs were preferred in some cases because they were non-invasive. Alternatively, parents might worry about their child’s or others’ responses to HAs, owing to social norms with regard to visible difference and disability. The response of interviewees indicates how a range of parental concerns may need to be addressed in order to recruit to a trial, including social experiences as well as clinical benefits and risks.

‘Protecting’ could be seen as focusing on perceived negative implications of a treatment, whereas ‘fixing’ suggested that individuals were making a positive choice of one way of managing OME over another. Parents described as ‘fixing’ believed that a particular treatment would be of greater benefit to a child and/or was more convenient than the alternative. Some considered the risks associated with VTs to be minimal, particularly if the insertion took place while a child was anaesthetised for another procedure. In contrast, HAs may be seen as requiring a longer commitment from parents, ensuring that children wore them, that batteries were replaced and that the HAs were not lost or broken. However, if VTs had not been an effective treatment, for example because they fell out soon after insertion, or if parents had seen the child flourish with HAs, they may believe this to be the most appropriate intervention for their son or daughter.

Parents who did not have strong beliefs about the benefits or risks of either VTs or HAs could be prepared to enter their children into a trial. In the case of those whom we characterised as ‘following’, experience again played an important role in their decisions; if they had a positive rapport with practitioners, whom they had trusted to guide them through previous treatments for CP, they may be happy to accept the request to be involved in a trial. This is not to say that those described as ‘protecting’ or ‘fixing’ mistrusted clinicians, but what was clear from their narratives were strong beliefs about treatments based on experience (whether personal or vicarious) and possibly limited knowledge to make an informed decision. If such beliefs could be addressed, there may be potential for these individuals to be more willing to agree to participation.

Altruism was an important motivation for interviewees agreeing to the potential trial, as reported in other studies. 33,34 However, the fact that parents decide about participation for a child leads to a bioethical debate about whether or not one can act altruistically on behalf of another. 19 Parents may be concerned about making the ‘right decision’, seeing the need to protect their child from harm as fundamental. 35 As a consequence, although some studies have found that parents are motivated to consent to a trial for altruistic reasons,36,37 it can be conditional on reassurances about concerns for a child’s safety and well-being. 35 This reflects findings that ‘protecting’ or ‘fixing’ were potential barriers to trial involvement.

Randomisation was a concept that many interviewees struggled to comprehend. The term itself may imply a haphazard approach to treatment allocation,29 with some people believing it is an unethical way of deciding how a condition will be managed. 38 Parents we interviewed talked about their dislike of the term ‘rolling a dice’, feeling that it devalued treatment choices and decision-making. Misunderstanding of randomisation shows that clear information is required when asking mothers and fathers to involve their child in a trial. Shilling et al. 19 reported that parents they interviewed liked being able to take away information on which to reflect and to share with family members. Likewise, in our study, parents wanted verbal details about the study to be accompanied by written material which they could discuss with others. However, previous research underlines that not everyone will read this information. 39 In addition, parents vary in terms of how much information they want about a trial. 40,41 A wish for more information has been associated with increased anxiety and less sense of control,42 but if parents feel inadequately informed it can dissuade them from participating. 43 Anxiety may be moderated by trust in medical research and the relationship parents have with practitioners. 35 Hence, confidence in a clinical team may be a factor influencing agreement to trial involvement. 19,44 A higher proportion of our interviewees may have said ‘yes’ to their child’s participation in a potential study comparing VTs and HAs if asked by a member of the cleft team, rather than a researcher they had only met for this qualitative study. Conversely, they may have felt more able to communicate their reservations to the researcher, without any fear of upsetting or letting down someone involved in the long-term care of their son or daughter.

Conclusion to strand 1

The diversity of views expressed in interviews indicates that parents did not share a consistent preference for either VTs or HAs. Data suggested that because many parents were not in equipoise when talking about VTs and HAs, recruitment to a trial comparing the two could be problematic. In order to perceive these two management approaches as equivalent, parents require information about how they work and potential drawbacks. Parental equipoise is a term we used to reflect the wider range of considerations, over and above clinical effectiveness, which might influence views on entering a child into a trial. The continuum of parental equipoise presented above (‘protecting’, ‘fixing’, ‘following’, ‘helping’) highlights potential factors driving decision-making around participation, and implies that specific issues may need to be considered when planning a trial comparing VTs and HAs:

• stressing the safety procedures within the trial set up

  • especially if parents are characterised as ‘protecting’

• emphasising and clearly explaining equipoise (recognising that parental equipoise is wider than clinical equipoise)

  • especially if parents are characterised as ‘protecting’ or ‘fixing’

• exploring people’s previous experience/understanding of the two treatments

  • especially if parents are characterised as ‘protecting’ or ‘fixing’

• being careful about who introduces the study to mothers and fathers

  • especially if parents are characterised as ‘following’

• highlighting how results will advance knowledge for future generations

  • especially if parents are characterised as ‘helping’.

As well as equipoise, parents’ understanding of randomisation, and how this process of allocation is described (e.g. throwing a dice, computer generated), could influence recruitment. Parents may believe that practitioners will select the treatment that is best for their child, which could make the concept of uncertainty and random allocation difficult to accept. Prior to recruitment, time may need to be set aside to talk about how randomisation is performed and reasons for its use in this type of research.

Defining outcomes

The Oxford English Dictionary45 suggests the noun ‘outcome’ refers to the way a thing turns out; a consequence. It is a term familiar to researchers when defining the end point measured in a study to examine whether or not a treatment is effective but is less common in everyday conversation in relation to health. This makes it a potentially difficult topic to explore with service users. Nevertheless, research has shown that collecting people’s perspectives on outcomes can be a fruitful endeavour. For example, in the field of arthritis care, feeling less fatigue was found to be more important to those with the condition than traditional areas measured, such as joint tenderness and stiffness. 46

Parents’ views when talking specifically about outcomes following treatment for otitis media with effusion

Towards the middle of an interview, parents were asked what results or improvements they would look for following treatment of OME. If they had difficulty responding, the researcher posed the question ‘What do you think grommets or hearing aids should do for a child with glue ear?’ In our research, two parents found it difficult to distinguish processes from outcomes, listing ‘good aftercare’ as an important result of treatment (see Table 13). However, eventually most individuals were able to say something about important outcomes (only P35 could not, stating it was too difficult because his daughter had not experienced VTs or HAs). Figure 3 shows the responses on electronic ‘sticky notes’ to this part of the interview for P15 and P16. The left-hand picture from the interview with P15 shows that this mother and father could not identify a key outcome, rating hearing and speech as being equally important (which was also the case for P26: see Table 13).

FIGURE 3.

An example of sticky notes from P15 and P16 (left-hand sticky notes relate to important outcomes; those on the right relate to a question about drawbacks of treatment received by their child).

Table 13 presents outcomes referred to by parents when they were asked to focus on this topic within their interview. Green squares indicate the outcome(s) they defined as most important. This tended to be hearing but not for everyone; some people rated communication or seeing their child less frustrated as key, although both were dependent on improved hearing. Parents whose son or daughter had not received treatment (the watchful waiting group) referred to a narrower range of outcomes compared with other interviewees (see Table 13). Hearing, in particular, was listed as key by this group, with only P12 rating something else as more important; she explained that her son had experienced repeated ear infections, so a primary result of treatment should be to address this problem. Similarly, P9 proposed this to be her key outcome, after her son had endured recurring ear infections, which she attributed to repeated VTs. This mother formed part of a group whose child had received VTs and HAs. Reducing pain was listed frequently as an outcome by these interviewees (see Table 13), because a son or daughter had often had HAs following a bad experience with VTs falling out on several occasions and believing they caused damage to the ear.

Those with a child who had received only HAs were similar to the no treatment group in their focus on hearing and communication as outcomes. In contrast, the VTs-only group listed a variety of potential outcomes (see Table 13). Members of this group talked about HAs not addressing what they believed to be the underlying problem of fluid build-up. They were the only individuals to refer to ‘OME not returning’ as an outcome, which suggests they viewed VTs as a cure for the condition. This could cause disappointment when OME recurred if VTs fell out soon after insertion:

Yeah it was a bit of a let down really that they’d, they’d said that it would be better if she had them [VTs] and then it came out more or less straight away . . .

P6 (m)

Education was mentioned by just a handful of parents in response to specific questions asking about outcomes (see Table 13). They listed ‘improved concentration’ and ‘being able to follow a teacher’s instructions’ on their virtual ‘sticky notes’. However, at other moments within the interview participants referred to their joy and relief at seeing a son or daughter advance at school once hearing loss had been addressed. Participants did not mention absence from school as an important outcome, yet during interviews they did raise the issue of time lost from school due to ear infections and for HA appointments:

. . . I’ve just had a right ticking off by the school because of his hearing. He has a green are good, amber is mediocre and red is warning and he’s just got a warning because of his attendance and that is just because of the time spent in hospital and that’s all because of his ENT appointments.

P5 (m)

Improved social interaction was a wish expressed by several parents for their child following treatment for OME. They wanted their son or daughter to be understood by others and for the child to understand others so they could communicate in social settings, with peers and at school. When unable to engage in this manner, children could remove themselves from socialising:

In the playground they would be playing and chasing each other and she wouldn’t be able to hear and she would have to try and look at faces so she would withdraw herself, she withdrawed herself for years.

P24 (m)

Age of children and outcomes

Given the wide age range represented in the study (0–11 years), potential differences in outcomes mentioned by parents for younger and older children were explored. Looking across ages, responses broadly followed the pattern outlined in Table 14, which suggests some differences based on age.

It is understandable that parents of preschool children tended to mention speech and language because at this age these skills are developing. There was some concern expressed about children being competent in speech and language prior to starting school so they could flourish in this setting. At school, children become less dependent on parents and begin developing their own social network. Hence, an ability to interact is important during this life stage. Parents of older children tended to worry about educational performance, perhaps because difficulties in this area were more evident at this age and concerns about coping at secondary school arose. Several of these parents had experienced OME in their child for several years and thought that hearing would have been ‘normal’ by this age (≥ 8 years). Hence, there was some anxiety among parents of older children that their son or daughter would have on-going difficulties with hearing.

Parental stress

Interview transcripts were replete with indications of parental anxiety and strain. For example, seeing a child in pain, usually due to ear infections or damage, was upsetting for parents, who felt helpless to relieve this discomfort. Parental stress could also stem from decision-making around whether to have VTs or HAs and whether or not to try VTs more than once. Some parents talked about the need to maintain HAs (e.g. cleaning them, ordering batteries), although this was depicted as a hassle rather than a major problem. What appeared to be more stressful was when a child lost a HA; parents then had a child who was unable to hear and they had the embarrassment of contacting an audiology service to request a new HA. Frustration exhibited by children when unable to hear caused parental disquiet in case others thought their son or daughter was ‘just being naughty’ (P22 m). It could also lead to disharmony at home, as children were moody with their siblings and parents disagreed about how to manage such behaviour:

. . . it [child’s behaviour] was causing a lot of stress in our house . . . parents could be falling out with each other, saying ‘well why are you not being firmer’ . . .

P18 (m)

They [school] even made an appointment for us to see somebody . . . about her behaviour. So that made me quite anxious and worried and thought that we were, you know, she was really naughty and going off the rails but when we sort of realised that it was more cause of . . . her ears then I was angry at the school more after that because I’d, not taken it out on her but . . . I’d been a bit upset and taking it out on myself more than anything, thinking it was something that we were doing or that I was doing personally.

P22 (m)

Physical and psychosocial outcomes

Parental interviews contained data on a mixture of physical and psychosocial outcomes (Table 15). Interview data suggest that when considering outcomes, parents saw the health problem (OME) and associated symptoms more broadly within the context of the individual child’s life. For example, the physical problem of not hearing appeared to contribute to difficulties in the child’s psychological functioning (e.g. frustration, fear, sadness) and social life (e.g. learning and communication, being unable to take part in activities like swimming). Physical outcomes were related, primarily, to VTs (e.g. infections and perforations). Some interviewees felt that a resolution of hearing loss should not be sought if it was going to cause further significant discomfort for the child:

. . . she had two major ear infections with them [VTs] and they are now blocked so she has to go back and get them looked at. So she’s had nothing but trouble with them . . . The infections and the pain and I don’t think you know her being in pain outweighs her hearing, if that makes sense.

P13 (m)

Parents described the change in their child once hearing loss had been addressed in very emotive terms, emphasising, once again, that OME could affect numerous areas of a child’s social functioning and psychological well-being:

He just was like in this bubble and it was just, I just, to me I can just picture the ears just opening up [when VTs were inserted] and everything getting in . . . So really it was just participating in more things perhaps, being involved a little bit more in everything that was going on around him, being able to give him the chance to do it all . . . He was even more enthusiastic and just a completely different boy to me really [laughs].

P4 (m)

Their narratives often centred on ‘normality’. They wanted their son or daughter to have the same opportunities in life as their peers and to reach their potential at school, noting that OME could prevent this from occurring. It was implied that successful treatment would help the child become confident and content by allowing them to hear what was taking place around them, so they did not feel like an outsider. When OME was present, parents suggested it could mean their son or daughter was disconnected from the social world, resulting in the child becoming agitated:

Oh, he was not a very nice toddler [laughs]. He was quite difficult as a toddler, like we’d go to like places, he was alright at pre-school, he was better for other people but when it was like, especially when he was really little . . . when other kids are all starting to speak and he couldn’t communicate properly with other children. So he’d get, he’d like lash out, you know, he’d try and talk to them but obviously he wasn’t making the right sounds.

P33 (m)

A desire for the child to have the same life chances as their peers and not to miss out educationally relates to a concept underpinning parental interviews around achieving developmentally appropriate independence. For example, parents worried if their child was reliant on siblings for friends and social activities, and encouraged them to pursue their own interests. Interviewees appeared to value a progression towards autonomy and talked about seeing the child’s personality emerging as a positive. Box 2 provides some exemplar quotations from parents on this topic.

Children’s views of outcomes

The term ‘outcomes’ could have been difficult for children to understand so they were invited to say what they thought was ‘good’ and ‘not so good’ about VTs or HAs. Responses to this question are listed in Table 16, which shows that children varied in their views, with some struggling to provide an answer because they had received treatment (usually VTs) several years previously or were in the watchful waiting group.

Hearing was referred to by children as a primary result of treatment for OME because it allowed them to understand others and to join in activities. However, one 10-year-old girl said an improvement in hearing had to be balanced against problems associated with interventions such as HAs:

They [HAs] make you hear properly. But I’d rather no hearing. I’d rather, I’d rather not hear properly than wear those things. I hate them so much.

What makes you hate them so much?

Everything about them. Sometimes they can get way too loud and I can’t really turn them down … and then sometimes they’re really, really low when you turn them on and then they stay like that . . . I’d be sitting there [in the classroom] with the hearing in my ear blocking all the sounds from my right ear cause, I’d be like, ‘what, what, what did you say sir?’

For participants who wore HAs, being involved in their design was a positive factor associated with these devices. Conversely, when discussing negative aspects of HAs, children mentioned their visibility; that they could fall out when playing; having to change batteries; an inability to alter the volume setting; and they could be uncomfortable in their ear. The main issues they associated with VTs included having to undergo an operation; a belief that they could cause infections; and feeling their presence inside the ear. In Box 3, ideas relating to children’s expectations of treatment for OME have been summarised into four main areas, alongside illustrative quotations. More views were expressed (positive and negative) about HAs compared with VTs, possibly because children had experienced them recently or currently wore these devices, whereas VTs may have been inserted several years prior to interview.

Combining children’s and parents’ views

Children and parents placed an importance on hearing as an outcome. Yet, as mentioned above, it was not referred to in isolation and interviewees did not focus on hearing as simply a physical improvement that could be measured. Instead, they associated it with psychosocial functioning, such as:

• independence (freedom for the child to go and do things on their own)

• performance at school (more attentive, able to read because they could hear words)

• able to interact and connect with the social world (being understood and being able to understand what others were saying)

• less frustration because of better communication, resulting in improved behaviour and self-confidence

• engagement in everyday activities (e.g. swimming, cinema)

• impact on home life (less shouting, TV less loud).

Views of outcomes from children and parents are merged in Figures 4 and 5. Figure 4 depicts physical outcomes mentioned within interviews, whereas Figure 5 centres on psychosocial ones. Figures 4 and 5 illustrate a pathway from hearing to physical and psychosocial outcomes, respectively, based on data from interviewees. White rectangles in these figures indicate outcomes mentioned by parents only, whereas shaded ones represent those referred to by children and parents. Children’s views were not inconsistent with those of parents, but mothers and fathers did have more to say about outcomes that could be used to assess the effectiveness of interventions for OME.

FIGURE 4.

Physical outcomes described in interviews (shaded boxes = mentioned by parents and children; white boxes = mentioned by parents only).

FIGURE 5.

Psychosocial outcomes described in interviews (shaded boxes = mentioned by parents and children; white boxes = mentioned by parents only).

Treatment-specific outcomes

Certain aspects of managing OME were specifically connected to a treatment. These are highlighted in Figure 6, which shows that VTs, in particular, were associated with physical outcomes. For example, parents often referred to VTs as bringing additional benefits compared with HAs by addressing fluid build-up inside the ear. In some interviews, parents believed that VTs reduced the frequency of ear infections experienced by their child. Conversely, others blamed VTs for constant pain:

I think him having the grommets has caused more agony and ear infections and sleepless nights . . . He’s had very many sleepless nights to the point where you give paracetamol but the child still says ‘I can’t sleep, it’s painful’ and you as a parent don’t have anything to do other than be there for your child, console them . . .

P9 (m)

Ventilation tubes were also associated with procedural issues and, to a lesser degree, social consequences. In contrast, HAs were related mainly to social consequences (see Figure 6). Although parents expressed concerns that HA would result in their child being bullied, in reality this fear did not appear to be realised; children who wore HAs were said to adapt very well to these devices and did not report the social stigma that parents anticipated. However, as children got older they appeared to become more self-conscious about having a HA. One boy, aged 8 years, talked about his frustration when classmates tried to touch his HAs:

They just talk to me all the time about them and everyone touches them.

Everybody touches them, at school . . . How do you feel about that?

I feel weird.

Why do you feel weird?

Because everyone keeps touching them.

FIGURE 6.

Outcomes associated with treatment described by parents and children (green rectangle = mentioned by parents and children; green oval = mentioned by parents only; white rectangle = mentioned by children only).

Discussion: strand 2

Interviews started with participants talking about experiences of OME and then midway through they were invited to focus discussion on the topic of outcomes. Therefore, most parents and children identified outcomes directly by listing them on a tablet computer and indirectly through describing how this condition and its management affected their lives; outcomes were explored within the context of this broader reflection on experiences, meaning that although reference to outcomes was sometimes explicit, it also involved drawing out from participants what were important consequences of OME based on their overall narratives about living with this condition.

The key outcome emerging in participants’ narratives was hearing. This was referred to by children and parents, who saw it as more than a restoration of physical functioning. Instead, hearing was described as important largely because of the psychosocial consequences associated with its loss, such as relationships with others and ability to progress in life. This could be summarised as the child’s ‘connection with their social world’. Figure 5, in particular, delineates how psychosocial variables were linked to hearing. However, some parents and children implied that hearing should not be sought at any cost; comfort and absence of pain were felt to be more important in some cases than being able to hear everything that was occurring. Hence, a degree of hearing that allowed children to function in the social world without excessive discomfort was a happy medium that certain interviewees depicted as acceptable.

Data suggest that particular outcomes may become increasingly important over time, with parents of older children being more concerned about social interaction and education; whereas earlier, focus on speech and language development may be a priority. Others have noted that hearing loss can leave individuals behind educationally. 47 Education was only mentioned by parents; children were more concerned with a wish to fit in with friends and not missing out on valued activities (e.g. swimming and playing). Parents identified consequences for children’s self-esteem and confidence as important outcomes, which could be related to a notion of experiencing a ‘normal’ childhood and reaching their potential. This highlights that outcomes discussed during interviews did not necessarily relate to a discrete endpoint, implying that a distinction may be required between immediate and longer-term consequences of treatment for OME.

Conclusion to strand 2

Hearing was a key outcome referred to by parents and children. This was probably to be expected, given the nature of the problem and aims of treatment. Yet although hearing was central to interviewees’ concerns, this was largely because of its psychosocial consequences. Hence, rather than absolute hearing at one moment in time, participants talked about the impact of not hearing on social development and psychological well-being; interviewees were looking for a health gain in terms of hearing to engender improvements in the child’s functioning in their social world and to bring them psychological comfort, including self-confidence, a move towards independence and the opportunity to develop their personality. Interviewees talked about significant psychosocial difficulties that stemmed from hearing loss, including poor social interaction and educational performance. Furthermore, both physical and psychosocial outcomes contributed to parental stress, with hearing difficulties adding to anxiety and pressures already associated with caring for a child who had a cleft.

Population

Children, with and without CP, aged < 18 years with OME.

Intervention

Any surgical intervention used to manage OME.

Comparison

Untreated control or other surgical interventions.

Outcome

Any outcome that was reported.

Criteria for considering studies for updating the Bristol review1 and Cochrane reviews identified through Cochrane Central Register of Controlled Trials search

We included the following types of studies:

• systematic reviews with/without meta analyses

• RCTs

• controlled clinical trials

• case series

• prospective cohorts

• retrospective cohorts.

Search methods for identification of studies

An identical search strategy to Ponduri et al. 1 was used and applied to the Cochrane Central Register of Controlled Trials (CENTRAL), EMBASE, MEDLINE, and Cumulative Index to Nursing and Allied Health Literature (CINAHL) (January 2006–April 2011).

The detailed search strategies applied are given in Appendix 4.

Multiple databases were utilised to maximise the sensitivity of a search. CENTRAL comprises only studies which are deemed to be controlled trials by a team of reviewers; EMBASE, MEDLINE and CINAHL include published research of various study designs. The advantages conferred by using CENTRAL in addition to the other databases is that trials from other sources of research (e.g. journals not indexed in MEDLINE and conference proceedings) are hand-searched, and controlled trials from these are included. This improves the chances of identifying all relevant studies.

Eligibility of studies

Two researchers (MOS and KOB) independently assessed the abstracts of studies resulting from the searches using a screening proforma. Full copies of all potentially relevant studies and those appearing to meet the inclusion criteria, or for which there were insufficient data in the title and abstract to make a clear decision, were then obtained.

The full-text papers were assessed independently by two review authors (MOS and KOB) and any disagreement on the eligibility of included studies resolved through discussion. Where resolution was not possible, a third review author (IAB) was consulted.

For the purpose of this study there was no synthesis of outcome data from the RCTs, and hence a critique of the methodological quality of the studies was not necessary.

Data extraction

Two reviewers (NH and IAB) independently extracted the data. NH and IAB then reviewed the extracted data together to assess consensus and to ensure that all outcomes have been identified. Disagreement was resolved through discussion, where resolution was not possible, a third review author (KO) was consulted. The following data was extracted from each study:

1. study type

2. author details

3. year and journal of publication

4. intervention(s) under investigation

5. whether the study population was exclusively paediatric and CP or mixed (children and adults, with or without CP)

6. age and number of children included in the study population

7. inclusion and exclusion criteria.

Outcomes:

1. the outcomes which were measured, including the method of measurement

2. the time points at which they were measured

3. if stated, the designated primary outcome

4. designated secondary outcome(s).

Data analysis and presentation

For analysis purposes the data was initially tabulated so that for each study outcomes were listed.

The outcome domains were then determined following a review of the extracted outcomes by the authors (IAB and NH). The outcomes were then grouped under these domains. Finally, the outcome domains and included outcomes were reviewed by the SAG to assess suitability of domain name and grouping of outcomes.

Description of included studies

The characteristics of included studies together with outcomes measured are given in Table 17. Ponduri et al. 1 reported 18 studies of children with CP and 17 have been included in this review. 62,63,65–78 One paper of the original 18 identified110 was excluded from the present study as the results included are reported in a later paper by the same author.

A further eight studies of children with CP have been included. 5,79–85 In addition, the search was expanded to studies of children without CP or where the presence or absence of a cleft was not specified, this yielded a further 24 studies. 86–89,91–109

Studies did not always clearly state the primary outcome and when this was the case the primary outcome was inferred from the sample size calculation, the study title and/or the results presented first in the results section. Papers which did not explicitly identify a primary outcome are detailed in Table 17. The primary outcome was determined independently by NH and IAB, and reviewed with any disagreement discussed further, there was disagreement for only two papers with further discussion required to agree the primary outcome. Papers with disagreement are noted in Table 17.

Each outcome measured was listed by study, only individual outcomes were included (e.g. where an outcome was measured using different methods this was counted as one outcome but the methods of measurement noted).

Generation of an outcome list for use in the Delphi survey of health professionals

The number of outcomes measured in an individual study varied with a median of 6 outcomes (range 1–14 outcomes) per paper. All outcomes measured were reviewed by NH and IAB, and grouped into possible outcome and outcome domain headings. After the initial review a total of 43 outcomes were listed under 13 domain headings (Table 18). Outcomes related to resource use were considered to be outside of the scope of the COS and would instead be considered as part of the VOI analysis (see Chapter 5). Consequently, the outcomes ‘necessity to visit doctor’ and ‘level of speech therapy support required’ were not considered in the initial list of 45 outcomes.

The draft list of outcomes was then circulated to the mOMEnt SAG. The SAG comprised cleft clinicians representing speech and language therapists (n = 2), cleft surgeons (n = 1), ENT surgeons (n = 2), audiologists (n = 2) and clinical psychologists (n = 1). Members of the SAG reviewed outcomes relevant to their clinical field. The SAG also commented on the suitability of the overall domain under which outcomes are grouped.

Following review by the SAG and further review by the Study Management Group (SMG) the following actions were taken (Figure 8):

• The SAG were asked if ‘temporary membrane retraction’ should also be included as an outcome as temporary retraction was included. All agreed that it was not necessary to include this in the list of outcomes.

• The outcome ‘language and comprehension’ was removed as this is duplicated by the individual outcomes of ‘receptive language’ and ‘expressive language’.

• Speech’ was grouped with ‘speech and language development’.

• Speech intelligibility was listed as a separate outcome.

• Outcomes related to behaviour were split into ‘externalising behaviour’ and ‘internalising behaviour’ based on the domains used in the Child Behaviour Checklist.

• ‘Intelligence and academic achievement’ was split into two individual outcomes.

FIGURE 8.

Development of final list of outcomes to be used in the Delphi survey of health professionals. SSC, Study Steering Committee.

Both the SAG and the Study Steering Committee (SSC) tested the online system with an opportunity to score each outcome and to add any additional outcomes that they felt were important. Two members of the SAG added outcomes related to HAs; however, these were outside of the scope of the COS, surgical management of OME, and were not included in the outcome list. The SSC noted two additional outcomes ‘child’s satisfaction with treatment’ and ‘child’s perspective of speech’ which were taken forward to the list of outcomes to be scored by clinicians.

The final list of outcomes used in round 1 of the clinician Delphi comprised 45 individual outcomes (43 identified from the systematic review of the literature and two by the SSC) grouped under 14 domains (Table 19). To aid all participants completing the Delphi an outcome tip was written for each outcome and suitability confirmed with the SAG.

Survey content

Parents and children were asked to score a list of outcomes which was based on the outcomes list generated from the systematic review. Each outcome was reviewed by NH and IAB, and a plain-language alternative suggested that was appropriate for a child aged 7–10 years. This description was tested for readability using the National Institute of Adult Continuing Education Simplified Measure of Gobbledygook calculator111 and was further checked for understanding with the CLAPA CYPC and the local Happy Faces Group. Some outcomes which related to specific clinical observations were combined so that parents and children scored a total of 36 outcomes representing the 45 outcomes identified from the systematic review and SAG/SSC contribution.

The same outcome wording was used for all participants with the exception of minor changes such as ‘your’/’your child’s’ to ensure appropriate context (Table 21).

Parents and adults were asked to consider the appropriate question described in Table 22 and to then score each of the outcomes listed using the Grading of Recommendations, Assessment, Development and Evaluations (GRADE) scale of 1–9. In the online survey the scale was presented in the format 1–9 with 1–3 labelled as ‘not that important’, 4–6 labelled as ‘important’ and 7–9 labelled as ‘really important’112 (Figure 9). Parents were also provided with an option to add anything else that they considered relevant in a free-text box.

FIGURE 9.

Screen shots of online survey for children aged 7–16 years and parents.

Children aged 7–16 years were shown the same list of outcomes as parents and were asked the question shown in Table 22. However, under the recommendation of the CYPC, scoring was completed using a traffic light system where scores of 1–3 were represented by a red box labelled as ‘not that important’, scores of 4–6 as an amber box labelled as ‘important’ and scores of 7–9 as a green box labelled ‘really important’ (see Figure 9). A free-text box was also provided so that participants could add anything else they considered relevant.

The study management team discussed whether or not parents, children and adults should see a free-text box first, before the list of outcomes, as there were concerns that participants might not feel that they had a voice once the list was shown. However, feedback received from parents and children was that it would be better to have the list first as that would help them to think about things and whether or not anything was missing.

Participants

Participants of the survey for parents and children were identified using the CLAPA mailing list and social media pages with a potential reach of 4710 and 9564 respectively. There is likely to be substantial overlap with membership of multiple Facebook pages (Facebook, Inc., Cambride, MA, USA) and groups, but it was not possible to assess this. An individual e-mail was sent to all those on the CLAPA mailing list together with a reminder in their e-newsletter. A link was posted on the Facebook page which included the researcher’s name (NH) and a link together with a photograph. Examples of the newsletter and social media post are provided in Appendix 5.

Consensus matrix: parents and children

The results of the one-off survey of parents and children were reviewed against the definition of consensus agreed prior to the start of the study (see Table 28). 10 Using this definition parents and children had reached consensus in 26 and 9 outcomes respectively (Table 26).

Participants: health professionals

The Delphi study was conducted with the clinical teams from all UK cleft centres. Health professionals were selected from only UK centres due to time and cost constraints. The clinical lead at each of the UK cleft centres was asked to provide the names and roles of all members of the cleft team. The clinical roles considered key for completion of the Delphi were audiologists, ENT surgeons, speech and language therapists and specialist nurses. Other health professionals (e.g. paediatricians, clinical psychologists, clinical geneticists and cleft surgeons) were identified after consultation with the clinical director or cleft service co-ordinator at each cleft centre and were invited accordingly.

All health professionals, identified through correspondence with the clinical lead at UK sites, were contacted directly by e-mail with information about the mOMEnt study and instructions on how they could participate (see Appendix 6). A summary of invited health professionals is given in Table 27. All participants were asked to confirm their clinical role and cleft centre on registration for the mOMEnt Delphi survey.

At the beginning of the exercise participants were reminded of the importance of completing the entire Delphi process. Reminder e-mails were sent to aid completion of each round together with follow-up telephone calls to encourage completion. Participants who took part were given a unique identifier to allow tracking of attrition at each round. All data was stored against the unique identifier and participants were not able to identify other participants or individual responses.

Delphi survey: round 1

In the first round of the online questionnaire participants were asked to provide their name and e-mail address together with their clinical role and cleft centre. This information was stored in a separate database to the scores and used to provide the respondent with a unique identifier. The unique identifier was used to allow identification of individuals completing all rounds of the Delphi exercise and to track attrition between rounds.

In round 1 participants scored the list of outcomes generated from the systematic review of the literature and through SAG and SSC contributions (see Table 19). Outcomes were ordered alphabetically, to avoid potential weighting of outcomes caused by the order in which they are displayed, and this was stated in the introductory text. Participants were asked to consider the question ‘What outcomes influence your management of children with CP, with, or at high risk of, OME?’ and to then score each of the outcomes listed using the GRADE scale of 1–9. In the Delphi exercise the scale was presented in the format 1–9 with 1–3 labelled as ‘not important’, 4–6 labelled as ‘important but not critical’ and 7–9 labelled as ‘critical’. Participants were also provided with an option to add additional outcomes that they considered relevant together with a score for each outcome added (Figure 12).

FIGURE 12.

Example of online system for participants to score each outcome and to add additional outcomes they consider relevant.

Delphi survey: round 2

In round 2, each participant was presented with the number of respondents and distribution of scores for each outcome for their particular stakeholder group. Participants were shown their score from round 1, asked to consider responses from the other members of their stakeholder group and asked to rescore the outcome.

Any changes to scores in light of the stakeholder group or overall response were documented. Those who did not take part in round 1 and did not provide a score were not invited to participate in round 2 or participate further in the Delphi.

Delphi round 3

In round 3, participants were shown the distribution of scores, for each outcome, for all stakeholder groups separately as well as a summary of the results from the survey of parents and children. Participants were asked to rescore all outcomes.

Consensus meeting

The final phase of the study was a face-to-face consensus meeting with participants who had completed either the health professional Delphi, parent and child survey or the qualitative study and members of the SAG. Additional ethical approval was required to invite parents who participated in interviews to attend the consensus meeting. This was obtained from the Greater Manchester East Research Ethics Committee (reference 11/NW/0586).

The responses from round 3 of the health professionals Delphi and the parent and child survey were used to inform the structure and content of the consensus meeting.

Definition of consensus

The classification described in Table 28 was used to determine if consensus was reached.

For consensus to have been reached that an outcome should be in the COS, agreement by the vast majority regarding the critical importance of the outcome, with only a small minority considering it to be not important at all, is required. Conversely, for consensus to have been reached that an outcome should not be in the COS, agreement by the vast majority regarding the lack of importance of the outcome, with only a small minority considering it to be critically important, is required.

Although the choice of thresholds is inevitably somewhat subjective, the specification of the definition of consensus upfront aimed to reduce the chance of consensus being defined post hoc in such a way as to bias the results towards the beliefs of the research team.

Statistical considerations

Free-text responses provided in round 1

Eighteen free-text responses were provided by health professionals in round 1, all of which were reviewed by the SMG and SAG for classification. Eight responses represented a comment only with no associated score (see Table 31), two related to the use of HAs and were considered outside of the scope of the study (Table 32), the remaining eight described outcomes (Table 33). Of the eight potential outcomes two related to attention and behaviour, these outcomes are included in the broad domains of internalising and externalising behaviour and so were not carried forward. One participant listed both attention and listening, listening skills was considered to be a separate outcome and was taken forward to round 2. Two participants listed outcomes related to psychosocial well-being and this also represented a new outcome taken forward to round 2. Two responses were received about parental involvement and patient willingness to participate in speech activities, these were not considered to represent an outcome of treatment.

Following completion of round 1 the scores were compared with the definition of consensus to determine which stakeholder groups had reached the definition of ‘consensus in’. The number of outcomes for which ‘consensus in’ was reached varied across stakeholder groups. ENT surgeons and audiologists/audiological physicians had reached consensus for the fewest outcomes (n = 6), whereas specialist cleft nurses considered 33 outcomes for inclusion in the COS.

The outcome with the most stakeholder groups (five out of six) reaching ‘consensus in’ after round 1 was ‘hearing’.

Details of the stakeholder groups reaching consensus in for each outcome are shown in Table 34.