Genedrive kit for detecting single nucleotide polymorphism m.1555A>G in neonates and their mothers: a systematic review and cost-effectiveness analysis

Prevalence of m.1555A>G variant and risk of aminoglycoside-induced hearing loss

Neonates with suspected infection are commonly treated with gentamicin, an antibiotic of the aminoglycoside family. These antibiotics are associated with a very high risk of damage to the ear (ototoxicity), including profound bilateral deafness, in people with the MT-RNR1 gene m.1555A>G mitochondrial variant. 2,3

Cohort studies in various countries suggest that the variant is rare. For example, in the UK, Rahman et al. have found similar prevalence rates of m.1555A>G in two representative samples of the UK population: the Avon Longitudinal Study of Parents And Children (ALSPAC) [0.19%, 95% confidence interval (CI) 0.10 to 0.28; 18/9371 participants]4 and the 1958 Birth Cohort study (0.26%, 95% CI 0.14% to 0.38%; 19/7350 participants). 5

Given these low prevalence rates, it is unsurprising that aminoglycoside-induced hearing loss (AIHL) has been investigated primarily in case–control studies, in families who have experienced hearing impairment due to maternal inheritance of the m.1555A>G variant. These studies have found that all people exposed to aminoglycosides experienced hearing loss. 2,3 However, these studies’ designs are likely to have overestimated the risk of aminoglycoside exposure. Cohort studies of hearing loss in people with the m.1555A>G genetic variant in broader populations (e.g. preterm infants, neonates in NICUs not selected on the basis of existing hearing impairment) have suggested greater uncertainty about the risk of AIHL.

A German study of preterm infants found that only 3 out of 10 infants with the m.1555A>G variant and exposed to aminoglycosides failed the newborn hearing screening test. 6 Two American studies conducted in NICUs also suggest that not all infants with the variant and exposed to aminoglycosides experienced hearing loss. Ealy et al. 20117 identified two infants with the m.1555A>G genetic variant who received aminoglycosides. Both passed the newborn hearing screening test. Johnson et al. 20108 identified three infants with the m.1555A>G genetic variant, all of whom were exposed to aminoglycosides. Only one of these infants failed the newborn hearing screening test.

However, these studies also have multiple limitations. For example, later hearing loss due to neonatal exposure to aminoglycosides cannot be ruled out in those infants who passed newborn hearing screening tests. In addition, these studies are based on very small samples of people with the m.1555A>G variant. Therefore, there is substantial uncertainty regarding how many neonates with the m.1555A>G variant and exposed to aminoglycosides are likely to experience hearing loss.

m.1555A>G variant and nonsyndromic hearing loss (without exposure to aminoglycosides)

The prevalence of nonsyndromic hearing loss in people with the m.1555A>G variant is a further uncertainty.

Case–control studies in people with the m.1555A>G genetic variant experiencing hearing impairment suggest that AIHL may not explain all hearing impairment in these populations. For example, one Spanish study2 found that 65% (45/69) of families who carried the variant experienced hearing impairment despite having no exposure to aminoglycosides. In another case–control study of 70 Spanish families, Estivill et al. 3 estimated that 39.9% of carriers of the variant who were not exposed to aminoglycosides still experienced hearing loss. However, these authors found a much lower median age for hearing loss among those treated with aminoglycosides (5 years) than among those not treated with aminoglycosides (20 years).

As indicated above, case–control studies may overestimate the risk of nonsyndromc hearing loss. For example, no evidence of hearing loss was found in people with the m.1555A>G variant in two UK population cohort studies conducted by Rahman et al. 4,5 However, no data on aminoglycoside use were available, and the sample of people with the variant was small in both studies. The Australian Blue Mountains Hearing Study had contrasting findings. Six participants (total sample size n = 2856 participants) identified with the m.1555A>G variant all experienced hearing loss, yet none reported aminoglycoside use. After statistical adjustment, three of six carriers of the m.1555A>G variant were found to have mean auditory thresholds higher than those of the general population.

Maternal inheritance of m.1555A>G variant

As it is a variant of mitochondrial deoxyribonucleic acid (mtDNA), the m.1555A>G variant is inherited maternally. Mitochondrial DNA variants are commonly heteroplasmic (when mtDNA varies widely within the same cell and mitochondrion). Therefore, most children have similar but not identical mtDNA to their mothers and other maternal relatives. However, some mitochondrial variants are homoplasmic (when all or most copies are identical throughout mtDNA), resulting in greater penetrance of the variant.

Most studies of this variant (e.g. Matsunaga et al. 9) have found that people are homoplasmic for the G allele. However, people with a heteroplasmic variant have been identified in several studies, including one in Spanish families with m.1555A>G and hearing impairment10 and a large genetic screening study (n = 24,349 neonates) in a Chinese hospital. 11 Del Castillo et al. 10 found that in six families there were 19 people had a heteroplasmic variant and 12 people had a homoplasmic variant. The proportion of variant copies differed widely in the heteroplasmic participants (3.75–96.60%). Although del Castillo et al. found correlations between variant load and hearing thresholds, the small sample size makes these data difficult to interpret. Luo et al. found that, of 46 neonates, most (n = 39) with m.1555A>G were homoplasmic and 7 were heteroplasmic. 11

Population

The population was all babies being considered for treatment with aminoglycosides. Possible subgroups of these patients included those who presented with early-onset (≤ 72 hours post birth) or late-onset (≥ 72 hours post birth) neonatal infection; neonates who needed antibiotic treatment (i.e. a decision to start antibiotics had already been made); neonates who were anticipated to need antibiotics (i.e. a decision to start antibiotics had not already been made); and neonates of different ethnicities. Additionally, we planned to include mothers tested for the variant pre-birth of the neonate. However, none of the subgroups could be examined due to the lack of data.

Intervention

The intervention was Genedrive MT-RNR1 ID Kit, used to determine a neonate’s MT-RNR1 m.1555A>G status, when used to test:

• the neonate directly, or

• their mother (pre-birth of the neonate).

Comparator

The comparator was no testing to determine a neonate’s MT-RNR1 m.1555A>G variant status prior to them receiving aminoglycosides.

Outcomes

The outcomes of interest were divided into intermediate measures of the usage of the equipment and its effects on antibiotic treatment plans, clinical outcomes, patient-reported outcomes and patient experience (Table 1).

Timing

Antibiotic treatment for neonates is recommended within 1 hour of the decision to treat. Therefore, the test is time sensitive.

Reference standard (for test accuracy data)

The reference standard was laboratory-based confirmatory genetic testing. Approaches may differ across genetic laboratory testing centres, including techniques, such as restriction enzyme assay and sequence analysis (such as Sanger sequencing).

Study design(s)

We considered all study designs that provide relevant outcome data, as listed in Table 1.

Setting(s)

The setting was secondary care (hospital, neonatal unit).

Diagnostic test accuracy

For patient selection, McDermott et al. 18,19 was rated as being at low risk of bias. This was based on the assumption that consecutive sampling was used, although this is not explicitly stated. Additionally, while a case–control design was used for the preclinical trial, a prospective study design was used for the implementation, from which the diagnostic accuracy results are presented.

The index test was also rated as being at low risk of bias, even though the test was modified at a point during the study (due to high failure rate) as all samples were retested with the updated version of the device. The question regarding thresholds was not considered for this assessment as it is a genetic variant that is either present or not. The conduct and interpretation of the test was reported in adequate detail. Details regarding the reference standard are unclear, with no information reported on whether those interpreting the test had knowledge of the index test result. Therefore, the reference standard is rated as being at unclear risk of bias. The final domain of flow and timing was rated as being at high risk of bias. This was due to the reported variation in numbers who underwent the test, compared with those not included in the analysis.

Other clinical outcomes

All outcomes except one (test failure rate, which was rated as being at low risk of bias) were rated as being at moderate risk of bias. This is because the failure rate, which was 17.1%, was not included in the analyses of the outcomes illustrated in Figure 2. Consequently, not including failure rate could affect the outcome results. All of the other risk-of-bias domains seemed to be reported adequately. See Figure 2 for a risk-of-bias visualisation using the ROBINS-I tool. 17

FIGURE 2.

Risk Of Bias In Non-randomized Studies – of Interventions tool visualisation by outcomes.

Bias due to confounding was rated low as there was a lack of apparent confounding effect in the causal relationship between the outcomes. Bias in selection of participants into the study was rated as low because all participants who would have been eligible were included in the study. Bias in classification of interventions was rated low as the intervention group was described in detail, and, although the intervention was most likely known before implementation, this is unlikely to have negatively impacted the outcomes. Bias due to deviations from intended interventions was rated as low as there were no apparent deviations. Bias in the selection of reported results was considered low as the method used appears to be the only way of measuring the outcomes.

Diagnostic test accuracy

In the preclinical trial, buccal samples were collected and genotyped from 159 participants, with 304 samples. The controls were split into two groups. The first was people who had received confirmation that they did not carry the m.1555A>G genetic variant (assessed via normal clinical laboratory processes; n = 74). Second, children on the NICU were recruited (n = 55, 110 individual specimens) to ensure that there were no factors specific to neonatal swab sampling that would impair the assay. The cases were individuals who previously had received confirmation that they carried the m.1555A>G genetic variant (n = 32, 62 individual specimens). The Genedrive MT-RNR1 ID Kit was validated for both adults and neonatal populations in this case–control study. The sensitivity was reported as 100% (95% CI 93.9% to 100%) and the specificity was reported as 100% (95% CI 98.5% to 100%). This part of the study was not assessed in the quality appraisal above.

In the prospective study, 424 of the 526 (80.6%) neonates who received antibiotic treatment were included in the analysis. Three neonates were identified to have the m.1555A>G variant and this was confirmed by Sanger sequencing. There were five false positives and no false negatives. The assay produced a sensitivity of 100% (95% CI 29.2% to 100%), a specificity of 99.2% (95% CI 98% to 99.7%), and an accuracy of 99.2% (95% CI 98% to 99.7%). Throughout the trial, the MT-RNR1 assay was updated to improve efficiency, a process that led to the identification of an issue with the buffer and cartridge, which was linked to the false-positive rates. The issue was resolved with an updated buffer and cartridge design.

Number successfully tested

Only neonates were assessed in this included study, with 424 successful tests of 526 admissions. Of the 526 admissions, 12 did not have an index test (no further information was provided regarding the reasons). The remaining tests failed (unsuccessful genotyping).

No mothers were tested.

Test failure rate

Among the 526 admissions who received antibiotics, 90 (17.1%) failed tests were reported. Among the whole cohort (n = 749), the failure rate was 128 (17.1%). The failure rate was determined to be caused by low signal intensity during the melting phase, which was resolved after the recruitment period by modifying the assay buffer and using a redesigned cartridge. Repeated testing of samples in which genotyping had previously failed led to a reduced failure rate of 5.7% in a clinical setting and 0% when performed in the laboratory. 19

Impact of test result on decisions about care

The study reports that ‘in all cases where a m.1555A>G genotype was identified, aminoglycoside antibiotics were avoided and alternative cephalosporin-based regimens were used’. 19

Impact of test implementation and use on healthcare resources

The MT-RNR1 point-of-care test analysis is automated without any user interpretation, providing the user with a ‘detected’ or ‘not detected’ actionable result in 26 minutes of initiating the analysis. The authors suggest an approximate 30 minutes from collection to an actionable result. 19

No further data regarding the impact of the test implementation and use on healthcare resources are reported.

Time to obtaining a sample for testing

The median time to swab throughout the study was 6 minutes (interquartile range 3–16 minutes).

Time to results

The MT-RNR1 point-of-care test was able to genotype the m.1555A>G variant in 26 minutes.

Time to antibiotic treatment

The study authors report that, prior to implementation, the mean time to antibiotic therapy was 55.87 (standard deviation 22.56) minutes based on 95 consecutive acute admissions over 1 month. During the study, the corresponding mean time to antibiotic therapy was 55.18 (standard deviation 23.82) minutes. The difference in mean time to antibiotic therapy (–0.87 minutes, 95% CI −5.96 to 4.23 minutes), before and after implementation of the MT-RNR1 assay, was not statistically significant. The 95% CI was within the prespecified boundaries of statistical equivalence.

Number of neonates identified with m.1555A>G

There were three neonates identified with the variant, five false positives and no false negatives.

Usability of the test

The study did not report on this outcome.

Mortality

The study did not report on this outcome.

Morbidity

The study did not report on this outcome.

Health-related quality of life

The study did not report on this outcome.

Patient experience

The study did not report on this outcome.

Developing a clinical pathway

To develop the clinical pathway [using GitMind (Wangxu Technology Co. Limited, Hong Kong)] for using the Genedrive MT-RNR1 ID Kit to detect m.1555A>G in neonates, we reviewed related documents to map out the treatment pathway in the NHS for the target population. This clinical pathway was checked with clinical experts consulted by the EAG and revised following their comments. The main documents that we initially used to develop clinical pathway are as follows:

1. NICE advice: Genedrive MT-RNR1 ID Kit For Detecting Single Nucleotide Polymorphism m.1555A>G in Newborn Babies21

2. NICE guideline 195: Neonatal Infection: Antibiotics for Prevention and Treatment12

3. ‘Clinical Pharmacogenetics Implementation Consortium Guideline for the use of aminoglycosides based on MT-RNR1 genotype’13

4. ‘Pharmacogenetics to Avoid Loss of Hearing (PALOH) trial: a protocol for a prospective observational implementation trial’22

5. World Health Organization report: Childhood Hearing Loss: Strategies for Prevention and Care23

6. Public Health England guidance: Newborn Hearing Screening Programme (NHSP): Care Pathways for Babies in Neonatal Intensive Care Units (NICU). 24

Pathway for the current standard of care

A simple structure of clinical pathway for the current standard of care is shown in Figure 4. In the current standard pathway, neonates with suspected infection or sepsis will receive an aminoglycoside, such as gentamicin, irrespective of whether they have the MT-RNR1 gene m.1555A>G mitochondrial genetic variant. The current pathway also considered the administration of antibiotics to women during labour who are at risk of early-onset neonatal infection. The risk factors for early-onset neonatal infection in women in labour are set out in box 1 in NICE guideline 195.

FIGURE 4.

Clinical pathway for the normal standard of care.

Owing to the inheritance pattern of MT-RNR1, the current pathway included mitochondrial mutation screening for neonates with a mitochondrial mutation, maternal history of deafness, or both, who need aminoglycoside prescription. The current pathway also considered the findings of any previous genetic test to determine an antibiotic prescription. For example, children with cystic fibrosis are tested for the variant once they are identified as having cystic fibrosis as it is expected that these individuals will require aminoglycoside antibiotics at some stage in their lives.

Pathway when using the Genedrive MT-RNR1 ID Kit

A simple structure of clinical pathway for the integration of the Genedrive MT-RNR1 ID Kit into the clinical pathway is shown in Figure 5. As with the current standard of care, inheritance data and previous genetic tests for mothers with relevant risk factors are considered when deciding whether or not to prescribe aminoglycoside to neonates with suspected infection or sepsis. Although the time taken to administer the Genedrive MT-RNR1 ID Kit is short (26 minutes), for some of the neonates who present with suspected infection or sepsis there is insufficient time to use the Genedrive MT-RNR1 ID Kit as they are in immediate need. This issue was discussed by clinical experts consulted by the EAG. Their view was that using the Genedrive MT-RNR1 ID Kit may cause a delay for some neonates; however, ˂ 5% of neonates will need immediate antibiotics.

FIGURE 5.

Clinical pathway when using the Genedrive MT-RNR1 ID Kit.

As shown in Figure 5, antibiotic prescription for neonates will be based on Genedrive MT-RNR1 ID Kit results, with aminoglycosides being prescribed only if the test results are negative. As also shown in Figure 5, there is the possibility that the Genedrive MT-RNR1 ID Kit will be conducted for a second time if the first test fails. If both the first and the second test were to fail, then there would be no more time for any extra tests within the ‘golden hour’ for the administration of the antibiotics. Clinical experts consulted by the EAG noted that, in this situation, neonates with suspected infection or sepsis would almost certainly be provided with other alternative antibiotics owing to the need to ‘play it safe’.

Developing an economic evaluation model

In the following subsection, we outline the structure and key assumptions for a full economic model. The proposed economic model seeks to capture the components of the care pathways described above and also to consider the long-term implications of preventing AIHL for a child presenting with suspected infection or sepsis who has the m.1555A>G variant. Figure 6 provides a schematic (albeit simplified) representation of this model. In this model the key long-term implications considered are those that follow AIHL.

FIGURE 6.

Schematic outline of the Markov model.

The model was developed in accordance with standard modelling guidelines. 25,26 The face validity of the economic model structure was checked by our clinical experts and methodological experts for appropriateness to the current NHS clinical and diagnostic pathways. The model’s calculations and proposed data inputs were also checked for technical correctness.

The model simulates the patient pathway from the initial diagnosis of neonates with the m.1555A>G gene variant to treatment for AIHL (e.g. a cochlear implantation) for a patient’s lifetime. As per NICE scope, the population defined in the model is neonates with suspected infection or sepsis. The patient pathway described by the Markov model involves a series of mutually exclusive health states between which a patient may move over time (Figure 6). Once someone is in a state, they stay in that state for a defined period of time called the cycle length. We have defined a 1-year cycle length, as it was thought that year would be sufficient to capture both cost and effectiveness impacts in the model.

Each Markov model includes at least one absorbing state. This is a state that a person can enter but cannot leave. In the context of a chronic disease, the absorbing state might be death. In our model, the probability of movement to death was informed by the UK National Life Tables. 27 All programming for the model was implemented in TreeAge Pro 2022 (Williamstown, MA, USA).

Set out below are some key features of the proposed economic model:

• The population modelled is neonates with early-onset and late-onset infection who need antibiotic treatment and are being considered for treatment with aminoglycosides.

• Some neonates will require antibiotic administration immediately (i.e. there is no time for the Genedrive MT-RNR1 ID Kit to be used before antibiotics must be started).

• Increased time to antibiotics will increase the risk of death for neonates with sepsis.

• The clinical pathways for neonates with early-onset and late-onset infection are different (in terms of how long antibiotics are prescribed for).

• There is a chance the Genedrive MT-RNR1 ID Kit will give a false-negative result.

• There is a chance the Genedrive MT-RNR1 ID Kit will give a false-positive result.

• There is a chance that the Genedrive MT-RNR1 ID Kit will fail to give a result.

• If the first Genedrive MT-RNR1 ID Kit fails to give a result, there is time for a second test.

• If both the first Genedrive MT-RNR1 ID Kit and the second Genedrive MT-RNR1 ID Kit fail, there will be insufficient time for further testing and neonates with suspected infection will not be treated with aminoglycosides and receive other antibiotics (such as cefotaxime).

• An increased time to antibiotics will increase the risk of death for neonates from sepsis.

• Where neonates are identified as not having the m.1555A>G variant using the Genedrive test, aminoglycosides (e.g. gentamicin) will be used.

• Where neonates are identified as having the m.1555A>G variant using the Genedrive test, alternative antibiotics (e.g. cefotaxime) will be used.

• Different antibiotics will have different adverse event profiles.

• For neonates with the m.1555A>G variant treated with aminoglycosides, there is a risk of AIHL.

• For neonates with the m.1555A>G variant not treated with aminoglycosides, there is a risk of nonsyndromic hearing loss.

• For neonates who experience hearing loss, the severity of this may vary.

• Women with risk factors (for sepsis) are eligible for the Genedrive MT-RNR1 ID Kit, but antibiotic prescription will be for neonates (after birth).

• Maternal inheritance may be considered before testing.

• The use of the Genedrive MT-RNR1 ID Kit, other than affecting time to administration and the type of antibiotic used, does not affect normal standard of care for neonates presenting with suspected infection or sepsis.

• There will be training costs for staff to carry out the test, which will vary by the size of type of hospital ward.

• Staff time is required to carry out the test, which will vary by the size and type of hospital ward.

• Additional audiological monitoring will be required for infants with AIHL.

• AIHL has associated adverse events.

• If AIHL occurs, neonates will require hearing aids, unilateral cochlear implants or bilateral cochlear implants.

• HRQoL will vary by age, level of hearing loss, type of cochlear implant and time since cochlear implant has been implanted.

• To demonstrate no adequate benefit from hearing aids, children need to have had a valid trial of an acoustic hearing aid for at least 3 months.

• There are pre-procedure, procedure and post-procedure costs associated with both unilateral and bilateral cochlear implants.

• There is a chance that the cochlear implant surgery will not be successful.

• It is possible to upgrade cochlear implants after they have been fitted.

• There may be complications associated with the implantation of cochlear implants (i.e. internal or external device failure, death).

• There are short-term and long-term adverse events associated with cochlear implants, such as dysgeusia and vertigo, that will impact both costs and utilities.

In the next sections, information on the health state utilities and costs required to populate the model is set out. As is described below, not all of these are used in the early economic model.

Utility values

The utility values used in the early economic model are based on those used in Bond et al.,30 a highly cited NIHR Health Technology Assessment investigating the effectiveness and cost-effectiveness of cochlear implants for severe to profound deafness in both children and adults. The study by Bond et al. 30 was the health economic evaluation submitted as part of TA166 (‘Cochlear implants for children and adults with severe to profound deafness’),44 which was subsequently updated in TA566. 45 These utility values are shown in Table 4 and further described below.

The utility values for profound hearing loss, unilateral cochlear implants and bilateral cochlear implants for children used in Bond et al. 30 are taken from Barton et al.,29 a cross-sectional study in which the parents of a representative sample of hearing-impaired children assessed the HRQoL of their children using the HUI3. The HUI3 is the HRQoL measure considered to be the most sensitive to the effects of hearing treatment on overall health status. 37 As reported in Bond et al.,30 the utility increment from cochlear implants in childhood will vary by time since implantation and whether the child has a unilateral or a bilateral cochlear implant, and therefore different utility values are provided for ‘less than 2 years since implant’, ‘2–4 years since implant’ and ‘over 4 years since implant’.

The utility value for no hearing loss in childhood is taken from Pogany et al.,46 which is the source of the HUI3 population norms reported on the website of the HRQoL tool. 47 Pogany et al. 46 report the HUI3 population norms for the Canadian general population by age band. 46 The value of 0.908 is a weighted average of the 5–12, 13–15 and 16–19 age bands. As this value is taken from the Canadian value set, there are likely to be small differences between the health preferences from Canada and those from the UK, impacting the generalisability of this utility value. However, the HUI3 was the measure used in the Barton et al. 29 study and there is no UK value set for the HUI3. It is worth noting that for all child utility values used in the early economic model it is assumed that the values for those aged ≥ 5 years generalise to those < 5 years. This is clearly a strong assumption.

The adult utility values for profound hearing loss, unilateral cochlear implants and bilateral cochlear implants used in Bond et al. 30 are taken from a UK Cochlear Implant Study Group study37 that estimated the cost-effectiveness of unilateral cochlear implants for deaf adults using the HUI3. It is worth noting that these utility values were also used in the recent Cutler et al. 32 study, which investigated the cost-effectiveness of unilateral cochlear implants in UK adults. The utility value of being profoundly deaf was estimated to be 0.433. There are utility increments associated with both unilateral (0.630) and bilateral (0.633) cochlear implants. It is worth noting that a recent network meta-analysis of both UK and non-UK studies estimated the utility increment of bilateral cochlear implants compared with unilateral cochlear implants to be 0.08,48 slightly higher than the 0.03 increment reported in Bond et al. 30 and used in the early economic model.

The utility value for no hearing loss (the adult population norm) was estimated to be 0.850, the HUI3 population norm value for adults reported in Pogany et al. 46 Once more, as this value is taken from the Canadian HUI3 value set it is unlikely to be fully representative of the UK population given the differences in health preferences between countries. However, the HUI3 is the HRQoL measure used in the UK Cochlear Implant Study Group study and there is no UK value set for the HUI3.

It has previously been shown that HRQoL decreases with age. 49 As argued in Bond et al.,30 using a single age-independent value for the utility increment associated with cochlear implants may result in a counterintuitive position whereby the utility of a cochlear implant recipient may be higher than that of their normal-hearing peers. As this is an EVA, aside from varying the utility values by time of implementation in childhood, age-adjustment has not been considered in the early economic model. In a definitive study, age-adjustment should be implemented in line with modelling good practice guidelines and NICE guidance. 20,25,26

Adverse event disutility values

As noted in Cutler et al.,32 there are adverse events associated with the implementation of cochlear implants that may be included in an economic model. The disutility values associated with these adverse events and the probability of these adverse events are shown in Tables 5 and 6. The disutility values used in Cutler et al. 32 and their duration are sourced from a number of previously published health preference studies,50–52 The probabilities of the adverse events used in Cutler et al. were sourced from a series of clinical studies reporting complications associated with cochlear implants. 53–56

Given the relatively short duration of many of these events (with the exception of long-term vertigo), the relatively low probability of occurrence (Table 6) and the relatively low cost of these adverse events (as shown in Table 7), the disutilities and costs associated with adverse events are not included in the early economic model. In a definitive study, the disutilities and costs associated with adverse events should be included in line with standard methods guidelines. 20,25,26 The data in Table 5 on utilities and probabilities in Table 6 suggest that adverse effects that may be included in a definitive economic model may have only a negligible impact on the overall conclusions regarding cost-effectiveness.

Non-staff costs of diagnostic test

Using the information provided by the test manufacturer and information gathered from various other sources (including NHS reference costs and the Unit Costs of Health and Social Care), the cost of implementing the Genedrive MT-RNR1 ID Kit was micro-costed (Tables 8 and 9). The work reported in this subsection addresses the first objective for the cost-effectiveness set out in Objectives.

The costs of the diagnostic test were assumed to include:

• cost of the Genedrive MT-RNR1 ID System (GS-002)

• cost of the Genedrive MT-RNR1 ID Kit

• cost of the Genedrive MT-RNR1 control kit

• cost of a Bluetooth printer

• cost of custom labels

• annual warranty fee for the Genedrive equipment.

Capital costs of the Genedrive MT-RNR1 ID System and Bluetooth printer were calculated using the equivalent annual cost methodology. 60 This method converts the initial capital cost into an annual sum that equals the resources and investment plus their opportunity cost. The equivalent annual cost of implementing the Genedrive MT-RNR1 ID Kit was calculated under the following assumptions:

• lifespan of the Genedrive MT-RNR1 ID System and Bluetooth printer: 6 years

• capital costs spread over its lifespan (6 years)

• weeks per year in use: 52 weeks

• Genedrive MT-RNR1 ID Kit usage: three times per day

• annual warranty fee

• discount factor of 3.5% (in line with NICE reference case).

Following a request for information from the manufacturer, the lifespan of the Genedrive MT-RNR1 ID System was assumed to be 6 years (Table 8). In documentation provided by the manufacturer, the company recommends running a positive and negative control (both contained in a single Genedrive MT-RNR1 control kit) once per month to confirm that the Genedrive MT-RNR1 ID System is working correctly. It was therefore assumed that each site would undertake the recommended quality control using the control kit once per month. It was also assumed that each site would purchase a Bluetooth printer to print labels (together with a charging cradle) and that custom labels provided by the company would also be purchased. It was further assumed that lifespan of the Bluetooth printer would also be 6 years, in line with the lifespan of the Genedrive MT-RNR1 ID System. As specified by the manufacturer, the Genedrive System has been designed to be easily integrated into a NICU and does not need special storage for either the Genedrive MT-RNR1 ID System itself or the Genedrive MT-RNR1 ID Kit, and therefore it was assumed that no costs were associated with modifying existing infrastructure to accommodate the system.

It was assumed that the Genedrive MT-RNR1 ID Kit would be in use throughout the year. However, estimating the test usage at a site level is complicated by the fact that usage will be determined by the size, type and geographical location of each site. In the NICE MIB document for Genedrive it was assumed there are approximately 90,000 annual admissions to NICUs for neonates with suspected infection in the UK. 21 Given that there are currently estimated to be 72 level 3 NICUs in the UK,61 this indicates that the average number of eligible admissions per NICU per day may be between three and four. In the PALOH study,19 751 neonates were recruited from two centres in an 11-month period (January–November 2020). Owing the COVID-19 pandemic, the majority (n = 713, 94.9%) of these admissions were from a single centre, giving an average number of admissions to the participating site per day of between two and three.

Given the information from both the MIB document and the PALOH study,19,21 in the early economic model, it was assumed that the Genedrive MT-RNR1 ID Kit was used three times per site per day. It is worth emphasising that this estimate is subject to a significant level of uncertainty, given that the use of the equipment per site could vary markedly. However, it is also worth emphasising that because the Genedrive MT-RNR1 ID System itself is a relatively inexpensive medical device, the cost per neonate tested of the Genedrive MT-RNR1 ID System would be negligible over the lifetime of its use even in very small sites, and therefore should not materially impact the cost-effectiveness results.

Staff costs

There are significant staffing requirements in NICUs, with NICE quality standards stating that the minimum standard should be 1 : 1 nursing for all neonates. 62 Additional time for nursing staff to be trained and undertake the diagnostic testing will have cost implications.

Cost of standard of care

Although there is no current standard care for MT-RNR1 testing in neonatal sepsis, expert opinion and company information suggests that pyrosequencing and Sanger sequencing are the two closest comparators in the NHS. 21 The estimated total costs of pyrosequencing and Sanger sequencing are shown in Table 10. These are retrospective investigations of the cause of hearing loss. Given the uncertainty regarding current standard care, in the economic model it was pragmatically assumed that Sanger sequencing was used, as this was the sequencing method used to confirm the results from the Genedrive MT-RNR1 ID Kit in the PALOH study. 19 Given the relatively small difference in the costs between pyrosequencing and Sanger sequencing, this assumption is likely to have little impact on the results from the early economic model. As well as being used retrospectively in standard care to confirm the cause of hearing loss, it was also assumed that an investigation of hearing loss would also be used retrospective to confirm any positive results from the Genedrive MT-RNR1 ID Kit.

Costs of antibiotics

The implementation of the Genedrive MT-RNR1 ID Kit will have an impact on the antibiotics given to neonates. The first-choice antibiotic regime for the empirical treatment of suspected early-onset infection (˂ 72 hours) is intravenous benzylpenicillin with gentamicin. The starting dosage of this antibiotic regime is 5 mg/kg every 36 hours administered in a single dose. A second dose may be given after 36 hours. A shorter interval can be used if clinical judgement suggests that this is needed. According to the BNF,63 the price of a single vial of benzylpenicillin is between £3 and £4 and the price of a single vial or ampoule of gentamicin is between £1 and £3 depending on the specific brand. For late-onset infection, the first-choice antibiotic regime is a narrow-spectrum antibiotic, such as intravenous flucloxacillin with gentamicin. The starting dose of this antibiotic regime is 50 mg/kg every 6–12 hours. According to the BNF,61 the price of a single vial is between £1 and £4 depending on the specific brand.

If m.1555A>G were to be detected, Clinical Pharmacogenetics Implementation Consortium guidance13 recommends that the use of aminoglycosides be avoided unless the infection is very severe and there is a lack of safe or effective alternative therapies. Therefore, alternative antibiotic therapies would be administered. Alternative antibiotic therapies include cefotaxime and amoxicillin, with the exact antibiotic regime used depending on local antimicrobial guidelines. In the PALOH study, when an infant was identified as carrying m.1555A>G, they were prescribed cefotaxime, which is considered to have comparable antimicrobial coverage to benzylpenicillin with gentamicin. 19 The starting dosage of cefotaxime is 50 mg/kg administered in a single dose. According to the BNF, the price of a single vial is between £2 and £4 depending on the specific brand. 63

As this is an EVA and the various antibiotics that may be used are relatively inexpensive, the antibiotic costs were not included in the early economic model, as their impact on the cost-effectiveness was predicted to be negligible. In a definitive study these costs should be included.

Costs of testing for hearing loss

As part of the NHS newborn screening programme, all neonates should be screened within 26 days of birth for possible hearing loss. 24 An automated otoacoustic emissions test is commonly used in the first instance. If the results are not clear, a second automated otoacoustic emissions test may be conducted, or an auditory brainstem response test may be used. Clinical experts commented that babies with AIHL may have discordant results and that, therefore, all those with a known m.1555A>G variant should be referred for immediate follow-up and additional audiological monitoring. The exact health resource requirements for this additional audiological monitoring are unclear. As all neonates are assumed to be screened as part of the NHS newborn screening programme, the costs of attending the newborn screening programme and the associated automated otoacoustic emissions and auditory brainstem response tests are not included in the economic analysis. Moreover, as the additional monitoring of those with a known m.1555A>G variant is not predicted to differ between current standard care and the proposed care pathway with the Genedrive MT-RNR1 ID Kit, these costs are also not included in the economic analysis.

Costs of hearing aids and cochlear implants

Estimation of cost-effectiveness

Using the data set out in Tables 16–19, two estimates of cost-effectiveness were produced:

• incremental cost per case of AIHL avoided

• incremental cost per QALY gained.

For each point, costs and effects for neonates that either follow current standard of care or are tested with the Genedrive MT-RNR1 ID Kit for detecting single nucleotide polymorphism m.1555A>G were estimated. From these incremental costs, effects and incremental cost-effectiveness were calculated.

Deterministic sensitivity analysis

We also conducted a deterministic sensitivity analysis to explore the uncertainty regarding key parameters in the early economic model, using the sensitivity analysis values shown in Tables 16–19. This sensitivity analysis focused on costs, QALYs and incremental cost per QALY only. Owing to the uncertainty regarding the majority of the parameters in the early economic model, a probabilistic sensitivity analysis was not implemented.

For the prevalence of the m.1555A>G variant, the high and low values used in the sensitivity analysis were the 95% CIs from Bitner-Glindzicz et al. 4 For the sensitivity and specificity values, the high and low values used in the sensitivity analysis were the 95% CIs reported in the PALOH study. 19 For the utility values, the high and low values for the sensitivity analysis were the 95% CIs from the original studies from which the values were sourced.

For the parameter related to the probability of AIHL for neonates with the m.1555A>G variant prescribed with aminoglycosides, the lower bound estimate (0.3) was taken from Göpel et al., a prospective cohort study in a German population. 6 For the other parameters (including all of the cost parameters), reasonable high and low values were chosen to explore the potential uncertainty related to these parameters.

Base-case results

Using the parameters shown in Model parameters, the base-case results from the early economic model are shown in Table 20 for the cases of AIHL avoided and Table 21 for QALYs. In terms of AIHL, the results show that using the Genedrive MT-RNR1 ID Kit is estimated to be cost saving over the lifetime of the neonate tested for the m.1555A>G genetic variant with the Genedrive MT-RNR1 ID Kit.

In terms of cost of per QALY, the results show that the Genedrive MT-RNR1 ID Kit dominates the current standard of care over the lifetime, as it is less costly and more effective (Table 21).

Results of deterministic sensitivity analysis

The results of the deterministic sensitivity analysis are presented in a tornado plot (Figure 9). The tornado shows the impact of the high and low parameter values specified in Tables 14–18 on the estimated incremental cost-effectiveness ratio (ICER).

FIGURE 9.

Tornado plot of Genedrive MT-RNR1 ID Kit pathway vs. standard pathway. EV, expected value.

As shown in Figure 9, the parameter values that have the largest impact on the ICER are the time horizon of the model, the sensitivity of the Genedrive MT-RNR1 ID Kit, the probability of neonates with the m.1555A>G variant prescribed with aminoglycosides suffering from AIHL, the prevalence of the m.1555A>G variant across the population and the cost of the Genedrive MT-RNR1 ID Kit. As Figure 9 shows, varying other parameter values (e.g. the utility values associated with bilateral cochlear implants and the probability of cochlear implants being successful) did not appear to materially impact the incremental cost per QALY.

The sensitivity of the results to the time horizon reflects the fact that from an NHS resource use perspective, significant costs are required to identify one neonate with the m.1555A>G variant, while the benefits (specifically cost savings related to cochlear implants avoided and utility gains from avoiding AIHL) are likely to only be felt in the medium to long term. Table 22 illustrates this by showing the impact on cost-effectiveness of varying the time horizon between 1, 10 and 50 years. As shown in Table 22, although the Genedrive MT-RNR1 ID Kit has a very large ICER when compared with normal standard of care for a 1-year time horizon, the kit has an incremental cost per QALY of just over £100 when using a 10-year time horizon and dominates the current normal standard of care when using a 50-year time horizon.

The impact of the sensitivity of Genedrive MT-RNR1 ID Kit on the cost-effectiveness results reflects the fact that the real-world sensitivity of the Genedrive test (as reported in the PALOH study19) is highly uncertain due to the very small number of positive cases. This uncertainty was reflected in the reported wide CIs used as the high and low values in the deterministic sensitivity analysis.

The sensitivity of the results to the proportion of neonates with m.1555A>G variant experiencing AIHL after being exposed to aminoglycosides reflects the inherent uncertainty related to this parameter. As discussed in Prevalence of m.1555A>G variant and risk of aminoglycoside-induced hearing loss, although there is clear evidence that m.1555A>G variant is a risk factor for AIHL, most evidence comes from case–control studies that may overestimate this risk, and, therefore, the precise level of this risk is unknown.

With respect to the sensitivity of the results to the prevalence of the m.1555A>G variant among the population, this parameter affects how many neonates need to be tested to detect a single neonate with the m.1555A>G variant. As the probability increases, fewer neonates need to incur the cost of testing to detect a neonate with the variant, and hence the cost-effectiveness of using the Genedrive MT-RNR1 ID Kit improves. Although no data were available to consider how cost-effectiveness varied by different subgroups, this sensitivity analysis helps illustrate how cost-effectiveness might vary if testing were focused on subgroups in whom the m.1555A>G variant is more prevalent.

For the cost of the Genedrive MT-RNR1 ID Kit, the analysis made some simplifying assumptions about what costs were included. Although there is some variation in the incremental cost per QALY, the sensitivity analysis shows that including these costs would not substantially alter the cost-effectiveness results over the lifetime horizon.

Social media posts

The 92 social media posts centre around three descriptive themes: neonatal sepsis experiences, infection causing hearing loss, and hearing loss from gentamycin use.

Neonatal sepsis experiences

Families of infants who have had sepsis are using social media to:

• share experiences of neonatal sepsis, connect with other parents with shared experiences –

After a load of tests … sepsis, a week on two broad spectrum antibiotics …

… other mums of sepsis babies …?

• understand and express fear of perceived long-term consequences of sepsis

  • future infection rate and impact –

… still suffers a lot of viral infections and bacterial infections now aged 3 …

• potential side effects of antibiotic treatment on feeding and digestion –

… what is wrong with Lo and his feeding … One of the things that I’ve read can cause it is antibiotics in early life …

Infection (sepsis or meningitis) as a cause of hearing loss

Families of infants who have had sepsis or meningitis and now have hearing loss are using social media to:

• explore the aetiology of their child’s hearing loss

• share experience of difficulties finding child care –

… childminders are not getting back to me …

• share experience of difficulties using hearing aids –

… hair is now brushing the back of the aids I wonder if that is causing distress …

• express a need for earlier hearing loss testing after infection –

… trying to arrange a hearing test but … audiology department is not being particularly forthcoming …

Hearing loss and gentamicin use

Families of infants with hearing loss potentially due to gentamicin:

• express a preference for treatment with gentamicin, with a perceived trade-off between side effects and effective treatment –

… antibiotics are given with best intentions at the time, and … was very poorly so I would rather … had the antibiotics …

• have a lack of clarity on alternate treatment options and comparative effectiveness –

… I’m not sure if there are options for other antibiotics which are strong enough …

• want use of treatments with ototoxic potential to be after informed consent given –

… doctor should have advised and obtained informed consent …

• indicate a lack of clarity on safe dosing –

doctor told us that … didn’t receive ‘too much’ however we don’t know the impact of what … did receive

• indicate the utility of test to inform future treatment for individual and offspring –

have it (variant) … this done as could have issues if needed … these drugs again … could pass this to … children

• desire clarity on the aetiology of hearing loss

  • emotional support dealing with a lack of clarity about aetiology of hearing loss and moving on from search for a cause –

I haven’t thought about it for a long time … move on from finding a cause … is what it is …

• identify that testing for the variant may not give this –

tested negative so we were no further forward

Focus group discussion

Two overarching analytical themes emerged from focus group discussion: information need to inform parents decision to test and/or treat; and testing desirability depending on context.

Information need to inform parents decision to test and/or treat.

Information was wanted on:

• The prevalence of the variant (see Prevalence of m.1555A>G variant and risk of aminoglycoside-induced hearing loss, evidence available) –

what is the chance of having the variant where you might be predisposed to losing your hearing.

• The test’s accuracy and safety both in children (see Diagnostic test accuracy, limited evidence) and in mothers (Table 1, no evidence).

So I suppose I’m just asking how … safe is this test

… it would be a non-starter for me if there was any question on the accuracy of the test …

… how accurate it is when they test the mother …

• The chance of hearing loss developing after aminoglycosides are taken when the variant is not present (see Prevalence of m.1555A>G variant and risk of aminoglycoside-induced hearing loss, limited evidence) and when the variant is present (evidence not sought) –

… the risk of taking the initial drug and losing your hearing …

• The chance of spontaneous hearing loss with the variant (see m.1555A>G variant and nonsyndromic hearing loss (without exposure to aminoglycosides), limited evidence) and without the variant (evidence not sought). This links to social media comments on a desire for clarity about hearing loss aetiology –

… I didn’t have the numbers of what the chance of them having hearing loss was, I would want to know that chance …

• The chance of morbidity and mortality from infection and from sepsis, as well as if and how this changes based on time to treatment initiation –

… I’m presuming that within that hour the quicker, the better it is even within that hour …

What other risks are there then? What other repercussions for your child?

… so, chances of like the long-term health effects increase the longer you wait?

• The longer-term health risks of infection and sepsis –

What are the other long-term or like health risks of having?

• The comparative effectiveness and tolerability of antibiotics (see Place of intervention in current pathway: treatment for neonatal infections and sepsis, limited evidence). This links with the lack of clarity expressed about treatment options and comparative effectiveness in social media posts –

I’d want to know the differences between the two drugs … taking the second one, does that increase the chance of mortality?

… why wouldn’t you just use the second antibiotic, whatever that might be, if that was equal …

You want to know which outweighs the other …

• The certainty in aetiology of hearing loss after testing. This links with social media data on the lack of clarity that testing for the variant may give about hearing loss aetiology –

Obviously they do a hearing test on newborn babies not long after they’ve been born. How do they know that it’s a drug that caused the hearing loss …?

Clinical effectiveness

Only one study (reported in two publications) met the eligibility criteria for our rapid review. 18,19 The study reported on the following outcomes of interest: diagnostic test accuracy, number of successful tests for neonates, test failure rate, impact of the results on care decisions, impact of test implementation and use on healthcare resources (e.g. the time taken to do and interpret the test), time to obtaining a sample for a test, time to results, time to antibiotic treatment, and number of neonates identified with the m.1555A>G genetic variant. However, it did not report on the following outcomes of interest: successful test of mothers, usability of the test, mortality and morbidity.

The diagnostic test accuracy of the Genedrive MT-RNR1 Kit was high, with no false negatives reported. However, estimates of real-world sensitivity of the test lacked precision, as only three participants with the m.1555A>G variant were identified in the study. 19

There were five false positives, which was suggested to have been rectified by updating the cartridge used in the machine. 19 Similarly, the Genedrive MT-RNR1 ID Kit was adapted to reduce the test failure rate. After this correction, in a laboratory-based setting there were no failed tests; however, with the intended point-of-care a failure rate of 5.7% was still observed. 19 Three neonates were successfully identified as carrying the genetic variant, leading to aminoglycoside antibiotics being avoided and alternative cephalosporin-based regimens being provided. 19 The time to results for the Genedrive MT-RNR1 ID Kit was consistent with predefined boundaries of statistical equivalence with standard care (mean difference –0.87 minutes, 95% CI –5.96 to 4.23 minutes). This finding justified the simplification of the early economic evaluation model, which did not address the impact of time to antibiotics on cost-effectiveness.

Regarding the usability of the test, the analysis provides an actionable result in 26 minutes, with an estimated time of approximately 30 minutes from the collection of the buccal swab to an actionable result (i.e. genetic variant detected or not detected). However, the time to obtaining a sample can vary (median 6 minutes, IQR 3–16 minutes).

Overall, these results suggest that the Genedrive MT-RNR1 ID Kit has promise as an accurate point-of-care diagnostic test. In addition, it has the potential to provide rapid identification within the time-sensitive period required to impact treatment decisions about neonates with the m.1555A>G genetic variant.

Cost-effectiveness

No existing economic evaluations were identified that addressed the topic of this study.

The costs of the Genedrive MT-RNR1 ID Kit were estimated using information provided by the company and assumptions made by the EAG. Considering the equipment needed to carry out the test, it was estimated that it would cost approximately £102 per diagnostic test inclusive of capital costs. It was also estimated that 30 minutes of nurse time would be needed to carry out each diagnostic test, raising the estimated total cost per diagnostic test to approximately £130. This estimate is subject to considerable uncertainty given the different types of hospital wards in which the Genedrive MT-RNR1 ID Kit could potentially be used. Although estimated to be relatively inexpensive at an individual level, the rarity of the m.1555A>G genetic variant means that the costs of identifying one neonate with this variant are more substantial. For example, under the strong assumptions of a test with perfect diagnostic accuracy, an estimated prevalence of the m.1555A>G genetic variant of 0.002 and an estimated cost of £130 per test, the cost to identify one neonate with the variant would be £65,000.

If the Genedrive MT-RNR1 ID Kit were to be recommended for use in clinical practice while further data are being collected (potentially to address some of the evidence gaps identified as part of this EVA), the sunk costs to the NHS would include the Genedrive System itself (£4995) and a Bluetooth printer (£200) for each site, the bulk purchasing of the Genedrive MT-RNR1 ID Kits and any accumulated training costs for healthcare professionals to carry out the test. As discussed previously, the training costs even for large sites with large numbers of nursing staff are likely to be relatively minor given the predicted short time of training and the fact that this could be provided free by the manufacturer. Estimating the total sunk costs at a national level is very difficult given the uncertainty regarding the type and numbers of sites that would potentially use the Genedrive MT-RNR1 ID Kit System. However, if each of the reported 55 level 3 NICUs in England was to purchase the Genedrive MT-RNR1 ID System and a Bluetooth printer, the total costs would be approximately £280,000. 61

The EAG developed an early economic model to identify the key drivers of the cost-effectiveness of the Genedrive MT-RNR1 ID Kit for detecting single nucleotide polymorphism m.1555A>G in neonates, and also to identify evidence gaps. A targeted literature review was undertaken to identify utility values relevant to the specific population. A detailed care pathway and model were constructed, and the evidence requirements were defined. There are several key evidence gaps that exist, including the magnitude of risk of AIHL in neonates with m.1555A>G, the risk of hearing loss for neonates with m.1555A>G genetic variant without exposure to aminoglycosides, the proportion of neonates potentially requiring different types of cochlear implants, and how data regarding maternal inheritance may potentially be used in the clinical pathway.

The early economic model was constructed to explore introducing the Genedrive diagnostic test into NHS services. This model focused on the likely key determinants of costs and QALYs. These key areas were arrived at by considering the likely impact on average costs or QALYs for the two clinical pathways (with Genedrive MT-RNR1 ID Kit and current standard care) and focusing on these parameters that might have the greatest impact on cost-effectiveness. This model was then subject to one-way deterministic sensitivity analysis to explore the impact of changes in these key parameter values. Of note is that some of the changes explored indirectly assessed whether the omission of an element could affect the model result. For example, in the early economic evaluation model the costs of training have been omitted, but the cost of the test has been varied in the sensitivity analysis over a range that would include the cost of testing had the training costs been included.

Overall, the base-case results from the early economic model suggest that the use of the Genedrive MT-RNR1 ID Kit could potentially be cost-effective, mainly driven by the high diagnostic accuracy reported in the PALOH study, the estimated relatively low cost per test and the avoidance of high future healthcare costs associated with the fitting of cochlear implants in those infants experiencing AIHL. From a deterministic sensitivity analysis conducted as part of the early economic model, the results were most sensitive to the time horizon of the model (which allows more time for the health benefits that flow from avoiding the accrual of hearing impairment), the sensitivity of the Genedrive MT-RNR1 ID Kit, the probability of neonates with the m.1555A>G variant experiencing AIHL after being exposed to aminoglycosides, and the prevalence of the genetic variant among the population. This suggests that research to identify more robust data on the sensitivity of the Genedrive MT-RNR1 ID Kit, the risk of AIHL for those with the m.1555A>G variant exposed to aminoglycosides and the prevalence of the m.1555A>G variant in the UK would be particularly useful. Given the limitations of the early economic model, these results are not sufficient to enable decisions about adoption, but they are suggestive that the generation of new data may be useful.

Clinical effectiveness

There are several limitations to the current evidence base. First, there has been only one study of the use of Genedrive MT-RNR1 ID Kit. This study was conducted in two specialised large NICUs, although one NICU did drop out of the study, and therefore it is unclear if these findings can be generalised to smaller neonatal units, especially as there is limited evidence on the implementation and use of healthcare resources associated with the test kit. Additionally, some infants are born and evaluated for infection in other venues that may not have access to this technology. 74

Second, the test was refined during the PALOH study (e.g. to reduce failure rate). Although there is preliminary evidence to suggest the benefits of these more recent iterations, further studies are needed to confirm that the reduced failure rate for the test can be replicated in different settings.

Third, our rapid review identified no studies investigating the clinical effectiveness or cost-effectiveness of the Genedrive MT-RNR1 Test Kit in mothers (pre birth of the neonates). Therefore, we have no evidence to inform the use of the test kit in this population. Mahmood and Leung75 have argued that it would be theoretically possible to pre-test expectant mothers with the Genedrive MT-RNR1 ID Kit. Clinical advice received by NICE also highlighted this potential application for the technology. This could be especially important when families are anticipating neonatal antibiotics based on a peripartum diagnosis, meaning that those neonates who were excluded from the PALOH study because they required antibiotics immediately19 could be included. 75

Fourth, prior to conducting the PALOH study19 there were some concerns regarding ethical challenges, such as testing prior to informed consent, and the burden of responsibility placed on the practitioner and the wider societal impacts of technology, such as the Genedrive MT-RNR1 ID Kit. 76 However, the test is relatively simple, needing only a buccal swab,77 and would only be used in an intended point-of-care healthcare setting. 78 Additionally, the point-of-care test could be considered integral to the broad package of care offered to the unwell neonate, for which broad parental consent is provided, or, if this is unavailable, is done in the best interest of the child. 79

Fifth, some have raised concerns regarding the antibiotics being chosen as a result of the point-of-care test. Specifically, gentamicin is suggested to be the gold-standard treatment for neonates with suspected/confirmed sepsis, and giving second-line care could increase the risk of death. 80 This concern was also raised in our focus group with parents (see Chapter 6). Therefore, there are alternative views on whether using the Genedrive MT-RNR1 ID Kit to identify the m.1555A>G genetic variant is required. However, our clinical advisors suggest that this risk of death due to the antibiotics given is low.

Finally, other variants in other genes that result in the same risk phenotype are currently not assessed by the current test. 74

Cost-effectiveness

Further limitations can be highlighted for the economics analyses. Foremost among these is that, given the lack of data, the results generated from the early economic evaluation are highly uncertain. For example, in the base-case analysis of the early economic model, the sensitivity and specificity of the test are assumed to be 100% and 99.2%, respectively, despite the significant uncertainty associated with the sensitivity value. This assumption was explored in the deterministic sensitivity analysis, which showed that getting more accurate data is likely to be highly important to the cost-effectiveness results. Other relevant data that are highly uncertain include the precise proportion of neonates with the m.1555A>G genetic variant who will suffer from hearing loss both with and without the prescription of aminoglycosides, the staffing requirements in different sizes and types of hospital wards where the Genedrive MT-RNR1 ID Kit may be implemented and the how the kit could be used to test mothers rather than neonates.

Second, several parameters have effectively been omitted from the early economic evaluation model. These include training costs, the costs of antibiotics, and the impact (in terms of both costs and utilities) of long-term adverse events related to cochlear implants. The justification for this is that it was felt that the cost per patient of adding these costs would have a minimal impact on the cost-effectiveness results; however, this may not be the case for all parameters. For example, there is uncertainty surrounding the proportions of neonates who experience some degree of hearing impairment who go on to use normal hearing aids or require unilateral/bilateral cochlear implants. The results of the sensitivity analysis show that the cost-effectiveness results could be sensitive to changes in the costs of these, and, by extension, the results will be sensitive to the proportions receiving each form of hearing aid. These omissions would need to be addressed in a full economic evaluation.

Third, other data used in the economic model may not be strictly relevant to the NICE reference case. For example, although the long-term utility data related to deafness and cochlear implants used in the early economic model are sourced from a pivotal study, these are based on data from the HUI3 rather than the EQ-5D. 30 This is potentially justifiable as the HUI3 directly captures the impact on hearing that the EQ-5D does not. A bolt-on for hearing for the EQ-5D is under development but not yet available and of course would not relate to NICE’s preferred source of utility values. The data for the HUI3 were taken from a study that was completed around 20 years ago, and technology related to cochlear implants has improved significantly since then. The utility values for the HUI3 are also approaching 30 years old and are derived from a Canadian population. It is also unclear how applicable they are to children under 5 years of age, as the HUI3 is validated only for those ≥ 5 years old. There is a scarcity of validated HRQoL instruments suitable for infants. Although the Infant health-related Quality of life Instrument (IQI) has recently been developed, none of the seven attributes is related to hearing, and, again, the tool is not compatible with the NICE reference case. 81,82

Fourth, related to the cost perspective, the economic model has thus far considered NHS costs only. There are likely to be costs needed that fall within Personal Social Services relating to other aids and adaptations. Furthermore, there may be broader societal impacts on children and families (e.g. increased caring responsibilities; possible impacts on the speech development and educational attainment of the child) that also need to be considered.

Finally, the scope of the economic evaluation may be too narrow in terms of capturing the broader implications of integrating the test into the clinical pathway. These are likely to vary substantially according to the centre. In addition, there may be some further impacts on laboratory testing. This is uncertain as the number of neonates presenting with suspected infection or sepsis is itself uncertain. The model also has not considered the impacts on antibiotic resistance. Clinical experts have advised that there are strong clinical concerns regarding resistance to alternative antibiotic therapies than may be prescribed instead of aminoglycosides, such as cefotaxime. Therefore, the incidence of antimicrobial resistance in the healthcare setting could be improved if testing reduces the use of these alternative antibiotics. 12 How this latter impact could be captured in a definitive model is, however, unclear.

Diagnostic accuracy and failure rate of test

There are several uncertainties regarding the accuracy of the Genedrive MT-RNR1 ID Kit. The NICE scope included testing mothers of neonates who may require antibiotics. We found no studies using the Genedrive MT-RNR1 ID Kit in this population.

Although the PALOH study provides data on the diagnostic accuracy of the Genedrive MT-RNR1 ID Kit, estimates of the test’s sensitivity are severely limited by the small number of neonates (only three) identified with the m.1555A>G genetic variant. 19 Therefore, although the test may be potentially very sensitive, the 95% CI was wide (mean sensitivity 100%, 95% CI 29.2% to 100.00%), indicating very high imprecision. As the early economic evaluation shows, estimates of cost-effectiveness are very sensitive to this imprecision.

The failure rate was originally 17.1% (90 of 514 neonates), but after modifications to the assay buffer and a redesigned cartridge consumable this was reduced. In a laboratory setting the failure was zero, while in a clinical setting this was reduced to 5.1%, when repeated testing was conducted. 19 This suggests that when the kit is used as the point-of-care test there is still a failure rate, and more than one test may be required. As the test is reported to take 26 minutes, this could cause issues with antibiotic prescribing that is required to be within the hour. Therefore, further work is required to ascertain the number of test failures that may be expected and the impact of this on prescribing. The early economic modelling suggested that this evidence gap would have only a modest effect on cost-effectiveness.

Clinical outcomes

We did not identify any studies assessing mortality or morbidity. Future studies should look to ascertain the effects of using the Genedrive MT-RNR1 ID Kit and the implications of these effects on these clinical outcomes.

Generalisability of results

The NICE scope included NICU and hospital wards. The PALOH study provided evidence that the Genedrive MT-RNR1 ID Kit did not substantially impact on time to antibiotics compared with standard care,19 a key outcome given the need to provide antibiotics within 1 hour of the decision to treat. However, the PALOH study was conducted in two large and well-resourced NICUs; therefore, some of our clinical experts indicated that it was unclear whether data from time to antibiotics in this study generalise to smaller NICUs and other hospital wards.

Estimating the magnitude of risk for aminoglycoside-induced hearing loss in neonates with m.1555A>G

There is clear evidence that the m.1555A>G genetic variant is a risk factor for AIHL. 13 However, data on the magnitude of this risk are uncertain. Case–control studies usually find that all people with the variant experience hearing loss. 3 However, selecting participants for hearing loss may overestimate the risk associated with aminoglycosides. Studies that do not select for hearing loss suggest that people with the m.1555A>G variant may not always experience hearing loss when exposed to aminoglycosides. 6 The early economic evaluation suggests that cost-effectiveness estimates are likely to be sensitive to the magnitude of this risk.

To understand the long-term benefits (on clinical outcomes and cost-effectiveness) of avoiding aminoglycoside use in neonates with the m.1555A>G genetic variant, more precise estimates on the magnitude of risk for AIHL in this population are required. It is also worth noting that there is an evidence gap related to hearing loss in those without exposure to aminoglycosides. However, the results from the sensitivity analysis in the early economic model indicate that this may not be important.

Maternal inheritance and use of point-of-care testing in mothers

The m.1555A>G variant is inherited maternally, and therefore identifying a mother’s m.1555A>G status may be another way of identifying a child’s m.1555A>G status. However, there are several uncertainties and evidence gaps related to this. First, there in uncertainty about how well a mother’s m.1555A>G status can indicate the risk of AIHL in her child. Although there are studies related to the variant load and hearing thresholds, these studies have small sample sizes. Second, there is uncertainty about the proportion of mothers in whom a clinically relevant genotype has previously been identified. Finally, in relation to using point-of-care testing for the mother, it is unclear what proportion of women are likely to be given aminoglycosides during labour or in other clinical settings.

Resource implications

The PALOH study provided evidence that no increase in nursing time was required to implement the assay into practice. 19 As stated previously, the PALOH study was conducted in two large and well-resourced NICUs that have significant experience of conducting research. There is, therefore, a clear lack of evidence related to the resource implications in different-sized NICUs and different hospital wards where the Genedrive MT-RNR1 ID Kit may potentially be used in the future.

Estimating the severity of aminoglycoside-induced hearing loss in neonates with m.1555A>G and quantifying the proportion of neonates requiring different types of cochlear implants in the long term

Alongside the uncertainty related to the risk of AIHL following exposure to aminoglycosides, there is uncertainty related to the severity of AIHL in those who experience it. Consequently, there is uncertainty about the proportion of neonates with AIHL who would require different cochlear implants of different types over the long term. This is important, as cochlear implants have significant NHS resource implications, with substantial costs related to surgery and annual maintenance and programming. Current NICE guidance states that cochlear implants are recommended for children (and adults) with severe to profound deafness who do not gain adequate benefit from acoustic hearing aids. 45 Severe to profound deafness in this case is defined as hearing only sounds that are louder than 80 decibels hearing level. In children, adequate benefit is defined as speech, language and listening skills appropriate to age, developmental stage and cognitive ability. It is currently unclear what proportion of neonates would require acoustic hearing aids only, unilateral cochlear implants and bilateral cochlear implants, making estimates of the long-term savings associated with preventing cochlear implants highly uncertain.

Utility values for health states related to hearing loss and cochlear implants that conform to the National Institute for Health and Care Excellence reference case for use in an economic model

As noted in Table 3 (see Results of the targeted search for health-related quality-of-life studies), the majority of previous economic evaluations related to hearing loss and/or cochlear implants in the UK population identified in this EVA used the HUI3 HRQoL tool to measure health state utilities. The main reason for this is that the HUI3 has a specific dimension related to hearing, which other commonly used HRQoL tools, such as the EQ-5D and SF-6D do not. Indeed, the HUI3 has been shown to have better validity and responsiveness than the EQ-5D and SF-6D in previous studies of patients with hearing impairments. 83

However, although the HUI3 may have some advantages over the EQ-5D in respect to validity and responsiveness in this clinical area, the EQ-5D is NICE’s preferred measure in the reference case. Furthermore, the value set used for the HUI3 is from a Canadian adult sample and is over 20 years old. Research is ongoing regarding the development of a hearing ‘bolt-on’ for the EQ-5D; however, the measurement properties of this are yet to be established. 43

Risk and severity of aminoglycoside-induced hearing loss in people with m.1555A>G variant

Further validation of Genedrive MT-RNR1 ID Kit

Assessment of clinical outcomes

Measurement of utilities associated with hearing loss, hearing aids and cochlear implants using preference elicitation techniques to validate existing values and use in economic model