INVESTIGATE-1 Invasive evaluation before surgical treatment of incontinence gives added therapeutic effect?): A mixed methods study to assess the feasibility of a future randomised controlled trial of invasive urodynamic testing prior to surgery for stress urinary incontinence in women

Article history

The research reported in this issue of the journal was funded by the HTA programme as project number 09/22/136. The contractual start date was in January 2011. The draft report began editorial review in October 2013 and was accepted for publication in August 2014. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The HTA editors and publisher have tried to ensure the accuracy of the authors’ report and would like to thank the reviewers for their constructive comments on the draft document. However, they do not accept liability for damages or losses arising from material published in this report.

Declared competing interests of authors

The authors have no current personal financial interests; the following non-financial interests are declared: Dr Hilton was a member of the National Institute for Health and Care Excellence (NICE) Interventional Procedures Advisory Committee (2002–7); a member of the National Institute for Health Research (NIHR) Evaluation, Trials and Studies Co-ordinating Centre (NETSCC) Health Technology Assessment (HTA) Therapeutic Procedures Panel (2007–8) and the NETSCC-HTA Clinical Evaluations and Trials Prioritisation Group (2008–10); chair of the NICE development group for clinical guideline on urinary incontinence in women (2004–7); and a member of the James Lind Alliance Working Partnership on research priorities on urinary incontinence (2007–9). These last two groups identified the research question underlying this report as an important area for further research. Dr Lucas currently chairs the European Association of Urology Guidelines panel on Urinary Incontinence which has produced guidance on the use of urodynamics in clinical practice based on existing evidence. Professor McColl is a member of the NIHR Journals Library Editorial Group although she was not involved in the editorial processes for this report.

Copyright statement

© Queen’s Printer and Controller of HMSO 2015. This work was produced by Hilton et al. under the terms of a commissioning contract issued by the Secretary of State for Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK.

Prevalence of urinary incontinence

Urinary incontinence (UI), while rarely life-threatening, may seriously influence the physical, psychological and social well-being of affected individuals. 1–4 The impact on the families and carers may be profound and the resource implications for the health service considerable. 5 Prevalence figures for UI range from 5% to 69% in women aged 15 years and older, with most studies in the range 25–45%. 6 More severe UI is reported in 4–7% of women aged under 65 years, and around 5 million women over 20 years of age in England and Wales may be affected. 7

Although absolute prevalence rates vary widely, the distribution of UI subtypes appears more consistent, with stress UI (SUI) or mixed UI (MUI) accounting for 65–85% of cases. 8 Isolated SUI accounts for approximately half of all incontinence, with most studies reporting 10–39% prevalence; MUI is the next most common, with prevalence figures of 7.5–25.0%; isolated urgency UI appears to be relatively uncommon, with 1–7% prevalence. 6

Costs of urinary incontinence and investigation

A study of UI across 14 European countries reported the mean annual per capita UI-related costs to range from €359 in the UK/Ireland (for patients predominantly treated in primary care) to €515 in Germany and €655 in Spain (for patients treated by specialists). 9 A systematic review of the costs associated with UI and overactive bladder (OAB) similarly found the annual per capita costs to vary considerably between individual studies and countries, with the highest reported being in institutionalised individuals in the USA at US$9872. 10 A UK study using 1999/2000 prices estimated the annual cost to the NHS in England of treating clinically significant UI in women at £233M, with total annual service costs (including costs borne by individuals) of £411M. 11

Several methods are used in the assessment of UI to guide management decisions, and invasive urodynamic tests (IUTs) may form part of this. Essentially these investigations evaluate functional aspects of the lower urinary tract; cystometry, the most commonly used IUT, looks at the pressure/volume relationships during bladder filling, storage and emptying, with a view to defining a functional diagnosis as distinct from a purely symptomatic one. The costing report associated with the National Institute for Health and Clinical Excellence [now, the National Institute for Health and Care Excellence (NICE)] clinical guideline on UI used an estimated charge of £176 for each IUT (2006/7 English national tariff), and calculated the annual national cost of urodynamic investigations as over £22M. 12 From this, the potential saving from not undertaking urodynamic investigations before conservative treatment was estimated at approximately £3M. 12

Changes in available operative techniques and, in particular, the introduction of less invasive approaches such as mid-urethral tapes, have resulted in dramatic alterations to surgical practice in recent years. 13 Hospital Episode Statistics (HES) demonstrate a 50% increase in surgery for SUI in the 10 years following the introduction of mid-urethral tapes in 1997, with numbers apparently plateauing at 11,000–13,000 procedures annually in England between 2006/7 and 2012/13. 14 The NICE costing report estimated further savings of £321,000 from more rational use of IUTs before surgery, although this is perhaps a conservative estimate being based on ‘current use’ of 70% (the actual figure is probably closer to 100%) and ‘future use’ of 50%. 12 A more realistic estimate of annual savings based on 2012/13 national tariff costs (£403 per procedure for Healthcare Resource Group LB42Z)15 and HES activity data would be approximately £3.3M. There would also be an additional ‘opportunity cost’ saving from the alternative use of staff and equipment currently devoted to invasive urodynamic testing. It remains to be demonstrated, but should be recognised, that this saving may come at no detriment to health and with the avoidance of what some women undoubtedly see as unpleasant and embarrassing procedures.

Existing literature on clinical utility of invasive urodynamic tests prior to surgery

Urodynamic tests comprise a group of investigations used to evaluate function of the lower urinary tract; some of these are invasive (requiring catheterisation) and some are non-invasive. The tests are most often used for diagnosis, planning of appropriate intervention and prediction of treatment outcome, although they can also be used repeatedly to monitor the progress of disease over time or as outcome measures in clinical research. While cystometry is the most commonly used IUT, videocystometry and ambulatory bladder pressure monitoring are used by some. The current position of invasive urodynamic testing in the diagnostic pathway is not agreed and practices vary considerably: in a UK survey in 2002, only half of the units surveyed had guidelines on indications for the tests and 85% carried out cystometry in all women with incontinence. 16 Current guidance from NICE suggests that cystometry is not required prior to conservative treatments for UI, or prior to surgery where the diagnosis of SUI is clear on clinical grounds [i.e. where there are no symptoms of OAB or voiding dysfunction (VD), no anterior compartment prolapse and no previous surgery for SUI]. 17,18

The National Institute for Health and Care Excellence, National Institute for Health Research (NIHR) Health Technology Assessment (HTA), The Cochrane Collaboration and the International Consultation on Incontinence (ICI) have each recently undertaken systematic reviews on the subject of urodynamics and called for further high-quality primary research confirming clinical utility. 17,19–23 The specific aim of the current study is to assess the feasibility of a future large randomised controlled trial (RCT) to address a key research recommendation of the NICE and Cochrane reviews of the subject. The clinical utility of invasive urodynamic testing was also among the top prioritised uncertainties identified within the James Lind Alliance Urinary Incontinence Priority Setting Partnership in 2008. 24,25

A decision-analysis study from the USA failed to find support for invasive urodynamics before surgery in women likely to have SUI. 26 A similar economic assessment within the NICE report on UI, using assumptions more applicable to current NHS practice, found that for every 10,000 patients assessed there would be approximately 13 additional cures using invasive urodynamics, at an additional cost per cure of £26,125. With a ‘willingness-to-pay’ threshold of £20,000 per quality-adjusted life-year (QALY), each cure would have to generate 1.3 QALYs for invasive urodynamics to be considered cost-effective. 17 Based on a gain of QALYs of 0.07 per annum for a woman cured compared with a woman not cured,27,28 this would require each cured woman to survive 19 years post treatment (assuming QALYs are not discounted); given that typical women undergoing surgical treatment for SUI are in their mid-40s (range 20s to 70s), their average life expectancy would be much greater than this, suggesting that invasive urodynamic testing may be cost-effective.

One small RCT showed no significant benefit from cystometry prior to conservative treatment, although interpretation is difficult, given that the control (not investigated) group in this study received both bladder retraining and pelvic floor muscle training (PFMT), whereas the intervention (cystometry) group received either bladder retraining or PFMT. 29 In a cohort study from the North Thames region, women were no more likely to benefit from incontinence surgery if they had undergone preoperative urodynamic testing,30 and a study of Medicare patients in the USA found that those who had preoperative testing appeared more likely to develop urge incontinence after their surgery. 31 A secondary analysis of data from a US randomised surgical trial found that preoperative investigation did not predict failure32 or postoperative VD. 33

Other studies ongoing during protocol development

Post funding, but during the refinement of the protocol for INVasive Evaluation before Surgical Treatment of Incontinence Gives Added Theraputic Effect? (INVESTIGATE-I), the investigators became aware of two other trials looking at the clinical utility of urodynamics in similar patient groups. One was from a multicentre group in the Netherlands [Value of Urodynamics prior to Stress Incontinence Surgery (VUSIS-1); www.controlled-trials.com/mrct/trial/385179/urodynamic], the other from the US Urinary Incontinence Treatment Network [Value of Urodynamic Evaluation (ValUE); www.controlled-trials.com/mrct/trial/472073/urodynamic]. 34 Both of these were full trials using a non-inferiority design. VUSIS-1 did not specifically define a non-inferiority margin, although the sample size was determined from a power of 70% using less than 5% difference between groups; this trial was terminated prematurely due to slow recruitment after achieving only 23% (59/260) of its planned accrual. 35 ValUE defined a non-inferiority margin of 11% (equivalent to a standardised difference of < 0.8), which we consider too high, that is we would look on a difference in outcome between groups of 11% as being clinically quite important and one that might potentially influence the decisions of both clinicians and patients. 36

In the ValUE study, women with a clinical diagnosis of SUI or stress predominant MUI, who also have clinically demonstrable stress leakage (i.e. a slightly different patient group from INVESTIGATE-I), were randomised to either no further assessment or to undergo urodynamic investigation (as in INVESTIGATE-I). In view of the recruitment difficulties with VUSIS-1, the Netherlands group proceeded to a further study of alternative design (VUSIS-2; www.controlled-trials.com/mrct/trial/474127/vierhout),37 in which all women underwent invasive urodynamic testing, and only those with discordant clinical and urodynamic findings were randomised between surgical treatment (as dictated by their clinical assessment) and individual treatment (dictated by the combination of clinical and urodynamic results); neither participants nor health-care professionals involved were blinded to the urodynamic results in either group.

The primary outcome in ValUE and both VUSIS studies was based on the Urogenital Distress Inventory (UDI) score at 12 months (ValUE used a 70% reduction in UDI along with a Patient Global Impression of Improvement score of ‘very much better’ or ‘much better’ as indicative of treatment success). Although we preferred the use of international standard outcomes as intended by the ICI Modular Questionnaire (ICIQ) as our primary outcome, we subsequently chose to include the UDI as an additional secondary outcome38 to facilitate easier comparison of results between these various studies and the incorporation of our results, even from this feasibility study, into a meta-analysis.

Each of these studies has been published during the period of recruitment and follow-up in INVESTIGATE-I;35,36,39 their results are discussed later in this report. How much they have already influenced clinical opinion and practice or will do so in the future is unclear, although a ‘point-counterpoint’ debate published after these studies (in 2013) makes it clear that there is still a question to be answered. 40,41 The most recent update of the Cochrane review of urodynamics for the management of UI in children and adults continues to emphasise the need for larger definitive trials, in which people are randomly allocated to management according to urodynamic findings or to standard management based on history and clinical examination. 42

Rationale for an initial feasibility study and pilot trial

Although NICE, NIHR HTA, The Cochrane Collaboration and the ICI have all called for large high-quality primary research to establish the clinical utility of invasive urodynamic investigations, there were several reasons to conduct a pilot trial and feasibility assessment before undertaking a definitive trial.

First, the sample size for a definitive trial was considered using estimates and assumptions from the modelling exercises cited above,17,26 and from a previous surgical trial. 43,44 However, such calculations are very sensitive to parameter values such as the proportion of recruits with SUI,26 the proportions of poor outcomes in the two arms and the effect size (target difference) of interest; currently available information is insufficient to plan a study that could be expected with reasonable certainty to produce robust results. Our own very preliminary sample size calculations gave figures between 1100 and 6700 per treatment arm. Since designing the feasibility study, the most recent Cochrane review of urodynamics in adults and children indicates that a similarly large sample size would be required to address this question. 42

Given the possible size of a definitive trial on this question therefore, a feasibility study was considered crucial to test assumptions made, give relevant estimates of key parameters and inform power calculations for the definitive trial.

Second, invasive urodynamic testing has been widely used in clinical practice over the last 30 years and, despite the lack of evidence of clinical utility, many clinicians look on cystometry as a mandatory part of the investigation of patients with UI, particularly prior to surgical treatment. 45–47 A survey of members of the British Society of Urogynaecology (BSUG) has shown a high level of disagreement with the NICE guidance in this respect,48 and others have questioned the safety of the recommendations. 49 We were aware that, although the ValUE study completed recruitment,36 the investigators encountered initial problems with lack of clinician equipoise (Peggy Norton, University of Utah Health Care, 2010, personal communication). Hence we needed to establish whether or not sufficient clinicians were in equipoise and willing to enrol and randomise patients within a definitive trial.

Finally, patients may not so easily see the importance of ‘testing a test’ in the same way as they might view testing a treatment. Indeed, they are willing and often keen to undergo investigation (even when this is invasive),50 in the belief that this will inevitably guide them and their clinicians towards appropriate treatment and away from inappropriate and possibly harmful interventions. Two HTA-funded trials of radiography for low-back pain were only able to recruit 23% and 51% of patients who were approached to enter the randomised arms. 51,52 The VUSIS-1 study was terminated prematurely when it had achieved only 23% of its planned recruitment. 35 Hence it was necessary to investigate patients’ willingness to take part in a RCT of this particular diagnostic test and to identify barriers to, and facilitators of, participation.

Overall, therefore, while we were encouraged that other researchers have similarly seen this topic as an important clinical uncertainty and have sought to undertake trials of similar design to that proposed in INVESTIGATE, we remained of the opinion that a feasibility study was an important step before embarking on a definitive trial using public funds.

It was recognised that a pilot RCT alone was probably inadequate to address the complexities of the determination of feasibility for a definitive trial in this aspect of health care. While most mixed-methods studies to date have been limited to combining qualitative methods and RCTs,53 we developed a protocol comprising a national survey of relevant clinicians, qualitative interviews with both trial participants (face to face) and clinicians (telephone), a randomised external pilot trial and a nested health economic analysis. Post hoc additions to the protocol included an update to the original clinician survey and a questionnaire to those identifying potential trial participants [research nurses and principal investigators (PIs)] regarding issues of screening sensitivity.

Specific objectives

The objective of the proposed future definitive trial is to address the question of whether or not invasive urodynamic testing compared with basic clinical assessment with non-invasive testing alters treatment decisions and outcomes in women suitable for surgical treatment of SUI or stress predominant MUI. The outcome measures proposed would include the quantification of post-treatment urinary leakage, impact on general health and condition-specific quality of life (QoL), adverse effects from investigation or treatment and health economic outcomes. Thus, in a possible future definitive trial, it might be established whether or not invasive urodynamic testing should indeed be offered to all women prior to surgery.

The objective of the current feasibility study (INVESTIGATE-I) was to inform the decision whether or not to proceed to such a definitive RCT and whether or not any refinements to the design or conduct of that trial are warranted.

Study components

A mixed-methods approach was chosen to assess the feasibility of a future definitive RCT. There were five components to the study, each addressing different aspects of the overall determination of feasibility:

1. A pragmatic multicentre randomised pilot (external or rehearsal pilot) trial (see Chapter 3). This was designed to rehearse the methods and processes of a future definitive randomised trial. As such, it evaluated patient identification strategies, recruitment numbers and patients’ willingness to be randomised. The rate of retention within the study and the effectiveness of outcome measures in terms of response and completion rates were also evaluated. The pilot was also designed to provide outcome data to inform sample size calculations for a future definitive trial.

2. A full economic evaluation undertaken within the above pilot RCT (see Chapter 4). The pilot study rehearsed the data collection for the economic evaluation, which included health state utilities and costs to the NHS and patients. To inform the definitive economic analysis, the pilot study assessed consistency of resource use in administration of the IUT and other tests, surgical and non-surgical treatments, and the ease of access to information from hospital databases about resource use. It also piloted the use of data collection instruments.

3. National online surveys of clinicians’ views about urodynamics (see Chapter 5). In order to assess the extent of ‘buy-in’ to a future definitive trial, the survey questionnaires explored surgeons’ views about the necessity for urodynamic investigations in a range of clinical scenarios and also their opinion about the importance of the research question underlying the INVESTIGATE studies. Since it was anticipated that a future robust trial would require a sample size very much larger than previous studies and seemed likely to need the involvement of a large number of units, clinicians’ workload in incontinence surgery and their willingness to randomise their own patients in a definitive trial was also assessed. A brief second survey was undertaken towards the end of the study to assess changes in clinical opinion over time as a result of other publications in the area. 35,36,39

4. Qualitative interviews with a subset of surgeons (see Chapter 6). The interview topic guide used here sought to illuminate the questionnaire responses from component 3 above. This complemented the results of the survey and explored further how clinicians use the results of IUTs to inform their decisions. The interview data were used to explore the differences between personal and community equipoise and the effect these may have on willingness to randomise patients into a future trial; they were also used to investigate some of the sociological aspects of diagnostic tests and, in particular, how clinicians approach a test that is widely used but lacking evidence of clinical utility.

5. Qualitative interviews with a subset of women eligible for the pilot trial to assess their experiences of the study (see Chapter 7). By approaching those who did and did not agree to participate, we sought to define the reasons behind these decisions. The interview topic guide was designed to facilitate exploration of patients’ experiences of being approached to take part in the trial, their perceptions of the study information sheets and the burden associated with study outcome questionnaires.

The methods employed, results obtained and key messages from these different study components are described separately in Chapters 3–7; discussion is combined in Chapter 8, and the overall consideration of feasibility is presented in Chapter 9. The latest version of the protocol is available on the NIHR Journals Library website. The report is made in line with the Consolidated Standards of Reporting Trials (CONSORT) statement;54 the CONSORT diagram for the randomised pilot trial is given as Figure 5; the CONSORT checklist is shown in Appendix 1.

Methods

This was a pragmatic multicentre randomised external (rehearsal) pilot trial to assess patient recruitment and willingness to be randomised, rehearse trial methods and processes, and provide outcome data to inform sample size calculations for a future definitive trial. All of these were considered important elements of the determination of feasibility.

Subsequent treatment within the trial

Following investigation, it would be expected that women randomised to the control (no IUT) arm of the study, i.e. those treated on the basis of clinical assessment and non-invasive tests (documented on CRF – ‘visit 2’ – see Appendix 19e), would undergo surgical treatment (documented on CRF – ‘visit 4′ – see Appendix 19g) (Figure 1). Given the pragmatic nature of the study, the choice of operation was left to the individual surgeon and patient; as only primary cases were included, it was anticipated that this would be either a retropubic or transobturator foramen mid-urethral tape procedure in most cases. Those randomised to the intervention (IUT) arm, i.e. undergoing invasive urodynamic testing (documented on CRF – ‘visit 3′ – see Appendix 19f), had similar surgical treatment when urodynamic stress incontinence (USI) was confirmed (documented on CRF – ‘visit 4’). Where other diagnoses were identified following investigation, alternative treatments might be offered (documented on CRF – ‘visit 5′ – see Appendix 19i); these included bladder retraining, anti-muscarinic drug treatments, neuromodulation, botulinum toxin injections (where DO was diagnosed), or clean intermittent self-catheterisation (where a VD was identified). Exactly which of these interventions was chosen depended on what conservative treatments had been used before entry into the trial; for example, if a woman had tried PFMT plus bladder retraining before entry, she was likely to be offered anti-muscarinic drug treatment if DO was shown on invasive urodynamic testing. In all centres the treatment algorithm employed was in keeping with the then current NICE recommendations (2006). 17 In some cases where mixed abnormalities were reported, women would first undergo one or more of these interventions (to stabilise bladder overactivity, or improve voiding efficiency) and then proceed to surgery for SUI. After the participant entered the study the clinician remained free to recommend alternative investigation or treatment to that specified in the protocol at any stage if they felt it to be in the participant’s best interest. In these cases the participant remained in the study for the purposes of follow-up and data analysis.

FIGURE 1.

Diagram of the study design and the flow of participants. a, The choice of non-surgical treatments is left to the clinician and patient, but may include bladder retraining, drugs, neuromodulation, botulinum toxin injections, and clean intermittent catheterisation, depending on IUT results, local protocols and previous trials of therapy. NAD, no abnormality detected; USI, urodynamic stress incontinence.

Any adverse events (AEs) or serious adverse events (SAEs) were documented in the CRF (see Appendices 20 and 21); SAE notification was faxed to the NCTU within 24 hours.

Statistical analysis

Given that this was a pilot trial, the statistical analysis was largely descriptive in nature and provided estimates of key trial parameters to inform the design of the future definitive trial. Screening and recruitment numbers were summarised in a CONSORT diagram. In addition, screening numbers were summarised by centre and recruitment numbers were summarised by month and centre. Results were reported at baseline and 6-month follow-up time points. Data analysis was by intention to treat.

Categorical variables were summarised as percentages per category by treatment arm. Questionnaire scale and subscale totals and continuous variables were summarised by mean and standard deviation (SD) and 5-number summaries [median, interquartile range (IQR) and range] by treatment arm and time point. The burden of missing data were summarised by response rates for each variable. No data imputation was attempted for any outcome [other than in the economic evaluation (see Chapter 4)]. The summary statistics for the primary outcome measure were combined with the target/minimum clinically important difference (MCID) and recruitment, retention and response rates to inform the sample size for a future definitive trial.

Results

Screening

Overall, 771 patients were identified by research nurses from clinic notes and correspondence as being potential recruits into the study, and were sent the PISs. Of those screened, 284 were deemed eligible for the trial, giving a ‘screen positive’ rate of 37%. The reasons for non-eligibility of screened patients are shown in Table 2; most commonly these were patients not having undergone supervised PFMT prior to referral (14%), urgency or urgency predominant MUI (12%), failure to attend clinic appointments (11%), patients not wishing to participate (8%), patients with prolapse requiring treatment (5%), or clinicians feeling that surgery was not appropriate (5%). Although the reasons for non-eligibility varied between centres, the overall figures were obviously heavily weighted by the centre screening the highest number of patients. In some units, patients not wishing to participate made up a larger proportion of screening failures; overall however, 78% of eligible women identified were recruited into the study.

TABLE 2 - Screening and recruitment numbers by centre including screening codes (1–15), for those women not randomised, sorted by overall frequency of reporting of codes

Code	Description	Newcastle	Gateshead	Wansbeck	Leicester	Swansea	South Tees	Sheffield	Total	Per cent
11	Patient has not undergone a course of PFMT	74	10	15	4	2	0	0	105	14
14	Urge incontinence	85	2	5	0	0	0	0	92	12
13	Other (give details)	52	5	17	8	2	0	2	86	11
15	Did not attend clinic	54	24	2	0	1	0	0	81	11
7	Patient does not wish to participate, include reason if offered	13	9	10	10	14	0	3	59	8
1	Symptomatic uterovaginal prolapse requiring treatment	22	0	8	4	1	5	0	40	5
8	Clinician feels surgery inappropriate	1	26	1	2	1	8	0	39	5
9	Patient does not wish surgery	5	11	4	0	1	0	0	21	3
2	Previous surgery for UI or POP	7	0	1	0	0	0	1	9	1
3	Urodynamic investigation within the last 3 days	5	1	1	0	0	0	0	7	1
10	Patient does not consider her family is complete	4	2	0	0	0	0	0	6	1
4	Neurological disease causing UI	1	0	0	0	0	0	0	1	0
5	Current involvement in a conflicting research study	0	0	0	0	0	0	0	0	0
6	Unable to give competent informed consent	0	0	0	0	0	0	0	0	0
12	Study not discussed at clinic visit (please give reason)	1	0	2	0	0	0	0	3	0
	Recruited	75	50	37	20	17	15	8	222	29
	Total screened	399	140	103	48	39	28	14	771	100
	Per cent of screened women recruited	19	36	36	42	44	54	57	29
	Per cent of eligible women recruited	84	85	76	67	55	100	73	78

The numbers screened at individual centres varied between 14 and 399; the percentage of eligible women recruited varied between 55% and 100%, but did not show an obvious trend with the number screened (see Table 2). Although a single code was assigned to each patient, it is possible that codes were used variably in the different centres, and that there may have been some inconsistency or overlap in the use of codes. For example, it is possible that ‘patient does not wish to participate’ could overlap with ‘patient does not wish surgery’. While the centres screening larger numbers of women also recruited larger numbers (Figure 2), the conversion from screening to recruitment decreased as the screening number increased (Figure 3).

FIGURE 2.

Numbers screened and recruited at individual centres.

FIGURE 3.

Number and percentage recruited to trial by number screened (shown on log scale) at each centre. The graph plots the percentage and number of participants recruited. The number of participants is indicated by black filled circles and the percentage of participants by green open circles.

Recruitment

Monthly recruitment by centre is shown in Table 4 and Figure 4, for the initial recruitment period (up to the end of March 2012) and for the period of extension (from April to December 2012). Regulatory requirements took approximately 3 months longer than anticipated and, as a result, recruitment targets were revised. Even once all approvals were in place, and all sites in a position to start recruitment, the rate of accrual was significantly less than required with some sites unable to identify any patients for some weeks after opening to recruitment. Although proposed in 2011,74 the NIHR 70-day benchmark for recruitment was not published until 2012 and was not a required of CLRNs until after 2013. 75 Nevertheless, several steps were introduced to improve recruitment, including the incorporation of additional clinicians on two of the existing sites and the establishment of an additional full recruiting site and two PIC sites. A request was made for a 9-month unfunded extension to the recruitment period.

TABLE 4 - Monthly recruitment numbers by centre. Original and revised predictions of recruitment are shown to the left of the table and actual recruitment by centre to the right; horizontal bars in individual site columns denote the date at which recruitment could have started (i.e. when all regulatory requirements were met)

Year.Quarter	Period	Date	Predicted recruitment			Actual end of month recruitment by site								Totals
			Original	First revised	Second revised	Newcastle	Gateshead	Wansbeck	South Tees	PIC sites	Leicester	Swansea	Sheffield	Monthly	Cumulative
1.2	Initial recruitment period	1 April 2011	0
1.2		1 May 2011	20
1.2		1 June 2011	40
1.3		1 July 2011	60	0		0	1							1	1
1.3		1 August 2011	90	20		1	6					0	0	6	7
1.3		1 September 2011	120	40		11	7	1			0	0	0	12	19
1.4		1 October 2011	150	60		12	8	1			0	0	0	2	21
1.4		1 November 2011	180	90		14	8	1			0	0	0	2	23
1.4		1 December 2011	210	120		20	8	2			2	1	0	10	33
2.1		1 January 2012	240	150	38	20	8	4			3	3	0	5	38
2.1		1 February 2012		180	54	23	13	8			10	3	0	19	57
2.1		1 March 2012		210	70	26	18	12			11	5	2	17	74
2.2		1 April 2012		240	87	26	24	16			13	6	2	13	87
2.2	+2 months	1 May 2012			104	34	27	18			13	9	4	18	105
2.2		1 June 2012			121	41	35	20		0	16	10	4	21	126
2.3	Extension period +7 months	1 July 2012			138	47	35	21		0	18	10	4	9	135
2.3		1 August 2012			155	51	40	28	0	0	19	11	4	18	153
2.3		1 September 2012			172	55	41	28	0	0	19	13	4	7	160
2.4		1 October 2012			189	59	43	31	5	0	19	14	6	17	177
2.4		1 November 2012			206	62	45	31	10	0	20	14	6	11	188
2.4		1 December 2012			223	66	47	33	11	0	20	16	7	12	200
3.1		1 January 2013			240	75	50	37	15	0	20	17	8	22	222

FIGURE 4.

Monthly total recruitment numbers. The original and revised predictions of overall recruitment are shown as continuous and dashed lines, and actual recruitment in histogram; the overall ‘BRAG’ flag or ‘RtT’ status is also illustrated.

The number of participants recruited per recruiting month (i.e. between the completion of all site-specific regulatory requirements and the end of the study) varied between 0.4 and 3.9 per month at the original sites (mean 1.9); at the additional full recruiting site this figure was 2.5 per month; the PICs did not identify any potentially eligible patients for referral to a recruiting site in the 8 months that they were active.

Randomisation

Of the 284 women screened positive, 222 agreed to randomisation into the trial, giving a trial consent rate of 78%. This recruitment total (222) represented 93% of the planned sample size (240) for the pilot trial. Overall, 110 women were randomised to the control or no IUT arm and 112 to the intervention or IUT arm. Immediately after randomisation it became apparent that one woman in the no arm was ineligible for the trial and she was withdrawn leaving a total of 221 eligible patients randomised (109 in the no IUT arm and 112 in the IUT arm).

The screening, recruitment, randomisation and trial follow-up are summarised in the CONSORT diagram shown as Figure 5.

FIGURE 5.

Trial CONSORT flow diagram. DNA, did not attend.

Retention

Demographic data and details of any subsequent treatment for incontinence were collected from hospital notes and CRFs (see Appendices 19a–k), and women were asked to complete questionnaires on clinical outcomes (see Appendix 17) and a 3-day bladder diary (see Appendix 18) at baseline and 6 months after the start of treatment (i.e. 6 months after the date of surgery, or the start of any non-surgical intervention, or period of ‘watchful waiting’). Baseline questionnaires were sent to 219 women and returned by 165; this represented a response rate of 75% overall, 72% in the IUT arm and 79% in the no IUT arm. At 6 months after treatment, questionnaires were returned by 63% (125/200) of those who were sent questionnaires at follow-up; 56% (54/97) in the IUT arm and 69% (71/103) in the no IUT arm.

Six women returned a completely blank questionnaire booklet (three in each study arm); one further woman in the IUT arm completed only the EQ-5D and SF-12 questionnaires, but for the purpose of return of primary outcomes this was categorised as returning a blank questionnaire, as the ICIQ-FLUTS was not completed. This information is summarised in the trial CONSORT diagram Figure 5. Six of the seven women who returned blank questionnaires reported ‘no significant urinary symptoms’ on the follow-up CRF (visit 6). The same six either annotated the front of their questionnaire or bladder diary, or in one case telephoned the NCTU indicating that they had not had urinary problems since their surgery. One of the women who returned a blank questionnaire reported ‘significant urinary symptoms’ on the follow-up CRF; she also annotated her diary to indicate that there had been little change in her urinary symptoms following her surgery, although she improved slightly with subsequent drug treatment.

The progress of recruitment and follow-up is shown in Figure 6. It also shows the anticipated follow-up at 6 months, although these predictions do not make allowance for individual centre waiting times for investigation and surgery; this was certainly an error that would require attention in planning a future definitive trial. The actual times at which follow-up questionnaires were posted out to participants (at 6 months after surgery or start of treatment) do reflect these waits, and illustrate an average additional delay to follow-up of approximately 4 months. Figure 6 also illustrates the actual rate at which follow-up questionnaires were received back at the NCTU. At the time of closure of the database for final analysis, 125 follow-up questionnaires had been received (exceeding the target of 120), although as per the CONSORT diagram, seven of these omitted primary outcome data – ICIQ-FLUTS total score.

FIGURE 6.

Actual recruitment with anticipated and actual receipt of follow-up questionnaires. Lines illustrate actual monthly recruitments, anticipated follow-up (based on 6 months after recruitment), anticipated follow-up (based on questionnaires posted – 6 months after treatment), and actual follow-up (outcome questionnaires received back).

Questionnaire data

Baseline

Table 6 shows the distribution of the questionnaire scales at baseline by trial arm. The ICIQ-FLUTS total score has a possible range of 0–48. The distribution of ICIQ-FLUTS total score at baseline was fairly symmetrical with a mean of 16.9 (SD 5.7) in the IUT arm and 16.4 (SD 6.3) in the no IUT arm. The distributions of the other scales and subscales were similarly well matched between the IUT and no IUT arms and were fairly symmetrical.

Six-month follow-up

Table 6 also shows the distribution of the questionnaire scales at 6-month follow-up by trial arm. The distribution of ICIQ-FLUTS total score at follow-up had a mean of 9.2 in the IUT arm and 6.9 in the no IUT arm. The distribution of ICIQ-UI SF (possible values 0–21) had a mean of 5.3 in the IUT arm and 3.3 in the no IUT arm. The distribution of ICIQ-LUTSqol (possible values 19–76) had a mean of 26.7 in the IUT arm and 25.3 in the no IUT arm. The distribution of UDI overall score (possible values 0–300) had a mean of 49.1 in the IUT arm and 33.9 in the no IUT arm. For all scales, typical scores were much lower than at baseline. The distribution of the ICIQ-FLUTS total scores at 6-month follow-up by trial arm is shown in the upper part of Figure 7. The shape of these distributions at 6 months was generally positively skewed, which reflects the fact that many women had experienced considerable relief from their initial symptoms, but some had not.

FIGURE 7.

Distribution of ICIQ-FLUTS total score at 6 months and changes between baseline and 6 months by trial arm. Graphs by randomisation group.

It is difficult to interpret any difference in mean scores between baseline and 6 months follow-up from Table 6, because many of the women who provided baseline data failed to do so at 6 months. Table 9 shows the distribution of the paired changes in scale scores for those women who had completed both questionnaires. It can be seen that the mean change in ICIQ-FLUTS total score was 7.8 in the IUT arm and 9.3 in the no IUT arm. The distribution of the change scores for the ICIQ-FLUTS total scores is shown in the lower part of Figure 7. Typically, there was a marked drop in these scores over 6 months, but little difference in the mean changes between the trial arms. This pattern was also seen in the other four scales. However, no formal comparison between arms is appropriate in a pilot study.

TABLE 9 - Summary statistics for paired changes in scale scores (baseline – 6 month)

Questionnaire	n	Mean (SD)	Median (IQR)	Range
IUT arm
ICIQ-FLUTS – overall score	31	7.8 (5.9)	7 (4–15)	–5–18
ICIQ-UI SF	34	8.9 (6.0)	11 (4–13)	–3–16
ICIQ-LUTSqol	29	20.0 (11.4)	23 (12–28)	–5–41
UDI – overall score	27	79.5 (45.5)	75 (51–122)	–21–161
No IUT arm
ICIQ-FLUTS – overall score	48	9.3 (7.3)	10.5 (5.5–15.0)	–9–22
ICIQ-UI SF	49	10.2 (5.8)	11 (6–15)	–4–21
ICIQ-LUTSqol	47	23.7 (13.9)	23 (14–35)	–3–50
UDI – overall score	41	94.1 (55.3)	92 (70–117)	–66–221

Adverse events and serious adverse events

Only two SAEs were reported. One woman in the IUT arm experienced bleeding from suburethral incision 12 days after surgery; she required readmission and vaginal packing. An operative vaginal injury had been identified and repaired primarily in the same woman (reported separately as an AE). One woman in the control arm developed breast cancer shortly after the operation and subsequently underwent a mastectomy. Both women had received their allocated treatment prior to the SAE; while one clearly related to the incontinence treatment, neither event was categorised as being related to the trial intervention (invasive urodynamic testing).

In addition, 23 AEs in 22 women were reported to the NCTU; these included three operative bladder injuries (3/185 = 1.6% perforation rate) and two vaginal injuries. Six episodes of urinary tract infection (UTI) were reported, two in the IUT arm, and four in the no IUT arm; all occurred following surgery, and none immediately after invasive urodynamic testing. Of the 22 patients in whom events were reported, 12 were randomised to the IUT arm and 10 to the no IUT arm; while most or all of these AEs could have been related to surgery, none were categorised as relating to the trial intervention itself (invasive urodynamic testing).

Key messages

• All the proposed trial processes and outcome measures likely to be required in a future definitive RCT of invasive urodynamic testing versus clinical assessment and non-invasive testing were effectively rehearsed within the pilot study.

• Greater clarity in the inclusion and exclusion criteria and an ‘assume eligibility’ approach might assist trial staff to identify potential recruits more appropriately.

• Thirty-seven per cent of women screened were eligible for inclusion in the trial and 78% of eligible women identified in each centre were recruited.

• Regulatory requirements took longer than anticipated. In addition, waiting times between initial assessment, trial recruitment, invasive urodynamic testing and admission for surgery varied between units. These delays would need to be more adequately addressed within the management plan.

• The recruitment numbers at individual centres ranged from 12% to 225% of the original planned centre targets, which will need to be considered in the definitive trial planning. The start up of an additional recruitment site improved recruitment, but establishment of PICs was not helpful in this particular study

• Regular communication through a range of media appears to have a positive effect on trial staff engagement, although the impact of this on recruitment is difficult to evaluate.

• Baseline questionnaires were completed by 75% of participants, although only 63% of those sent follow-up questionnaires (56% of those recruited) returned them at 6 months after start of treatment.

• A small number of participants returned blank follow-up questionnaires, although most of these included some annotation to indicate that the respondent was free from symptoms. Changes to the design of the booklets might obviate this problem in a future trial.

• Although the rate of return of questionnaires was lower than expected, missing data within the returned booklets were few. The ICIQ-FLUTS overall score could be calculated for 98% of subjects at baseline; ICIQ-UI SF, ICIQ-LUTSqol and overall UDI score could be calculated for 99%, 95% and 84%, respectively. At 6 months, not only were fewer questionnaires returned, but the completion rates were also slightly lower at 90%, 91%, 87% and 81%, respectively. We would rationalise the questionnaires used in a future trial.

• Bladder diaries were less often completed than questionnaire booklets; only 68% of the baseline diaries and 53% of those sent follow-up diaries at 6 months were returned. Although patterns of voiding could be ascertained from all diaries returned, only 65% at baseline and 40% at 6 months provided information on pad use. If bladder diaries were to be used in a future trial, modification to the recording of pad use should be considered.

• A small number of women elected to defer treatment, although 95% of women in the control arm underwent surgical treatment, compared with 80% in the IUT arm, reflecting changes in the management plan and the use of further non-surgical treatments following invasive urodynamic testing.

• Few AEs were recorded during the study; these were evenly spread across the study arms. Most were expected AEs related to treatment, and none were related to the trial intervention itself. The effectiveness of our detection of UTI following invasive urodynamic testing might be questioned, and should be modified for a future definitive trial.

• The pilot trial was a crucial element of the package of feasibility studies, and identified several important issues for the planning of a future definitive trial.

Methods

The economic evaluation rehearsed data collection and analyses to inform a definitive trial. In terms of data collection, we assessed the ease of collecting information and consistency of resource use in administration of the IUTs, other tests, surgical and non-surgical treatments, and piloted the use of economic data collection instruments. In terms of data analysis, a cost–utility analysis was performed where health state utilities for each participant were based on data obtained using self-administered SF-12 and EQ-5D-3L questionnaires completed by participants at baseline and at 6-month follow-up. Stochastic and deterministic sensitivity analyses were used to assess the importance of statistical and other uncertainties.

Results

There were 222 patients initially randomised to the pilot trial comparing the IUT arm and the no IUT arm. From these 222 patients, information on 218 patients was used in the economic analysis; 110 in the IUT arm and 108 in the no IUT arm.

Cost–utility analysis

As noted earlier, only an illustrative example of the cost–utility analysis is presented, which is meant to be exploratory and should be interpreted with caution for the following reasons:

• The study sample size was not powered for the results of analysis to be definitive.

• Information on non-invasive tests was collected for the feasibility study but was not used in the economic analysis. These tests can be performed on patients in both the IUT and no IUT arms so it is difficult to determine the bias, if any, that the exclusion of these tests has on the results of the economic analysis.

• Information on participants’ out-of-pocket costs was not included in the analysis.

• Micro-costing for the IUT was conducted based on information from one site, therefore, could be under/overestimating the costs of the IUT arm.

Further details below describing the extent of data available for the analyses illustrate why analyses based on these data are not sufficient to inform changes in practice.

There were 54.5% missing data on health-care resource use during the follow-up period in the IUT arm and 38.9% missing data in the no IUT arm. There was also incomplete information in the CRFs with regards to the IUT and surgery. QALY values were generated using the responses to the EQ-5D-3L questionnaire but there were only complete data for 40.9% of the IUT arm and 55.6% of the no IUT arm. The missing data leads to an underestimation of average costs, whereas the direction of the effect on QALYs is uncertain. The patients’ and caregivers’ costs, despite being collected, were not included in the economic analysis presented below. The reason for this was due to the low available number of completed responses to the PCQ part B. We assumed therefore that both arms would incur similar time and travel costs if they required follow-up treatment. The cost–utility analyses conducted were a rehearsal for a future definitive trial and hence NHS costs were felt to be sufficient for this purpose.

An illustrative analysis was conducted from a NHS perspective only using all patient records collected during the feasibility study where missing information on the CRFs was imputed. QALY values used in this example were based on EQ-5D-3L scores, missing QALY values were estimated using multiple imputation. This analysis was chosen as it was considered to be less biased than other analyses. This was because major NHS costs information was collected on the CRFs, especially the IUT and surgery were the main cost drivers in treating the condition of SUI, and by imputing cost values on this information, we could minimise potential bias on costs; whereas we could not be certain about the direction of impact due to missing QALY data. In Appendix 27, a further set of analyses are reported that explore the implications of missing data using alternative assumptions.

Illustrative example of the cost–utility analysis: imputed values used for missing case report form data

In this analysis, imputation was adopted for missing CRF data on resource use related to the IUT and surgery, which were the key cost drivers affecting the cost-effectiveness of the IUT. The imputed values included using the median length of time (39 minutes) of an IUT, using a consultant as the main operator of an IUT and using a day case as the length of admission after surgery (as only a minority of patients were admitted overnight after the surgery). Table 17 presents the cost–utility results using imputation for missing CRF data. The IUT arm has a lower average cost per patient and has a lower average QALY value than the no IUT arm. The cost per QALY for the IUT compared with surgery alone is £8090. These results need to be interpreted with caution as it is argued that there needs to be a difference of 0.075 in EQ-5D values for there to be a significant impact on cost per QALY ratios. 85 The probability of the IUT being cost-effective decreases as the society’s willingness to pay for a QALY threshold increases.

TABLE 17 - Cost–utility results using imputation for missing CRF data

Investigation strategy	Cost (£)	QALY	ICER (£)	Probability that the IUT is cost-effective for different threshold values for society’s willingness to pay for a QALY
				£0	£10,000	£20,000	£30,000	£50,000
IUT	1507	0.3857	8090	96%	80%	56%	45%	37%
no IUT	1661	0.4047		4%	20%	44%	55%	63%

Figure 8 presents the results of the bootstrapping simulation, which addresses the statistical uncertainty around costs and effects. As the majority of the iterations generated from the bootstrapping simulation were generally in the south-west quadrants, it suggested that the IUT arm tended to incur less cost than the no IUT arm but provided lower QALY values. The location of the average of the incremental cost and QALY pair simulations on the cost-effectiveness plane supports this, the average mean QALY difference is –0.006. This highlights the uncertainty around the cost–utility results. The cost-effectiveness of the IUT will depend on the threshold chosen to evaluate cost per QALY. This is further supported by the cost-effectiveness acceptability curve seen in Figure 9, which demonstrates that if society had zero willingness to pay for an additional QALY then IUT was 96% likely to be cost-effective; as society’s willingness to pay for a QALY increased, the likelihood of IUT being cost-effective decreased. Further economic analyses including complete case analysis, sensitivity analysis and base-case analysis with SF-6D used to calculate QALY values can be found in Appendix 27.

FIGURE 8.

Incremental cost-utility scatterplot with imputation of CRF data: IUT vs. no IUT.

FIGURE 9.

Cost-effectiveness acceptability curve with imputation of CRF data: IUT vs. no IUT.

Key messages

• The economic evaluation rehearsed data collection and analysis to inform the definitive trial.

• The ease of data collection and the consistency of resource use in administration of relevant investigations and treatments were evaluated, and the use of economic data collection instruments was piloted.

• The response rates for PCQs were 53% for part A and 46% for part B excluding blank responses, and of those returned, the majority were completed appropriately. Modifications to part B in particular should be considered in a definitive trial to make the PCQ questionnaires less burdensome for patients.

• A cost–utility analysis was performed with QALY values from EQ-5D-3L and SF-12. The cost calculation included NHS resource use collected in the CRF. Complete data to undertake the cost–utility analysis were available in only 41% of the IUT arm and 56% of the no IUT arm; sensitivity analyses were adopted to assess the significance of statistical and other uncertainties.

• The IUT arm had a marginally lower average total cost than the no IUT arm; however, a wider spread of costs of the IUT arm was observed. This may reflect the fact that surgery was not the chosen treatment for some patients in the IUT arm. All of the economic analyses found that the IUT arm had a lower average cost per patient than the no IUT arm in their incremental results except the complete case analysis (see Appendix 27). However, when a bootstrapping technique was performed on the incremental results to present the uncertainty around the cost-effectiveness ratio, the majority of iterations from the bootstrapping simulation were in the southern quadrants for all of the economic analyses. This highlights the potential cost savings experienced by the IUT arm.

• Quality-adjusted life-years determined from the SF-6D were slightly higher in the IUT arm, though the difference in QALYs calculated from the EQ-5D-3L and the SF-6D was not statistically significant. In the economic analyses in Appendix 27, the average result from the bootstrapping technique was positioned in the southern quadrants but on the y-axis. These analyses supported our original findings from the t-test; there is no significant difference in QALY values between the IUT and no IUT arms.

• Several limitations to this evaluation were recognised. While there were few missing data points within questionnaires, the number of completed questionnaires was low. The costs of non-invasive tests were omitted from the cost–utility analysis. Only NHS costs have been included in the analysis. The micro-costing of IUT was based on the cost of resources at one site; this might lead to an over/underestimation of the cost. Finally, costs and QALYs were only estimated over 6 months; this might be too short a time horizon for the full consideration of costs and QALYs. The impact of extending a time horizon is unclear at this stage but would need assessing as part of a modelling exercise informed by the results of the definitive trial.

• While we would propose that these limitations be modified in a future definitive trial, they meant that the present analysis must be interpreted with caution.

Methods

The intended recipients of the survey were those clinicians likely to be undertaking surgical treatment for women with SUI; members of the BSUG and the British Association of Urological Surgeons (BAUS) Section of Female, Neurological and Urodynamic Urology (BAUS-SFNUU) were chosen. The survey was designed to be distributed and completed electronically. An introductory e-mail was drafted that included:

1. a description of the INVESTIGATE studies

2. links to further information on the NIHR-HTA (www.hta.ac.uk/project/2272.asp) and trial (www.investigate-trial.com) websites

3. a link to the SurveyMonkey site where the questionnaire was hosted

4. contact details should potential respondents prefer to obtain a paper-based questionnaire and a reply-paid envelope (none did).

A copy of the paper-based questionnaire is included as Appendix 14.

The questionnaire itself sought categorised demographic data regarding respondents’ grade or rank, role (specialty and extent of specialisation), gender, time since graduation, access to and current use of IUTs, and their current workload in surgery for SUI. In order to assess current use of urodynamics in the patient group of interest, respondents were asked:

Do you currently arrange invasive urodynamic tests for most (say 75%) of your female patients with stress or stress predominant mixed incontinence?

Respondents were then presented with the following clinical scenario:

A 45-year old woman with two children, who has been sterilised; she has previously undergone pelvic floor muscle training and possibly other conservative treatments (in some scenarios), without benefit; she has not had any previous continence surgery.

They were then given six urinary symptom descriptions of varying complexity:

1. Complains of stress incontinence, but no frequency, nocturia, urgency or urgency incontinence; no symptoms of voiding difficulty; stress incontinence IS demonstrated on clinical examination.

2. Complains of stress incontinence, but no frequency, nocturia, urgency or urgency incontinence; no symptoms of voiding difficulty; stress incontinence IS NOT demonstrated on clinical examination.

3. Complains of stress incontinence, mild frequency, urgency and urgency incontinence, but describes the stress as the more significant problem; no symptoms of voiding difficulty.

4. Complains of stress incontinence, frequency (× 10 per day), nocturia (× 2 per night), urgency and urgency incontinence, with stress and urge of similar magnitude; no symptoms of voiding difficulty.

5. Complains of stress incontinence, frequency (× 15 per day), nocturia (× 2 per night), urgency and urgency incontinence, but describes the urge as the more significant problem; no symptoms of voiding difficulty.

6. Complains of stress incontinence, but no frequency, nocturia, urgency or urgency incontinence; also reports hesitancy, poor flow, and feeling of incomplete emptying.

Using a modified version of a bidirectional scale developed for measuring clinician and patient preferences in surgery,86 respondents were asked to rate the strength of their views about the necessity for IUTs before undertaking surgical treatment on an 11-point Likert scale from ‘unnecessary’ (+5) through ‘undecided’ (0) to ‘essential’ (–5) (Figure 10). They were specifically asked to respond on the basis of their own opinion, regardless of their current practices, and regardless of what they might have read in recent literature or current guidelines.

FIGURE 10.

Modified bidirectional scale developed for measuring clinician and patient preferences in surgery, after Young et al. 86

Respondents were then asked to use a Likert-type categorical scale graded ‘not at all important’, ‘somewhat important’, ‘very important’ or ‘extremely important’ to express their views about the importance of the research question:

Does invasive urodynamic testing prior to surgical treatment of stress or stress predominant mixed urinary incontinence improve the clinical- and cost-effectiveness of treatment compared to clinical assessment with non-invasive testing?

A vignette of the design of a proposed definitive trial was described, as:

The design of such a study is anticipated to be similar to that of our pilot study, i.e. a pragmatic multicentre RCT, randomising women with stress or stress predominant mixed incontinence, who fail to respond to pelvic floor muscle training, to receive either:

• no further assessment prior to surgical treatment (over and above the basic clinical assessment and non-invasive tests that they would have previously undergone)

or

• invasive urodynamic tests (conventional cystometry, video urodynamics or ambulatory urodynamics), with subsequent treatment dictated by the investigation results.

Respondents were asked about their willingness to participate and to randomise patients within such a trial. For those unwilling to randomise, open questions with free-text responses were asked about their reasons for their view and about acceptable alternative trial designs. Finally, the questionnaire asked about respondents’ willingness to participate in a short telephone, qualitative interview to explore further whether or not and how they use the results of urodynamic investigations to inform their clinical decisions and to contextualise the questionnaire responses; if willing, respondents were asked to provide preferred contact details and optimum time for contact.

Initial draft versions of the survey materials were piloted for content validity and functionality of the online system by a small group of gynaecologists and urologists, who were neither members of the BSUG nor the BAUS-SFNUU and therefore who would not be recipients of the finalised questionnaire. Eighteen invitations were distributed and 12 responses obtained. Following minor alterations, survey information materials and data collection instruments were submitted for approval by the research committees of the BSUG and the BAUS-SFNUU. In order to maintain confidentiality of e-mail addresses, the organisations themselves then circulated study information and invitations to participate to their respective memberships in August 2011 (see Appendix 4).

Reminder e-mails were sent at 3 and 6 weeks after the initial circulation to all potential respondents, as it was not possible to target the reminders to non-respondents. The survey site was closed 12 weeks after the initial invitations. There are very few individuals who are members of both organisations, but a footnote was appended to the invitation letter apologising for dual circulation and requesting that individuals make only a single response.

Survey update

Given the length of time between the circulation of the initial survey (August 2011) and the conclusion of this study, and the known emerging publications from other studies,35,36,39 it was felt appropriate to undertake a further brief survey before publication of this report. An e-mail (BSUG) or newsletter (BAUS-SFNUU) invitation to complete an abbreviated questionnaire was again circulated to members of the BSUG and the BAUS-SFNUU by their respective secretariats in June 2013; this contained a link to the SurveyMonkey site where the questionnaire was hosted. This was much briefer than the initial questionnaire and included only six questions regarding respondents’ current clinical role, their view as to the importance of the research question and their willingness to randomise patients into a definitive trial. They were also asked to comment on the proposed primary trial outcome (ICIQ-FLUTS) and possible alternatives, and asked for their opinion about the MCID for this outcome (see Appendix 15). This was done, recognising the lack of existing data on the MCID, and the proposal from the ‘DELTA study’ that expert opinion might be one approach to establishing the target difference. 87 Bearing in mind the rate and speed of response seen in the original survey (v.i.), a single reminder e-mail was sent to members of both specialist societies 2 weeks after the initial circulation, and the survey site closed for analysis after 4 weeks.

Statistical analysis

The analyses of survey responses were carried out on the data sets following closure of the survey websites. Basic descriptive statistics including response rates, percentages in categories and summary statistics were used for all relevant outcomes. No attempt was made to impute missing data for any of the outcomes. ‘equipoise ratios’ (ERs) were calculated after Young et al. 86 to report the three proportions for each scenario: those clinicians who regarded IUTs as essential (to a greater or lesser extent), i.e. gave scores of –5 to –1; those who had no preference between using IUTs or not, i.e. gave a score of 0; and, those who regarded it as unnecessary (to a greater or lesser extent), i.e. gave scores of +1 to +5.

Results

Original survey responses

The BSUG and BAUS-SFNUU membership databases are fluid, with new members joining and others leaving continuously throughout the year; hence the numbers sent reminder letters were slightly different from the number of initial invitations. For the first survey, initial invitations went to 332 BSUG members and 185 BAUS-SFNUU members, with most of these, plus a small number of new members, being sent reminder e-mails to follow up the initial invitation. In calculating response rates, we used as the denominator the number receiving the initial invitation. A total of 176/517 (34%) responded to the survey, with the majority answering most of the questionnaire. The response rate was not significantly different between urologists (36%: 67/185) and gynaecologists/urogynaecologists (32%: 106/332), with three responses coming from individuals who did not report their specialty.

Of those responding, 55% did so after the initial circulation, 36% after the first reminder letter and 9% after the second reminder. Following each circulation, the majority of responses were received within the first week (97%, 63% and 100%, respectively), with 97%, 79% and 100% being within 2 weeks (Figure 11).

FIGURE 11.

Original survey responses by week after invitation.

Clinical scenarios

Responses in terms of the necessity for IUTs in the six clinical scenarios are given in Figure 12 and Table 19. For each of these scenarios, only between 2% and 7% of respondents were undecided, with most reporting highly polarised opinions, i.e. towards the left or right ends of the Likert scale. For the three more complex symptom descriptions, over 90% responded –5 to –1 (i.e. IUT ‘essential’ to a greater or lesser extent); between 2.0% and 4.5% responded 0 (i.e. ‘undecided’); and, less than 7% responded +1 to +5 (i.e. IUT ‘unnecessary’ to a greater or lesser extent.) For the three simpler symptom descriptions, which might be summarised as ‘SUI or stress predominant MUI’ and comprise the patients intended as eligible in the pilot and future definitive trials, a greater range of opinions was expressed. However, even in scenario 1 (pure SUI with clinically demonstrable leakage on coughing), two-thirds thought IUTs necessary to a greater or lesser extent (i.e. gave scores of –1 to –5), with over one-third of respondents considering IUTs essential (i.e. gave a score of –5): the ER was 66 : 1 : 33.

FIGURE 12.

Responses to the necessity for IUTs in the six clinical scenarios. (a) Scenario 1: pure SUI – stress leak IS demonstrable (ER = 66 : 1 : 34); (b) scenario 2: pure SUI – stress leak is NOT demonstrable (ER = 83 : 6 : 12); (c) scenario 3: STRESS predominant MUI (ER = 89 : 5 : 6); (d) scenario 4: EQUAL severity MUI (ER = 96 : 2 : 3); (e) scenario 5: URGE predominant MUI (ER = 92 : 2 : 6); and, (f) scenario 6: pure SUI – symptoms of VOIDING difficulty (ER = 93 : 4 : 3).

TABLE 19 - Strength of clinicians’ views about the necessity for IUTs before undertaking surgery in the six clinical scenarios (given as % of n responses for each point on scale, and summary ER = sum essential: sum undecided: sum unnecessary)

11-point scale	Essential					Undecided	Unnecessary
	–5	–4	–3	–2	–1	0	1	2	3	4	5
Scenario 1 (n = 154)	Complains of stress incontinence, but no frequency, nocturia, urgency or urgency incontinence; no voiding difficulty; stress incontinence IS demonstrated on clinical examination. Pure SUI; stress leak IS demonstrable
Opinion (%)	38.5	10.5	9	5	2.5	1.5	1.5	3	5	2	21.5
ER (%)	66					1		34
Scenario 2 (n = 154)	Complains of stress incontinence, but no frequency, nocturia, urgency or urgency incontinence; no voiding difficulty; stress incontinence NOT demonstrated on clinical examination. Pure SUI; stress leak NOT demonstrable
Opinion (%)	58.5	13.5	6	2.5	1	6.5	1	3	3	1	4
ER (%)	82					6.5		11.5
Scenario 3 (n = 152)	Complains of stress incontinence, mild frequency, urgency and urgency incontinence, but describes more significant problem; no symptoms of voiding difficulty. STRESS predominant MUI
Opinion (%)	75	7	5	2	0	5	1	2	1	0	2
ER (%)	89					4.5		6.5
Scenario 4 (n = 153)	Complains of stress incontinence, frequency × 10, nocturia ×  2, urgency and urgency incontinence, of similar magnitude; no symptoms of voiding difficulty. EQUAL severity MUI
Opinion (%)	86	4.5	3	2	0	2	0.5	0	0	0	2
ER (%)	95.5					2		2.5
Scenario 5 (n = 154)	Complains of stress incontinence, frequency × 15, nocturia ×  2, urgency and urgency incontinence, urge as the more significant problem; no symptoms of voiding difficulty. URGE predominant MUI
Opinion (%)	84	4	2.5	0.5	0.5	2	0.5	1.5	0.5	0	4
ER (%)	91.5					2		6.5
Scenario 6 (n = 154)	Complains of stress incontinence, but no frequency, nocturia, urgency or urgency incontinence; also poor flow, and feeling of incomplete emptying. Pure SUI; symptoms of VOIDING difficulty
Opinion (%)	83	4	3	2	0	4.5	2	0	0.5	0	0.5
ER (%)	93					4.5		2.5

Views about a future definitive trial

The results above could be interpreted as indicating that clinicians had little doubt about the value of IUTs and would be unlikely to be interested in a future clinical trial. However, when asked to rate the importance of the research question, 24% rated it ‘extremely important’, 45% ‘very important’, 26% ‘somewhat important’, and only 5% thought it ‘not at all important’ (Figure 13).

FIGURE 13.

The importance of the research question.

Inevitably, the number of generalists included in the survey was small, and responses did not differ markedly between gynaecologists and urologists, although generalists (n = 6) were somewhat less likely to look on the question as being ‘extremely important’ or ‘very important’ (33%), than consultants with a special interest (69%) or subspecialists (74%) (Figure 14).

FIGURE 14.

Importance of the research question by level of consultant specialisation within gynaecology and urology. Six general urology and gynaecology consultants are excluded from the figure.

On the 10-point Likert scale of ‘willingness to randomise’, 64.6% gave a score of seven or over. The breakdown of scores by degree of specialisation showed that 61.3% of consultants with a special interest gave a score of ≥ 7 compared with 81.9% of subspecialists (Figure 15).

FIGURE 15.

Likert scale of consultant ‘willingness to randomise’ by level of specialisation. Six general urology and gynaecology consultants are excluded from the figure.

Survey update responses

There were 145/498 (29%) responses to the survey update. Allowing for slight differences in the timing of distribution of the initial request and reminder by the BSUG and the BAUS-SFNUU, 49% of responses were returned in the first week after distribution, 55% within 2 weeks, and 96% within 3 weeks (1 week after the reminder e-mail).

On this occasion, we specifically sought responses from consultants/specialists only, so as to capture the views of those surgeons who might potentially wish to collaborate in a future definitive trial. Of all the responses, 4.1% came from general obstetricians and gynaecologists/urologists, 60.7% from consultants with an interest in urogynaecology/female urology, and 30.3% from subspecialists in urogynaecology/female urology. The number and timing of responses and the demographics of respondents were therefore, perhaps unsurprisingly, very similar between our initial survey and the more recent update.

Of all respondents, 68% still rated our research question ‘very important’ or ‘extremely important’ (Figure 16).

FIGURE 16.

The importance of the research question as reported by consultants in the initial survey (August 2011) and in the update (June 2013).

On the Likert scale of ‘willingness to randomise’, 61% recorded a score ≥ 7/10 (Figure 17). A total of 102/145 (70%) of respondents provided e-mail addresses, indicating their interest in contributing to a possible future definitive trial. Although a number of these came from e-mail servers external to the NHS (e.g. www.doctors.org.uk and www.yahoo.co.uk) and 20 came via the generic server nhs.net, comparison of e-mail addresses suggested that these 102 respondents represented 89 separate hospitals or NHS Trusts in England (79), Scotland (4), Wales (3), Northern Ireland (2) and the Republic of Ireland (1).

FIGURE 17.

Likert scale of ‘willingness to randomise’ as reported by consultants in the initial survey (August 2011) and in the update (June 2013).

When asked about the appropriateness of ICIQ-FLUTS as a primary outcome measure, 15% either had no opinion or omitted the question; 77% (91% of those expressing an opinion) felt this was an appropriate outcome. In response to an open question, 10 respondents suggested alternative primary outcomes as shown in Box 1.

BOX 1 - Alternative primary outcomes suggested by respondents

Alternative primary outcome measures proposed

Q11 (SUI) from ICIQ-UI.

ICIQ-UI SF (two respondents).

Electronic Personal Assessment Questionnaire.

Composite outcome of complications and failure.

Complication rates (e.g. failure, retention, OAB) (two respondents).

Combined subjective and objective evaluation.

Simple measure of incontinence (yes/no).

Respondents were asked to categorise what they considered the MCID in ICIQ-FLUTS score, given a maximum score of 48, and bands 1–4, 5–8, etc. Of all respondents, 50% either had no opinion or omitted the question. Of those responding, the modal response was in the range 9–12 (Figure 18).

FIGURE 18.

Respondents views on MCID for ICIQ-FLUTS.

Key messages

• The response rates to the initial survey (34%) and update (29%) were disappointingly low given that the circulation was to members of relevant specialist professional societies. This raises the question as to just how representative of the totality of surgical practice in incontinence the responses are, although it is likely that the respondents were those with a particular interest in the subject matter or in clinical research.

• The surgical workloads reported by respondents amounted to a total of approximately 8300 procedures per year; HES for England reported approximately 12,500 procedures for SUI in 2009–10 and 2010–11. While informal, self-reported surgical activity is notoriously unreliable, the respondents clearly embrace a significant proportion of incontinence surgery.

• All respondents reported having access to IUTs, with 79% undertaking investigations themselves, confirming the relevance of their opinions to the survey questions.

• Following the initial survey, the majority of responses were received after the first circulation, and over 90% were received after a first reminder. The majority of responses were received within 2 weeks of original distribution or reminders. This would suggest that with similar e-mail/online surveys no more than one reminder is necessary, and the time between initial distribution and reminder, and between reminder and survey website closure can be limited with minimal loss to responses. The initial survey sought responses from all grades of society membership, whereas the update sought responses from consultant/specialist grades only. There were 158 consultant responses to the initial survey, and 145 responses to the update.

• The majority (88%) of consultant respondents reported undertaking IUTs on most of their patients with SUI or stress predominant MUI. When asked to rate the necessity for IUTs in a range of clinical scenarios of varying symptom complexity, few respondents were undecided, with most reporting highly polarised opinions. For the three more complex symptom descriptions, 83%–86% looked on IUTs as essential (i.e. graded –5 on a scale of –5 to +5). For the three simpler symptom descriptions, a greater range of opinions was expressed. However, even in the situation of pure SUI that is clinically demonstrable, two-thirds thought IUTs necessary to a greater or lesser extent (i.e. graded –1 to –5), with over one-third of respondents considering IUTs essential (i.e. graded –5).

• Despite the apparent strength of professional opinion favouring IUTs in women with SUI or stress predominant MUI, when asked to rate the importance of the research question underlying INVESTIGATE, 69% rated it ‘extremely important’ or ‘very important’ and 65% gave a ‘willingness to randomise’ score of seven or over (on a scale of 0–10).

• Although the number of general gynaecologists and urologists responding to the survey was small, they were somewhat less likely to look on the question as being important, and unlikely to be willing to recruit patients into a definitive trial. This should be borne in mind when selecting possible sites for a future trial.

• Despite the publication of two other trials addressing the issue of the clinical utility of urodynamics in female SUI (both of which concluded that clinical assessment was not inferior to invasive urodynamic testing prior to surgery for SUI or stress predominant MUI), these latter opinions persisted largely unchanged two years after the initial survey. Over 100 respondents, representing approximately 88 NHS Trusts across the UK, indicated a willingness to become involved in a future definitive multicentre trial.

• The majority of respondents (77% overall, or 91% of those expressing an opinion) felt ICIQ-FLUTS was an appropriate outcome; 50% ventured an opinion as to the MCID for this scale.

Methods

As indicated above, this was an ‘opt-in’ follow-up from the initial survey responses. A purposively selected subsample was drawn from those respondents indicating a willingness to take part in the interview study, who provided contact details. Interviews continued until a point of saturation was reached (i.e. that no new material was emerging from the interviews).

An information sheet was provided (see Appendix 9), and while return of a completed questionnaire was taken as indicative of implied consent to participate in the clinician survey, written consent to take part in the interview was sought (see Appendix 13).

Telephone interviews were undertaken by an experienced qualitative researcher (see Acknowledgements) using a topic guide based on the survey and developed through discussion within the project team. The topic guide ensured all areas of interest were covered, but was used flexibly with the aim of allowing interviews to flow as freely and naturally as possible and to allow participants to discuss issues that were important to them. The interviewer prompted as appropriate to ensure that all views were fully explained, and the meaning of participants’ responses were clear. All interviews were audio-recorded and transcribed verbatim.

Qualitative analysis

Analysis of the interview data were based on the constant comparative method. 88 Transcripts were read three to four times and open codes initially applied line by line to the data to represent the meaning or significance of each sentence or group of sentences by the data analyst. Generation of the open codes proceeded sequentially, with no attempt at this stage to impose any framework on the data. The open codes were then incrementally grouped into organising categories or themes, by the analyst and study lead together. These categories were modified and checked constantly as further open codes were incorporated as analysis proceeded. When categories had been created to express all of the open codes, explicit specifications were written for each of the categories to assist in determining under what circumstances data should be assigned to any given category. The categories and their specifications (the coding scheme) were then programmed into the NVivo 10 (QSR International, Warrington, UK) qualitative software. The coding scheme was then used to process the data set systematically by assigning each section of text to a category, according to the category specifications.

Results

Of the 176 survey respondents, 87 (49%) agreed to being approached for interview and provided contact details. A diverse sample was recruited purposively to include, gynaecologists and urologists; those who did/did not routinely use IUTs; those with different approaches to when invasive urodynamic testing was needed; those with different perspectives on both the planned RCT, and their willingness to randomise patients. A total of 18 interviews were carried out, by which point data saturation was attained.

As would be expected from the quantitative results, and given the nature of the purposive sampling method, interview participants tended to be polarised in their view and regarded invasive urodynamic testing as either essential or of limited use.

For those interviewees who undertook invasive urodynamic testing regularly, the tests seemed to have a range of functions that clinicians regarded as valuable. The first of these was to add to the overall clinical picture and help inform the best course of action.

Well it helps with someone who has a history of stress incontinence and you have not been able to demonstrate it. Then you want to try and quantify the leakage and urodynamic testing can help you do that sometimes.

Participant 05

A second function was for invasive urodynamic testing to act as a ‘safety net’ to prevent unnecessary or inappropriate surgery. The fact that many of these patients would be offered surgery was important, and several participants very clearly framed future surgery as further underpinning the need to be as sure as they possibly could be about the diagnosis.

I would use urodynamic tests on anyone that I was going to operate on, it’s very easy to operate, but it’s not very easy to un-operate, so if you have a complication that arises as a result of your surgery, you can’t go back and say ‘well I would have liked that information, if I’d known that beforehand, I would have done something different’.

Participant 02

A third function of invasive urodynamic testing was to facilitate the appropriate counselling of patients. The clinician’s job was understood as being to gather all the available information that could then be presented to the patient along with treatment options and likely outcomes.

It gives you reasonable scientific evidence to sit with the patient and say ‘that is what you have got, that is what we are going to do, and that is the outcome’.

Participant 14

Interestingly, those who reported using IUTs routinely did not always do so because they perceived value in the tests. For a minority, there was an element of ‘fitting in’ with what colleagues did and adopting local customs and practices.

I have just moved to a new trust, my colleague investigates all patients who have stress incontinence before surgery. In my previous post I didn’t actually undertake urodynamics in patients who had pure stress incontinence symptoms so at the moment I, I suppose you could say that I’m doing it because it’s, it’s sort of departmental protocol really.

Participant 09

For those interviewees who used IUTs relatively rarely, this position was underpinned by a range of factors. The first of these was the understanding that the use of invasive urodynamic testing is not currently recommended by NICE prior to conservative treatments, and that, while it may be needed in more complex clinical scenarios, there is no evidence to support its use prior to surgery where the diagnosis of SUI is likely based on clinical assessment alone.

It [his/her current practice] is based on the NICE guidance which suggests you don’t have to do it in every woman.

Participant 03

Those clinicians who did not use IUTs routinely were much more attuned to the potentially unnecessary time and cost implications and weighed these against the likely ‘added value’ of IUTs. Unless a case was complicated, they believed that IUTs would not alter the treatment plan and that the information that could be obtained from other sources (such as patient history, examination and bladder diary) was sufficient.

We have things like flow meters which, you know, in the clinic, we have bladder scans, we can measure residuals, patients are quite good at filling in frequency volume chart [...] and a good physical examination combined with these non-invasive tests that I’ve just mentioned, I think gives you more information than urodynamics.

Participant 01

As in the overall survey responses, about two-thirds of the clinicians interviewed thought the basic research question of the INVESTIGATE studies to be an important one. For some, this was because they believed there was genuine uncertainty about the benefit of IUTs.

I think it is worth doing because as well as telling us whether urodynamics is useful, there will be a lot of information which will tell us in what ways it can be useful, it will say these are the things you should be looking at.

Participant 16

However, for others, the desire for a definitive trial was because they believed they knew the answer to the question but felt the need for ‘harder evidence’ to support their practice and encourage others to change theirs. Within the sample, there were examples of both interviewees who believed a trial would show IUTs should be used, and those who believed a trial would show the opposite.

I still think it is important that we answer this question because you know my certainty up to now is based on what I have been taught and what I have observed but that is not based on research particularly so I still think it is a very important research question.

When we asked you whether or not you thought the question that investigate is addressing is important, you said ‘very important’.

Well on the basis of what the NICE guidelines said, if we stopped doing it in the large number of cases that they suggest we should stop, then it would free up funding for something else.

As a deliberate outcome of the purposive sampling, we interviewed fairly even numbers of both those who would be willing to randomise into a definitive trial and those who would not. Unsurprisingly, those who always undertook invasive urodynamic testing and regarded it as essential were least likely to be willing to randomise, even if they had indicated they thought it an important research question. In these cases, they wanted the question answered in order to provide hard evidence that invasive urodynamic testing is necessary, but, because they were not personally in equipoise, they were not prepared to allow their patients to be part of producing that evidence.

I wouldn’t be happy [to randomise patients], no. That’s in keeping with my belief that it is an important test.

Participant 12

In contrast, those who appeared genuinely uncertain about invasive urodynamic testing, or at least were happy not doing it, were the ones most prepared to randomise.

I don’t have a problem not doing the urodynamics . . . so it makes perfect sense to put our patients into the trial . . . I wouldn’t see a problem with that at all . . . and I think the, you know, if we can answer the question it would be very worthwhile.

Participant 09

Key messages

• The interviews facilitated a more detailed understanding of whether or not and how participants used the results of IUTs within their practice and the relative value that they attached to these.

• The majority of those using invasive urodynamic testing routinely were convinced of its clinical utility in terms of helping to decide the best course of action and helping to counsel patients, although a small number reported that their practice in relation to invasive urodynamic testing was influenced more by local norms around its use rather than any personal commitment to it on their part.

• In contrast, those who used invasive urodynamic testing relatively rarely saw little additional benefit from its use (the information that could be obtained from other sources such as patient history, examination and bladder diary was sufficient) but significant potential costs (e.g. in terms of time, financial implications).

• The analysis of the interview study data also gave some insight into the apparent inconsistency in survey responses between lack of personal equipoise over the value of invasive urodynamic testing on the one hand, and the majority view that the basic research question was important and associated with a high degree of willingness to randomise patients into a definitive RCT on the other hand.

• While some clinicians’ views were shaped by genuine uncertainty about the value of IUTs, more commonly the research question was regarded as important because clinicians believed they personally knew the answer and wanted research in order to change others’ practice and bring it in line with their own.

• This could introduce a significant bias to randomisation, if clinicians who regarded invasive urodynamic testing as essential were unwilling to have some of their patients denied it; or alternatively if those who use invasive urodynamic testing relatively infrequently were unwilling to risk their patients being exposed to what they see as unnecessary tests. While recognition of a degree of community equipoise may allow many to ‘suspend’ their lack of personal equipoise and agree to randomise their patients into a future definitive trial, it is likely that some will find this unacceptable.

Methods

Interviews were carried out to explore women’s understanding and their experiences of the study, the consent processes and their decision to participate. Purposive sampling was used to include women from a range of ages, trial participation status (did not agree to randomisation; randomised and retained to final follow-up; randomised but did not provide full follow-up data), allocation status (IUT or basic assessment), treatment received (surgery or conservative management) and study site.

The PIS included a description of this part of the study and an indication that women might be approached for interview. Those women who did not agree to being randomised within the trial were approached as soon as possible thereafter for interview. Women who did agree to randomisation were approached at the end of the trial, so as to capture both their reasons for agreeing to participate and their overall experience of taking part in the study.

A specific PIS was provided for the interview study (see Appendices 7 and 8) and written consent was obtained from all interviewees (see Appendices 11 and 12). The interviews were carried out by an expert qualitative interviewer (see Acknowledgements) and, with permission of interviewees, were audio recorded and transcribed verbatim. The vast majority of interviews were carried out face to face but a small number were completed by telephone due to participants’ availability and preferences. The interviews were semistructured using a prompt guide with broad topic areas but the emphasis was on encouraging women to discuss their own perspectives freely and allowing them to discuss issues that were important to them. The interviewer prompted as appropriate to ensure that all views were fully explained and the meaning of participants’ responses were clear. The prompt guide was developed from a literature review and discussions within the project team and was modified as the interviews progressed to incorporate issues raised by earlier interviewees. The purpose of the interviews was to explore women’s understanding and experience of the study, their decisions around participation and their perceived barriers to and facilitators of participation in a RCT. This information will inform the decision of whether or not to proceed to a definitive trial (i.e. whether or not women are likely to participate) and enable us to refine the content of the information given to women and the recruitment and data collection procedures used.

Data collection and analysis was iterative, using the constant comparative method. 88 Data collection continued until saturation of themes was reached, that is the point at which interviews no longer generated new concepts. As for the clinician interviews, transcripts were read three to four times and open codes were initially applied line by line to the data to represent the meaning or significance of each sentence or group of sentences by the data analyst. Generation of the open codes proceeded sequentially, with no attempt to impose any framework on the data. The open codes were then incrementally grouped into organising categories or themes, by the analyst and study lead together.

These categories were modified and checked constantly as further open codes were incorporated as analysis proceeded. When categories had been created to express all of the open codes, explicit specifications were written for each of the categories to assist in determining under what circumstances data should be assigned to any given category. The categories and their specifications (the coding scheme) were then programmed into NVivo 10 qualitative sofware. The coding scheme was used to process the data set systematically by assigning each section of text to a category, according to the category specifications.

Results

All women who declined to participate in the pilot study were invited to take part in an interview. A total of 51 were approached but, unfortunately, none were willing to be interviewed.

A total of 111 pilot study participants were invited to take part in an interview. A diverse sample was approached in order to include, those from different study sites; those from the two study arms; those who did and did not complete all follow-up; and a wide range of ages. A total of 36 women indicated they were willing to be interviewed. Of these, 29 were interviewed; two withdrew from the interview study before the interview could be arranged; one had moved and so was no longer covered by our research governance approvals; and four were not interviewed as they were from groups already well represented in the sample (all were contacted to explain this). Details of the final interview sample are shown in Table 20.

Key messages

• Women were, in general, very positive about the study and found the decision to participate straightforward; this is in keeping with the finding that the majority of eligible women agreed to randomisation within the trial.

• We had hoped to interview some women who had declined randomisation, feeling that their views may be crucial to successful planning of a definitive trial. While it was regrettable that no ‘decliners’ agreed to interview, the fact that so few eligible women in fact declined randomisation mitigates the impact of this gap in our knowledge.

• In addition to the ‘altruistic’ factors motivating participation, the potential to avoid having IUTs was an important factor for some women specific to this study.

• Trial PISs (both short and full versions) were appreciated by interviewees. Supplementary information from trial and clinic staff was also seen as important, emphasising the need for appropriate training of all staff involved.

• The importance of having information about the trial prior to consultation was emphasised; this triangulates with the need to have effective screening processes in all centres.

• Questionnaires used at baseline and 6 months after the start of treatment were generally seen as being easy to complete, if a little repetitive (especially at 6 months); this is in keeping with the low rate of missing information from those questionnaires that were returned.

• Repeating questionnaires at 6 months when many women had few, if any, symptoms to report was sometimes felt to be burdensome and irrelevant; this is in keeping with the number of blank follow-up questionnaires returned. In a future definitive trial, it would be important to emphasise the need to complete and return questionnaires even if there are few symptoms, but also to modify questionnaires to allow ‘short-cutting’ of irrelevant areas.

The randomised external pilot trial

The pilot trial can be considered a success. Although recruitment was initially slow, and was more successful in some centres than others throughout, we were able to recruit patients from all our study centres in sufficient numbers to confirm that recruitment was feasible, and that women were happy to engage with the study objectives and be randomised. The study procedures were seen to be adequate and functional in most areas, and we have gained important insights to inform the design and efficient conduct of a future definitive trial. These include, allowing a realistic time frame for regulatory approval and site start-up; employing a range of strategies to retain trial centre engagement (e.g. website, newsletters, recruitment updates, RtT thermometer); and modifying screening instructions and procedures to ensure that an ‘assume eligibility’ approach is employed. The potential for running standard-setting screening exercises for centres is an important consideration.

The pilot trial clearly demonstrated that there remains a need for a definitive study. We identified a change in planned treatment for 19% of the women randomised to the invasive urodynamic testing arm (compared with 4% in the no IUT arm). This confirms that undergoing invasive urodynamic testing does influence practice, and is in keeping with some other studies in this area,39,42,92 albeit the results have not been consistent across all studies. 36 Based on the outcome measures reported for the women at 6 months, the pilot trial suggests that there is a small difference in outcome as a result of this change in practice. Whether or not this difference is statistically, clinically and economically significant remains unproven, and will require a larger trial. Given the uncertainty around the cost-effectiveness outcomes (see below), the occurrence of a measurable change in practice with a limited difference in outcome and uncertain cost–benefit leads us to conclude that a definitive study is necessary.

Knowing the completion rates for the various questionnaire outcomes we have piloted is useful and will help to inform a future trial. Completion rates were high for all questionnaires with a similar rate and spread of missing items. It is however recognised that the completion of questionnaires can be burdensome for participants. 93 This may be particularly the case for those with few or no symptoms; this may account for the number of blank questionnaires returned at 6 months, and was apparent from the patient interview study. Accepting that the UDI was the fourth instrument in a booklet of 6 questionnaires in total, it had a slightly lower completion rate at both baseline and 6 months. The questions in ICIQ-UI SF overlap considerably with those in the longer ICIQ-FLUTS and so we recommend omitting both UDI and ICIQ-UI SF from the definitive trial to reduce respondent burden. We anticipate that this may improve completion of the remaining items. Greater emphasis needs to be placed on the importance of returning a completed questionnaire even in the absence of any remaining symptoms. Cost questionnaires could be modified to allow ‘short-cutting’ of irrelevant areas.

Bladder-diary data and pad-test use were poorly completed in our pilot. This may be because many of the women would have completed similar diaries or frequency/volume charts earlier in their continence assessment; it may be seen as rather more intrusive than simple questionnaire responses; it is possible that the diary design mitigated against consistent completion of pad-use data. The trial recruitment process enrolled only women with SUI or stress predominant MUI, and the diary data did not show any evidence of abnormal urinary frequency or nocturia and there appeared to be no change at 6 months in either arm (other than in pad-use). We recommend consideration of omitting or modifying diary data and pad use in the definitive trial, so as to focus on incontinence episodes in order to increase the completion rate of these data.

Few AEs were recorded during the study; these were evenly spread across the study arms; most were expected AEs related to treatment, and none were related to the trial intervention (IUT) itself. The most common anticipated AE following the trial intervention (IUT) is UTI, with reported incidences between 3% and 20%. 94 Information about the occurrence of UTI or additional GP visits was sought at the postoperative clinical review (CRF – visit 6). Patients would conventionally be advised to report persistent symptoms of increased frequency of micturition, dysuria or haematuria following IUTs. It is quite likely however that most episodes of UTI occurring following IUTs would have been reported to and treated by GPs rather than trial staff. Such episodes may not have been documented on CRFs nor reported to the NCTU, and their incidence may therefore be significantly under-reported here. We recommend introduction of a system for more effective capture of this information in a future definitive trial, for instance by giving patients a contact telephone number, or an event diary to note such events during follow-up.

There was a high rate of loss to follow-up after treatment. Although 75% of women had either face-to-face or telephone follow-up (typically at 2 to 3 months) after surgical treatment, only 56% (63% of those circulated) returned follow-up questionnaires at 6 months. The lack of follow-up questionnaires being returned may reflect the fact that most were happy with the outcome of their treatment, and we found some evidence to support this. Nevertheless, in a future definitive trial it would be necessary to ensure a much higher rate of response to the primary outcome. As suggested above, it would be desirable to rationalise the number of instruments and hence burden associated with questionnaire completion. Alternative modes of completion for follow-up questionnaires (e.g. telephone or web based), and providing incentives to return questionnaires,95 are further evidence-based strategies that might enhance retention rates for data collection. A further possibility is to link questionnaire completion at follow-up to the face-to-face clinic review, thereby allowing a check by a research nurse or trial co-ordinator of item completion before patients leave the clinic area; this would, however, require a change to the current practice of some units, and risk some of the pragmatic nature of the trial.

Clinicians’ views

The clinician survey and qualitative interviews were directed at identifying professional opinion within a specialist group interested in the management of female UI. The overall response rate to the initial survey (34%) and subsequent update (29%) must leave any conclusions open to question, due to the potential for non-response bias. A number of previous surveys of similar national and international professional groups have been published, with response rates between 21% and 67%. 48,96–98 None of these studies employed incentives to take part, and indeed none used reminder letters or e-mails. 95,99 Clearly the level of interest or excitement generated by the topic in potential respondents is of importance in encouraging responses. It is, however, difficult to explain why surveys on such similar topics should achieve such varying response rates in different countries (21% to 57%)96,97 or why a survey on a clinical guideline48 should achieve a different response rate from one on a major recommendation from the same guidance (i.e. this study) (64% vs. 34%).

The limited information on the specialty group membership makes comparison of respondents and non-respondents difficult. It is possible that those who did respond may hold systematically different views on the use of invasive urodynamic testing and on the research question than those who did not participate in the survey. While this cannot be entirely refuted, the following findings would argue against this possibility. The surgical workloads reported by respondents to the initial (2011) survey amounted to a total of approximately 8300 procedures per year; HES for England reported approximately 12,500 procedures for SUI in 2009–10 and 2010–11. 14 While informal, self-reported surgical activity is notoriously unreliable, the respondents clearly embrace a significant proportion of incontinence surgery. All respondents reported having access to invasive urodynamic testing, with 79% undertaking investigations themselves, confirming the relevance of their opinions to the survey questions.

We found that the majority of respondents to the survey considered invasive urodynamic testing to be necessary to a greater or lesser degree before surgical intervention in SUI, whether or not patients have additional symptoms suggestive of OAB or VD. Not only were few clinicians apparently undecided on the issue (i.e. were in personal equipoise), but there was little evidence of professional community equipoise. Only when clinicians are in equipoise on an issue or recognise it to be an area of genuine uncertainty are they likely to feel comfortable to randomise their patients between treatments or, as in this case, investigation strategies. Hence, measuring surgeon preference is a crucial component of trial feasibility.

Despite this lack of personal equipoise and the fact that invasive urodynamic testing was considered necessary across all scenarios, the majority of respondents regarded the basic research question as being important (70%), and most would be prepared to randomise patients into a definitive RCT to address this (60%). These views persisted over the 2 years between our initial survey and the update, despite publication of two other trials addressing a similar research question. 36,39 Analysis of the interview study data gives some insight into the reasons for this apparent inconsistency. It might be anticipated that clinicians would only regard a research question as important and be prepared to randomise their patients in a study where they themselves were uncertain of the best course of action and are looking to the study as a means of resolving that uncertainty. However, discussion of these issues in interviews revealed a more complex picture. While some clinicians’ views were shaped by genuine uncertainty about the value of invasive urodynamic testing, more commonly the research question was regarded as important because clinicians believed they knew the answer and wanted research in order to change others’ practice and bring it in line with their own. This could introduce an important complicating factor to whether or not they would be prepared to randomise patients because clinicians who regarded invasive urodynamic testing as essential may not be willing to have some of their patients be denied it. However, in contrast, those who appeared genuinely uncertain about invasive urodynamic testing, or were happy not doing it, were the ones that seemed happiest to randomise. While recognition of a degree of community equipoise may allow many to ‘suspend’ their lack of personal equipoise and agree to randomise their patients into a future definitive trial,100 it is likely that some will find this unacceptable. We are not aware of any evidence regarding whether or not these different stances may affect willingness to fully engage with the trial or pursue it to completion. From the survey update, however, there is an indication from a majority of our target group of their willingness to recruit patients into a future definitive study.

Patients’ views

The patient interview study showed our patients to be generally very positive towards all aspects of the trial and found the process of approach, screening, consent and recruitment to be accessible, straightforward and easy to understand. The trial processes before investigation and during follow-up were not burdensome, although we obtained some helpful comments in relation to the completion of long questionnaires in the absence of residual symptoms. It was interesting to learn that a number of respondents had a previously undeclared preference for avoiding invasive urodynamic testing and, while willing to be randomised, expressed relief at being allocated to no further testing. This finding certainly resonates with clinical experience that many women find the urodynamic assessment (or the anticipation of it) to be worrying or slightly distressing. 101,102 This is in marked contrast to the overwhelming view of the majority of clinicians responding to the survey,103 for whom invasive urodynamic testing is seen as essential in most clinical situations prior to surgical treatment; it also contrasts with the perception that invasive testing may be ‘what women want’. 50 This dichotomy of views stresses the importance of this research question to define more clearly in what situations invasive urodynamic testing can be avoided, and therefore provides further support for proceeding to a definitive study.

Determining the sample size for a future definitive trial

As recommended in a forthcoming monograph on ways of specifying a target difference for a trial, we tried to determine estimates from more than one approach. 87 The first approach was to try to elicit information for a future trial by a survey of consultant members of the BSUG and the BAUS-SFNUU (see Chapter 5). Among other things, the update survey in June 2013 asked these clinicians the following question:

The ICIQ-FLUTS questionnaire is scored between 0 and 48. What do you consider is the minimum difference in ICIQ-FLUTS combined symptom score that you would consider to be clinically important (as opposed to statistically significant)?

The ICIQ-FLUTS scale has not been used in many published studies to date, and, perhaps unsurprisingly, only 50% of consultants responding expressed an opinion. They were given a choice of seven ranges of the scale to define the clinically important difference (from 1–4 to > 24) and all these ranges were chosen by at least one clinician, with the modal range being 9–12 (see Figure 18). It is not known how strong and informed their views were. However, in separate discussions, members of the study team did not find it easy to choose a target difference based on the limited use of the scale so far.

Another approach to setting the target difference was to use data from the external pilot trial: the SD of the primary outcome would inform the sample size calculation and allow any target difference to be expressed as a standardised effect size. When the pilot trial results became available, it became apparent that the distribution of the ICIQ-FLUTS total score at 6 months, and the difference between scores at baseline and 6 months, typically had low values. The mean score (SD) at 6 months in the ‘no-IUT’ arm was 7.3 (5.3) and the mean change between baseline and 6 months was 9.3 (7.3). It was therefore not realistic to see differences in mean outcomes between trial arms in the order of 9–12 units. Given the trial results, the study team then decided that differences of 2, 3 or 4 units would be a realistic and meaningful difference that might be achieved in any comparison of an intervention for women eligible for a future trial. It was felt that a difference of around three units would also be of clinical interest since a decrease of this level would equate to complete recovery for one of the symptoms assessed in the ICIQ-FLUTS score. Given the observed SDs, these target differences of 2, 3 or 4 units are equivalent to standardised effect sizes of 0.33, 0.50 and 0.67 when comparing mean scores at 6 months, or 0.29, 0.43 and 0.57 when comparing mean changes in score over 6 months. In contrast, a difference of 9–12 units would equate to a standardised effect size of 1.5–2.0, which is a very large difference; many trials are planned on a standardised effect size of around 0.5. Cohen has suggested that standardised differences of 0.2, 0.5 and 0.8 correspond to ‘small’, ‘medium’ and ‘large’ effect sizes. 104

If a study is planned on the basis of a ‘realistic’ value for the target difference, then consideration has to be made of whether or not this is also a ‘clinically important’ difference. If it is clear that this is not a ‘clinically important’ difference, then there are real doubts whether or not the trial should take place. In this case, the modal estimate of a ‘clinically important’ difference from the clinician survey was much higher than our estimate of ‘realistic’ target differences having seen the pilot trial results. However, these ‘realistic’ differences correspond to small or medium standardised effect sizes and recovery in one of the symptoms investigated. In addition, the current lack of data from published trials using ICIQ-FLUTS, and therefore evidence on which to base expert judgement, casts some doubt of the usefulness of a survey of experts in this situation. We have therefore used the pilot trial results to derive target differences on which to plan a future definitive trial.

The key parameters necessary to calculate the sample size are shown below:

1. type 1 error: 5%

2. power: 90%

3. eligibility rate among those screened: 37%

4. recruitment (consent) rate of those found eligible: 78%

5. response rate for ICIQ-FLUTS at 6 months for those recruited (i.e. retention rate for primary outcome): 56%

6. SD of ICIQ-FLUTS at 6 months: 6

7. SD of change in ICIQ-FLUTS from baseline: 7

8. correlation between ICIQ-FLUTS at baseline and 6 months: 0.25

9. smallest difference between mean ICIQ-FLUTS scores in trial arms that is of clinical interest – as chosen: 2, 3 and 4.

There are two possible approaches to analysis, and hence sample size calculations, when data are available at baseline and follow-up:

1. comparing mean changes between baseline and follow-up, or

2. comparing means at follow-up adjusting for baseline.

Tables 21 and 22 show the necessary numbers that would have to be screened, approached for recruitment to trial and provide response data at follow-up. Table 21 shows that, if the minimum difference of interest in change scores was two units, then a total of 516 responses on the primary outcome (258 per arm) would be needed. This would require 3194 women to be screened, of whom 1182 would be eligible and asked to take part in the trial and 922 would need to be recruited.

The numbers required for a trial comparing mean changes are greater than those comparing means at follow-up adjusting for baseline. However, as shown in Figure 7, the distribution of the primary outcome at follow-up was very positively skewed, so the sample size calculations based on the SDs of this variable are potentially misleading. Those based on the change scores are therefore more appropriate. If a future definitive trial was designed on the more conservative basis of the sample size necessary for a change score analysis, this should also provide at least 90% power for a comparison of means at follow-up adjusted for baseline.

Results integrated into Cochrane meta-analysis

The Cochrane review on urodynamic investigation for the management of UI in adults and children was first reported in 2002, with new citations added in 2006, 2011 and 2012. The most recent review was undertaken during the course of the INVESTIGATE-I study, and included two new trials. 34–36,39 A pre-publication version of this review was shared with the current authors,42 and one of the authors of the Cochrane review, a member of the INVESTIGATE-I TSC, agreed to incorporate outcomes from the pilot trial into appropriate meta-analyses. The following outcomes were incorporated:

• number with incontinence within first year (subjective) (ICIQ-FLUTS Q10a; response = ‘> never’) – see Figure 19

• number reporting SUI at clinic visit within first year (from subjective reports on CRF – ‘visit 6’) – see Figure 20

• number treated conservatively (from non-surgical treatments on CRF – ‘visit 5’) – see Figure 21

• number treated with drugs (from non-surgical treatments on CRF – ‘visit 5’) – see Figure 22

• number treated with surgery (from surgical treatments on CRF – ‘visit 4’) – see Figure 23

• number whose treatment was changed after urodynamics (from non-surgical treatments on CRF – ‘visit 5’) – see Figure 24

• number with urgency symptoms or urgency incontinence after treatment from subjective reports on CRF – ‘visit 6’) – see Figure 25

• number of AEs/complications after treatment (from AE and SAE reports to NCTU) – see Figure 26.

FIGURE 19.

Forest plot: number with any incontinence within first year (subjective). (Updated from graph 1.1 in Cochrane review. 42)

FIGURE 20.

Forest plot: number reporting SUI at clinic visit within first year. (Updated from graph 1.16 in Cochrane review. 42)

FIGURE 21.

Forest plot: number treated conservatively. (Updated from graph 1.5 in Cochrane review. 42)

FIGURE 22.

Forest plot: number treated with drugs. (Updated from graph 1.6 in Cochrane review. 42)

FIGURE 23.

Forest plot: number treated with surgery. (Updated from graph 1.7 in Cochrane review. 42)

FIGURE 24.

Forest plot: number whose treatment was changed after urodynamics. (Updated from graph 1.8 in Cochrane review. 42)

FIGURE 25.

Forest plot: number with urgency or urgency incontinence after treatment. (Updated from graph 1.11 in Cochrane review. 42)

FIGURE 26.

Forest plot: number of AEs/complications after treatment. (Updated from graph 1.18 in Cochrane review. 42)

The authors’ conclusions from the Cochrane review included the following:42

When women with incontinence are assessed using urodynamics in addition to clinical methods, they are more likely to receive different treatment, and to have their management plan changed. However, the evidence was not conclusive in showing whether these differences in management resulted in differences in health outcomes, such as incontinence, quality of life or economic outcomes after treatment compared to women who did not have urodynamic tests.

Clement et al. ,42

The addition of the data from INVESTIGATE-I adds weight to the conclusion relating to changes in treatment (see Figure 24), since more women in the IUT arm received conservative and drug treatments than those in the control arm (see Figures 21 and 22). There was no significant difference in the proportion of women treated by surgery overall (see Figure 23); although fewer women in the IUT arm of INVESTIGATE-I received surgery, this analysis is dominated by one of the larger studies. 36

The review found no statistically significant differences in the rate of UI symptoms in the first year after treatment, and the addition of data from INVESTIGATE-I would not change this conclusion (see Figures 19 and 20). No other study had used our primary outcome ICIQ-FLUTS, so the meta-analysis cannot add to our pilot trial results. The Cochrane reviewers also indicate that:

in order to give a definitive answer to the question of whether urodynamic studies are no better than clinical assessment in significantly reducing incontinence in women at one year follow up, a trial of 3222 participants would be required. Assuming the incontinence event rate is similar to that of the four trials already included in this analysis, 1611 patients per arm would reduce the confidence interval of the risk ratio to ± 10% [RR would be 1.02 (95% CI 0.94 to 1.10)]

Clement et al. ,42

This calculation differs from that we produced based on the pilot trial; this is because the outcome used (incontinence or not, at 1 year) is binary and this usually requires a larger sample size than a numeric scale. In addition, the reviewers have chosen a precision of ± 10% around the risk ratio as the criteria for deriving the sample size; it is not clear why this would make such a trial definitive.

However, bearing in mind the anticipated size of a definitive trial to follow INVESTIGATE-I, and the inevitable narrowing of the uncertainty such a large study will provide, it would have a high likelihood of narrowing the uncertainties present in the majority of these comparisons and hence of answering the question of the role of invasive urodynamic testing in women with SUI or stress predominant MUI.

Why should further research in this area be commissioned?

The current position of invasive urodynamic testing in the diagnostic pathway for UI is not agreed and practices vary considerably. The existing evidence base to guide practice is limited, and several systematic reviews have concluded that there is a need for large clinical trials to establish clinical utility; patients and clinicians in a James Lind Alliance working partnership also identified this as an area of significant uncertainty and a research priority.

While currently the majority of clinicians managing patients with SUI in secondary care see invasive urodynamic testing as essential prior to surgical treatment, many also recognise the lack of evidence to support this view. The mismatch between clinicians’ and patients’ views over the application of invasive testing in this area justifies urgent attention.

We believe that this feasibility study and the lessons learned will facilitate the effective delivery of a definitive trial to address the continuing uncertainties regarding invasive urodynamic testing in women with SUI, which may therefore have a significant impact on the delivery and cost of continence services in the UK in the future.

Contributions of authors

Paul Hilton is the lead grant holder, he conceived the study, led on the protocol development, questionnaire design and writing the manuscript, and approved the final version for publication.

Natalie Armstrong, Denise Howel, Douglas G Tincello, Malcolm G Lucas, Brian S Buckley, Christopher R Chapple and Elaine McColl are coholders of the grant, and along with Luke Vale contributed to protocol development and to writing the manuscript, and approved the final version for publication.

Catherine Brennand was the trial manager, contributed to protocol development, questionnaire and database design, and to writing the manuscript, and approved the final version for publication.

Jing Shen, Andrew Bryant and Tara Homer contributed to statistical and economic analyses, as well as to writing the manuscript, and approved the final version for publication.

Natalie Armstrong additionally led on the interview studies.

Denise Howel additionally led on the statistical analysis.

Luke Vale additionally led on the health economic analysis.

Elaine McColl additionally led on the survey questionnaire design and formatting.

Paul Hilton, Natalie Armstrong, Douglas G Tincello, Malcolm G Lucas and Christopher R Chapple additionally contributed to data acquisition.

Study governance references

Study outputs

Disclaimers

This report presents independent research funded by the National Institute for Health Research (NIHR). The views and opinions expressed by authors in this publication are those of the authors and do not necessarily reflect those of the NHS, the NIHR, NETSCC, the HTA programme or the Department of Health. If there are verbatim quotations included in this publication the views and opinions expressed by the interviewees are those of the interviewees and do not necessarily reflect those of the authors, those of the NHS, the NIHR, NETSCC, the HTA programme or the Department of Health.

Illustrative example of the cost–utility analysis: imputed values used for missing case report form data with Short Form 6D quality-adjusted life-year scores

Table 23 presents the cost–utility results using imputation for missing CRF data and the SF-6D to measure outcome data. The no IUT arm was dominated by the IUT arm as the no IUT arm had higher average cost but lower average QALY, however the probability of the IUT being cost-effective decreases as the society’s willingness to pay for a QALY threshold increases.

Figure 27 presents the results of the bootstrapping simulation, which addresses the uncertainty around costs and effects. As the majority of the iterations generated from the bootstrapping simulation were in the southern quadrants, it suggested that the IUT arm tended to incur less costs than the no IUT arm. The average of the cost and QALY pair simulations is situated close to the y-axis, indicating that there is not a significant difference in QALY values between the IUT arm and the no IUT arm. This supports the findings from the t-test conducted on the mean QALY differences between the two treatment groups in the main body of the monograph. This highlights the uncertainty around the cost–utility results and is further supported by the cost-effectiveness acceptability curve. Figure 28 demonstrates that if society had zero willingness to pay for an additional QALY then the IUT was 96% likely to be cost-effective; as society’s willingness to pay for a QALY increased, the likelihood of the IUT being cost-effective decreased.

FIGURE 27.

Incremental cost–utility scatterplot with imputation of CRF data with SF-6D QALY scores: IUT vs. no IUT.

FIGURE 28.

Cost-effectiveness acceptability curve with imputation of CRF data: IUT vs. no IUT.

Illustrative example of the cost–utility analysis: complete case analysis

Table 24 presents the cost–utility results using complete case analysis. The results from this analysis need to be interpreted with caution as only 81 patients had complete information (IUT arm n = 30; no IUT arm n = 51), this means that the analysis is vulnerable to outliers in the data. The cost of an additional QALY gained from the IUT is £4944 when compared with no IUT. At zero willingness to pay, the IUT option is not cost-effective when compared with no IUT, however, the cost-effectiveness of the IUT increases as the willingness-to-pay threshold increases.

Figure 29 represents the bootstrapping results from the complete case analysis. Similarly to the previous analyses we can clear see the majority of the iterations are in the northern quadrants which suggests that the IUT is more expensive on average than no IUT. The complete case analysis has generated different incremental cost results because only two patients in the IUT group did not have surgery as a treatment option, the other 28 patients in the IUT group had the IUT and surgery and thus incurred a higher cost on average. Again there is uncertainty around the QALY difference between the two groups, with the average of the iterations positioned close to the y-axis suggesting there is no evidence of QALY difference between the two groups. Figure 30 represents the cost-effectiveness of the no IUT at a zero willingness-to-pay threshold which differs from previous analyses, in this analysis the IUT is only 21% cost-effective. However, as the willingness-to-pay threshold increases so does the cost-effectiveness of the IUT.

FIGURE 29.

Incremental cost–utility scatterplot for complete case analysis: IUT vs. no IUT.

FIGURE 30.

Cost-effectiveness acceptability curve for complete case analysis: IUT vs. no IUT.

Illustrative example of the cost–utility analysis: imputed values used for sensitivity analysis

Sensitivity analysis was conducted around the unit costs used in the base-case analysis; Table 25 presents the cost-utility results using the alternative costs. Within the sensitivity analysis the cost of containment products was included for patients in the IUT group who had not received surgery as a treatment option and were not classed as watchful waiting. (It was previously assumed that patients who were classed as watchful waiting would use containment products.) It was assumed that these patients would still be classed as incontinent and would need containment products. The analysis here used higher unit costs to assess the impact unit costs have on the cost-effectiveness of the IUT. The cost per QALY for the IUT compared with no IUT is £5046. At zero willingness to pay the IUT option is cost-effective when compared with no IUT however the cost-effectiveness of the IUT decreases as the willingness-to-pay threshold increases.

Figure 31 represents the bootstrapping results from the sensitivity analysis. The majority of the bootstrapping iterations are positioned in the southern quadrants illustrating the cost savings for the IUT compared with no IUT. The position of the average result from the bootstrapping iterations suggests again that there is no significant difference in QALY values for the IUT compared with no IUT. This sensitivity analysis has resulted in the biggest QALY difference between the IUT and no IUT however the difference is less than 0.01. At zero willingness to pay the IUT is 89% cost-effective, the cost-effectiveness of the IUT decreases as the willingness-to-pay threshold increases, as highlighted in Figure 32.

FIGURE 31.

Incremental cost–utility scatterplot for sensitivity analysis: IUT vs. no IUT.

FIGURE 32.

Cost-effectiveness acceptability curve for sensitivity analysis: IUT vs. no IUT.