Devices for remote continuous monitoring of people with Parkinson's disease: a systematic review and cost-effectiveness analysis

Pharmacological treatment

Levodopa is the most commonly prescribed treatment for managing the motor symptoms of PD in the early stages. 2 However, it may be associated with significant motor complications, including response fluctuations and dyskinesias (involuntary movements), particularly after long-term use. Response fluctuations are characterised by large variations in motor performance, with normal function during the ‘on’ period, and weakness and restricted mobility during the ‘off’ period. ‘Wearing off’ of the drug or ‘End-of-dose’ deterioration with progressively shorter duration of benefit can also occur over time. Sleep disturbances such as insomnia, nocturia (night time urination) and restless leg syndrome (‘jumping’ of the legs and/or arms) can be caused by ‘wearing-off’ periods during the night. Dopaminergic therapies can also cause non-motor adverse effects, such as impulse control disorders, excessive sleepiness or sudden onset of sleep and psychotic symptoms such as hallucinations and delusions. Patient preferences are key to treatment decisions; the benefits of treatment must be balanced against the potential side effects.

Dopamine agonists, monoamine oxidase type B (MAO-B) inhibitors or catechol-O-methyltransferase (COMT) inhibitors are offered as additional treatment for people who have developed dyskinesia or motor fluctuations despite optimal levodopa therapy. If the dyskinesia remains uncontrolled, amantadine can be considered.

The NICE guideline for PD in adults recommends adjusting medicines to reduce the occurrence of daytime sleepiness or sudden onset of sleep, having first sought advice from a healthcare professional with specialist expertise in PD. Modafinil should be considered to treat excessive daytime sleepiness if a detailed sleep history has excluded reversible pharmacological and physical causes. Clonazepam or melatonin may be considered to treat rapid eye movement sleep behaviour disorder if a medicine’s review has addressed possible pharmacological causes. 2

Advanced Parkinson’s disease

The symptoms of advanced PD may still be responsive to adjustments in the dose and combination of levodopa with adjuvant MAO-B and/or COMT therapies. 6 Intermittent apomorphine injection and/or continuous apomorphine infusion may also be considered for people with advanced PD. Deep brain stimulation (DBS) can be considered in people with late-stage PD whose symptoms do not respond adequately to best medical therapy. Clinical experts highlighted that this procedure is only normally considered for people who have been taking medication for PD for over 5 years.

Levodopa–carbidopa intestinal gel (LCIG) is currently available through an NHS England clinical commissioning policy. It can be considered in certain people with advanced levodopa-responsive PD with severe motor fluctuations that have not responded to available medications. NICE recommends that this policy is reviewed in light of NG71 (NICE guidelines for PD in adults, section 1.8.4). 2

Personal KinetiGraph Movement Recording System (Global Kinetics)

The PKG Movement Recording System (Global Kinetics) is a Class IIa CE marked system that uses a wrist-worn PKG watch/logger that continuously measures movement over a period of at least 6 days. It is intended to quantify kinematics of movement disorder symptoms in conditions such as PD, including tremor, bradykinesia (slowness) and dyskinesia (involuntary movements). It has event markers for medication reminders and patient acknowledgement. It is also intended to be used to monitor activity associated with movement during sleep. The company states that PKG is an adjunct to clinical practice and should be used in combination with patient and healthcare consultation. They envisage that the PKG is used twice a year, although there is some uncertainty about the best time to use the PKG; varying between every 6 months, regardless of current symptoms, to only when there is a suspicion that medication is not adequately controlling symptoms.

Healthcare professionals can order the PKG online. The company sends the watch directly to the person who will wear it (for a period of at least 6 days), also providing a paid and addressed envelope for the watch to be returned to the company. Data are then extracted and processed by cloud-based algorithms and a report is generated for the healthcare professional to view online.

The PKG measures bradykinesia, dyskinesia, tremors, motor fluctuations, immobility and when the watch is not being worn. It can also prompt the user to take their medication at prescribed times and the user can register when they have taken their medication. As well as providing the raw data, it can generate a report based on movement over a 6-day period using validated proprietary algorithms. The report includes summary graphs showing measurements over time and a summary following results, along with a suggested target range for interpretation:

• a bradykinesia score;

• a dyskinesia score;

• a fluctuation dyskinesia score;

• percentage of time with tremor; and

• percentage of time immobile (indicative of daytime sleepiness).

The company has stated that new versions of the technology will include 24-hour measurements of sleep-related functions. The device is intended to be interpreted only by trained technicians or clinicians, and as an aid to existing clinical methods. It is not intended to be the sole or primary means of clinical assessment. The company states that the PKG is suitable for 70–80% of PwP, particularly when managing patients remotely, managing complex patients and those being considered for (or already on) advanced therapy. The company does not recommend use of the technology for patients who have restricted movement (e.g. patients confined to bed or wheelchair users) or for patients who operate heavy machinery for prolonged periods.

The company provides healthcare professionals with education and training, and state that healthcare professionals should complete an average of 15–20 PKGs to be proficient, supported by an eLearning module, which takes approximately 1–2 hours.

Kinesia 360 motor assessment system (Great Lakes NeuroTechnologies)

The Kinesia 360 motor assessment system (Great Lakes NeuroTechnologies) is a Class I CE-marked system that monitors physical motion and muscle activity to quantify movement disorder symptoms and assess activity. The Kinesia 360 system consists of a tablet, sensors and charge pad, USB cable and charge pad power cable. Sensors worn on the wrist and ankle combined with a mobile application continuously record data, including bradykinesia, dyskinesia and tremor. While the device can be worn at night, the motor sensors can record up to 16 hours of motion data continuously before they need to be recharged. Typical use involves wearing the sensors during the day and recharging/data upload overnight. The mobile application also includes electronic diaries for capturing patient-reported outcomes and customisable medication diaries.

When the Kinesia Sensor bands are returned to the charging pad, data from the motion sensors are automatically downloaded and then uploaded to the Kinesia Web Portal and algorithms are used to detect symptoms and calculate severity scores. Clinicians can view web-based reports that include:

• a dyskinesia score;

• total and percentage of time with tremor;

• total and percentage of time at rest;

• total and percentage of active time (but not walking);

• number of steps;

• a symptom summary report that displays how tremor, slowness, dyskinesia and walking change over time; and

• a dose report that shows how tremor, slowness, dyskinesia and walking change as a function of different medication or therapy doses.

Healthcare staff can be trained in Kinesia 360 in approximately 30 minutes.

KinesiaU motor assessment system (Great Lakes NeuroTechnologies)

The KinesiaU motor assessment system (Great Lakes NeuroTechnologies) measures tremor, slowness and dyskinesia using a smartwatch and smartphone application. Patient symptoms can be monitored continuously during activities of daily living (iOS only as of February 2022) and discretely during standardised tasks (iOS and Android). Patients can view reports in real-time and healthcare professionals can view their patients’ data remotely through the KinesiaU provider portal. The product is to be used only under the direction of a qualified clinician and all changes to therapy regimens are to be based solely on the clinical judgement of the clinician. The company is seeking CE-marking. A number of new features are planned to be added to the KinesiaU system, including additional data reports, enhanced patient diaries, electronic health record integration, patient medication reminders and continuous monitoring for Android smartwatches.

The reports rate the severity of tremor, slowness and dyskinesia symptoms according to good, mild, moderate and severe categories. This can be measured through specific active tasks or through continuous recording. To start a continuous (all day) recording, the user must tap the ‘Continuous’ button on the home screen. The smartwatch application must be kept open during the recording. Active tasks may be performed during the continuous recording.

Reports can be produced throughout the day and over the course of days, weeks and months in response to therapy and activities. The report page on the smartwatch application displays the severity of the selected symptom (tremor, slowness and dyskinesia) averaged for the selected time range. The symptoms can be displayed individually or averaged together and shown as ‘All symptoms’. The mobile application also includes customisable medication and exercise diaries, which can be added to the report. Patients can view reports in real-time and share reports (PDF format) with their healthcare professionals.

Healthcare staff can be trained in KinesiaU in approximately 30 minutes.

PDMonitor (Parkinson’s disease Neurotechnology)

The PDMonitor system (PD Neurotechnology) is a Class IIa CE-marked system that measures activity/posture, bradykinesia, freezing of gait, gait disturbances, wrist tremor, leg tremor, dyskinesia and ‘on’ and ‘off’ periods. The duration and frequency of use are decided by the physician. The device should be removed when performing intense fitness activities.

The PDMonitor system consists of the SmartBox, five monitoring devices and a PDMonitor mobile application. The devices are worn on both wrists, both ankles and one is worn on the waist, and acquire movement data for assessing motor symptoms. The PDMonitor SmartBox is a docking station for charging the monitoring devices, collecting, storing and processing data and uploading it to the PD Neurotechnology storage service. The SmartBox must be connected to the PD Neurotechnology storage service to be properly configured, either via an ethernet cable or an available Wi-Fi network; this requires an internet connection. A web-based application can be used by healthcare professionals to view and download patient reports. The PDMonitor mobile application is an electronic diary for medications, diet and symptoms related to PD. It also provides a summary of daily activity as recorded by the PDMonitor system.

An ‘Induction and Usage Training’ is offered to healthcare professionals, either in groups or in person, to help them understand the PDMonitor system. There is also a physician user manual for the physician tool.

STAT-ON (Sense4Care)

The STAT-ON (Sense4Care) is a Class IIa CE-marked, waist-worn inertial recorder, configured by a doctor and used by the patient in clinical, ambulatory or home environments. It measures motor disorders and events when worn by someone with PD, but does not measure tremor. The device measures dyskinesia, ‘on’ and ‘off’ periods, gait parameters (including bradykinesia and freezing of gait), falls, energy expenditure and posture. It can also register when medication has been taken and up to 10 alarms per day can be set.

Health professionals should manage the use of the device; they should provide the sensor to the user correctly configured and charged. Results can be used to adjust or evaluate a therapy or to adjust a person’s diet.

The STAT-ON system consists of a monitoring device, its base charger, a belt and a mobile application. The device collects data and uses artificial intelligence algorithms to process it. Results are stored in its internal memory. The smartphone application connects to the STAT-ON device via Bluetooth. The mobile application is used for configuring the system and for downloading the data. It also sends the data as a report via e-mail.

The company has advised that the STAT-ON could be worn during the night to monitor movement. The user should wear the device for a minimum of 5 days (ideally for 7 days), totalling a minimum of 24 hours over the 5 days to generate sufficient data. After this, a report can be generated at any time. A health professional can download the report to their phone using the STAT-ON application, which automatically generates a report of the motor state and symptoms during time of use. Reports include a summary of activity and prevalence of symptoms during the monitored period, including:

• total freezing of gait episodes and average number of episodes per day;

• average minutes walking and number of steps per day;

• number of falls;

• time in ‘Off’/Intermediate/‘On’; and

• time with dyskinesia.

As well as numerically, data are also presented in graphs. In addition to a summary report, a more detailed report with further data analysis can also be produced.

The STAT-ON device is not indicated for children or for PwP with Hoehn and Yahr (H&Y) Scale 5. The device should not be worn by a person in a wheelchair or using crutches as the results will not be valid.

Training sessions last an hour and a half. Quick guides are provided for healthcare professionals and quick videos to understand how the system is configured. A complete graphical document is also provided with user cases, examples and how to interpret the report.

Association outcomes

Association between outputs of remote monitoring (such as bradykinesia score, dyskinesia score, sleep disturbance and tremor measures) and clinical measures, including:

• Rating scales such as the UPDRS, MBRS and the H&Y scales.

• Other measures of bradykinesia and dyskinesia, sleep disturbance or tremor.

• Clinical assessment.

• Patient-reported symptoms.

• Any measure of association, such as sensitivity and specificity, measure of correlation or results of regression models.

Intermediate impact of monitoring

All impacts on clinical decision-making:

• Changes in therapy (e.g. starting levodopa).

• Modification of current therapy dose or timing (primarily levodopa, and including potential changes to therapy identified, which were contraindicated or declined by the patient).

• The use of additional interventions (including pharmacological and non-pharmacological interventions for management of motor and non-motor symptoms associated with PD).

• Adherence to medication.

• Number and length of clinical appointments.

• Incidence of remote appointments.

• Ease of use/acceptability by clinicians.

Clinical outcomes

Measurable clinical impact of using the technologies:

• Change in clinical symptoms.

• On–off periods.

• UPDRS, MBRS, H&Y scores.

• Dyskinesia and bradykinesia scores

• Sleep disturbance.

• Tremors.

• Number and length of hospital admissions.

• Other morbidities (including falls, hip fracture, cognitive functioning, other non-motor outcomes, adverse effects of treatment).

• Mortality.

Patient- and carer-reported outcomes

• Health-related quality of life (HRQoL).

• Ease of use and acceptability for patients and carers.

• Patient and carer experience (including quality of care, patient and carer satisfaction and engagement, e.g. impact on discussions about symptom management, communication and relationship between patients and clinicians).

It was expected that data would be unavailable for many of these outcomes. They are listed here to present a complete list of outcomes of interest.

Costs

Costs for consideration may include:

• Costs related to using the intervention (including any time analysing and storing data, communicating results and arranging for use of the technology).

• Cost of staff training.

• Cost of review appointments.

• Cost of further tests.

• Cost of treatment (including costs of any adverse events).

Search strategies

Comprehensive searches of the literature were conducted to identify all studies relating to the use of the remote continuous monitoring devices PKG, Kinesia 360, KinesiaU, PDMonitor and STAT-ON for monitoring motor symptoms in PwP. An Information Specialist (HF) designed the search strategy in Ovid MEDLINE in consultation with the research team. The strategy consisted of terms for the population, which were then combined with specific interventions of interest, or broader terms that reflect remote monitoring technologies. Text word searches for terms appearing in the title, abstract or keyword fields of database records were included in the strategy alongside searches of relevant subject headings. Date, language and study design limits were not applied. The final MEDLINE strategy was adapted for use in all resources searched.

The searches were carried out on 1 February 2022. The following databases were searched: MEDLINE(R) ALL; EMBASE; EconLit; APA PsycInfo; Cochrane Central Register of Controlled Trials (CENTRAL); Cochrane Database of Systematic Reviews (CDSR); Database of Abstracts of Reviews of Effects (DARE); Health Technology Assessment (HTA) Database; NHS Economic Evaluation Database (NHS EED); and the International HTA Database.

In addition, the following resources were searched for ongoing, unpublished, or grey literature: ClinicalTrials.gov; EU Clinical Trials Register; and the World Health Organization (WHO) International Clinical Trials Registry Platform. All search strategies are presented in full in Appendix 1.

Search results were imported into EndNote 20 (Clarivate Analytics) and deduplicated. As a supplementary search method, reference lists of relevant reviews were scanned to identify additional potentially relevant studies. Company websites were also searched for additional relevant studies.

The companies who manufacture or control the devices of interest were contacted (via NICE) and invited to supply any material they considered relevant. This could include journal articles (published or unpublished), conference abstracts, lists of papers, study data and details of unpublished, ongoing or planned studies. These company submissions were examined for relevant eligible material.

Selection criteria

Two reviewers (RW and NM) independently screened all titles and abstracts using Covidence systematic review management software. Full papers of any titles and abstracts that were thought to be relevant were obtained where possible and independently screened by the two reviewers according to the criteria below. Any disagreements were resolved by consensus or by consulting a third reviewer (MS). Conference abstracts were included where sufficient data were reported to confirm eligibility.

Scoping eligible studies

Studies that met the inclusion criteria were scoped in order to prioritise studies reporting the most relevant outcomes for full data extraction. Studies reported only as abstracts were not subject to full data extraction, but are tabulated in appendices.

Data extraction

A data extraction form was developed, piloted and finalised to extract study and patient characteristics and eligible outcomes. Data were extracted by one reviewer (RW or MS) and independently checked by a second reviewer (MS or RW), with discrepancies resolved through discussion. Data from relevant studies with multiple publications were extracted and reported as a single study, where it was possible to determine that the publications included the same patients. The most recent or most complete publication was used in situations where we could not exclude the possibility of overlapping populations across separate study reports.

Quality assessment

The quality of the diagnostic accuracy studies was assessed using the Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) tool. 14 QUADAS-2 evaluates both risk of bias (associated with the population selection, index test, reference standard and patient flow) and study applicability (population selection, index test and reference standard) to the review question.

Risk of bias in randomised controlled trials (RCTs) was assessed using the latest version of the Cochrane risk of bias tool. 15 A tool for assessing the risk of bias of non-randomised studies was developed using relevant criteria as outlined in CRD’s guidance on undertaking systematic reviews. 16

Quality assessment was performed by one reviewer (RW or MS) and independently checked by a second reviewer (MS or RW). Disagreements were resolved through discussion. Quality assessment was performed only for included studies with full publications. Conference abstracts were not quality assessed due to the lack of information to merit a full assessment. Quality assessment was not performed for studies reporting association outcomes without reporting diagnostic accuracy.

Methods of data synthesis

The results of data extraction were presented in structured tables and as a narrative summary. A broad thematic synthesis was used to identify key issues arising from the extracted evidence, including key areas of agreement or disagreement across the included literature.

A statistical synthesis using meta-analysis was proposed in the protocol. However, due to the substantial diversity in study populations, conduct and outcomes reported, it was not possible to combine any studies in meta-analyses. Therefore, a narrative and thematic synthesis approach was used throughout.

Analysis of individual participant data

One clinical trial of PKG has deposited its original trial data on a repository for reanalysis. 17 The authors of the study gave permission to the External Assessment Group (EAG) to reanalyse the trial data, and have supplied it to the EAG.

The supplied data were checked, compared to the published results and reanalysed. Linear regression was used to analyse continuously distributed outcomes (e.g. UPDRS score) by considering the change from baseline to follow-up time for each outcome, and analysing the mean difference in change from baseline between PKG and non-PKG patients. Logistic regression was used for dichotomous outcomes (e.g. change in medication), analysing the odds ratio between PKG and non-PKG patients. Analyses were adjusted for potential confounding factors, chiefly the number of clinical visits and duration of PD (see Woodrow individual participant data for further details).

Methods for estimating quality of life

Health-related quality of life associated with disease severity was estimated. It was expected that measures of disease severity would be expressed in terms of different instruments of disease activity (e.g. UPDRS, Modified-UPDRS, MBRS, H&Y). In accordance with the NICE reference case, HRQoL utility values should be based on the EuroQoL – EuroQol-5 Dimensions (EQ-5D) instrument. Therefore, a pragmatic review of utility studies was carried out to identify relevant studies that (1) directly estimate EQ-5D utility values, and (2) establish the relationship between EQ-5D utility and measures of disease severity (including mapping studies).

Diagnostic accuracy

Seven papers reporting diagnostic accuracy data [sensitivity and specificity, or area under the curve (AUC)] for PKG were identified. 39–45 One further conference abstract was found,46 which is not discussed here due to limited reporting of data.

The QUADAS-2 risk-of-bias assessment of these studies is summarised in Table 3.

The risk-of-bias assessment identified substantial concerns with the included diagnostic accuracy studies. Reporting was frequently poor, leading to an ‘Unclear’ assessment and, where risk could be assessed, studies were often at high risk of bias. Four of the studies were case-control studies,40,41,43,44 which are generally accepted as having high risk of bias, as the patient’s condition is known before the PKG assessment is performed. In most studies the reference standard was not described in detail, often limited to just stating that it was clinical opinion. Similarly, the exact test being assessed was rarely described. The EAG have assumed that it was the output of the PKG device in some form, but it is unclear whether the output or algorithm used is the same as for the current device in actual use, hence our ‘Unclear’ classification for the applicability of the index test in these studies. There were also concerns with the flow and timing component of risk-of-bias assessment, because it was generally not clear when the reference standard and index tests were performed, and whether each was assessed blinded to the results of the other.

It should be noted, however, that some of these risk of bias issues may be due to the nature of the studies and the condition. There is no clearly established reference standard for measuring PD symptoms beyond clinician and patient assessment (e.g. by using UPDRS). This is unlikely to be a perfect reference standard. Indeed, a possible benefit of PKG (and the other technologies) is that they may provide a more accurate evaluation of symptoms than patient recall or clinical opinion; this cannot be easily determined from a diagnostic accuracy study. Also, the studies do not appear to have been designed as formal diagnostic accuracy studies. Most were proof-of-concept studies where diagnostic accuracy data were reported alongside other information. This may explain why some aspects of bias risk were not clearly reported.

A summary of the diagnostic accuracy data reported in the included studies is shown in Table 4. None of the studies were from the UK, but most were from Australia or the USA, and so are likely to have results that generalise to the UK population. Studies varied substantially in size: from 26 to 373.

Diagnostic accuracy results were generally poorly reported. None reported actual numbers of true positives, etc., and most did not report confidence intervals (CIs) or standard errors for the reported estimates. Each study examined a different outcome, with no replication of outcome across different studies. Most studies used some form of clinical judgement as the reference standard, generally using UPDRS to measure symptoms.

The three studies that reported bradykinesia, dyskinesia and tremor39–41 showed high diagnostic accuracy of PKG to detect these, with sensitivities above 90% and specificities ranging from 83% to 92.9%. This suggests that PKG is able to measure key PD motor outcomes. The one study of sleep disturbance showed slightly poorer diagnostic accuracy (80% sensitivity, 86% specificity),44 suggesting that PKG may not be as effective at identifying people with sleep disturbance.

Three studies examined the diagnostic accuracy of PKG for making treatment decisions. 42,43,45 The two studies by Khodakarami showed that PKG had a reasonably good ability to identify patients’ levodopa response (92% AUC) or need for device-assisted therapy (AUC 93%). One small study showed slightly poorer performance for accuracy of treatment classification (AUC 83.1%),45 but the clinical relevance of this classification was unclear.

Association outcomes

Eleven papers reporting association outcomes for PKG were identified. 47–57 Ten further conference abstracts were found, but are not discussed here due to limited reporting (see Appendix 4, Table 54). 58–67 A summary of the results of studies reporting association data for PKG is reported in Tables 5 and 7. One study was from the UK; others were mostly from Europe, Australia or the USA. Studies varied in size; from 18 to 228 patients.

In general, PKG bradykinesia score, per cent time in bradykinesia (PTB) and per cent time in tremor (PTT) scores were moderately correlated with UPDRS III scores (across three studies). Bradykinesia score and PTB was also moderately correlated with Parkinson’s Disease Quality of Life 39 Questions (PDQ39) scores (two studies). There was a statistically significant correlation between PKG dyskinesia score and UPDRS II in one study. PKG dyskinesia score was also significantly correlated with modified abnormal involuntary movements scale (AIMS) and PKG bradykinesia score was significantly correlated with bradykinesia measured by the ‘dot slide’ test (one study). In the same study, a subgroup of patients with bilateral PD, there was significant correlation between ‘global median dyskinesia score’ and UPDRS IV, and ‘global median bradykinesia score’ and UPDRS III.

However, in one study the Wearing-off Questionnaire-9 (WOQ-9) had a very weak, non-significant correlation with the PKG fluctuation and dyskinesia score (FDS) and dyskinesia score. Another study found that PKG fluctuator scores significantly differentiated early fluctuators and troublesome fluctuators, as well as dyskinetic and non-dyskinetic patients, but could not discriminate subtler motor fluctuations.

Results relating to sleep outcomes were more mixed. In one study, high Epworth Sleepiness Score was correlated with PKG proportion of time immobile (PTI). In the two studies that reported them, correlations between PKG variables and 3-day daytime sleepiness diaries were generally weak and non-significant. In a subgroup of patients with excessive daytime sleepiness, the PKG’s parameters for quantity and quality of night-time sleep correlated significantly with the total burden of non-motor symptoms of PD as measured by Non-Motor Symptoms Questionnaire (NMSQuest) (one study). In non-sleepy patients there was no significant correlation. There was also a moderate to high (though non-statistically significant) correlation between PKG night-time sleep markers and the Parkinson’s Disease Questionnaire-8 (PDQ8) in the excessive daytime sleepiness group. In a subgroup of patients who were immobile for > 30 minutes/day and underwent ambulatory daytime polysomnography (n = 7), periods of immobility on PKG were highly correlated with detection of sleep by polysomnography.

The ratio of medication acknowledgements/number of doses was strongly correlated with ratings of Impulsive-Compulsive Behaviours in 19/25 patients; however, 6 patients were clear outliers and fell into the false negative group; these patients had normal response ratios, but high Impulsive-Compulsive Behaviour scores.

The Bergquist (2018)59 conference abstract is worthy of note; it describes preliminary data from the ongoing WestPORTS registry study, which compares randomly selected PwP in West Sweden (n = 154) with retrospective data from clinically motivated recordings in PwP suspected to have motor fluctuations (n = 248). The PKG scores were significantly different between the two populations: median bradykinesia score 30.4 versus 23.0 (p = 0.014) and dyskinesia score 1.0 versus 3.0 (p < 0.0001) in the randomly selected population and clinically motivated recordings, respectively.

Intermediate impact of monitoring

Twenty-five studies reporting on the intermediate impact of monitoring were identified; 8 reported in full publications68–75 and 17 as conference abstracts. 76–92 The conference abstracts are not discussed here due to limited reporting (see Appendix 4, Table 57); however, results were largely consistent with those of the full papers.

The eight studies reported in full were assessed for quality using a tool developed for the review using relevant criteria. The results are summarised in Table 6.

Five of the six comparative studies (where PKG was compared against clinical assessment before reviewing the PKG data) had a low overall risk of bias. 70–73,75 However, one of the comparative studies did not clearly define the study inclusion criteria, attrition was high and it was unclear whether the clinician was blinded to PKG results at the time of outcome assessment. 74 The two uncontrolled studies had a high risk of bias. 68,69

A summary of the results of studies reporting the intermediate impact of monitoring is reported in Table 7. Two studies were from the UK; others were from Australia, the USA and Sweden.

Three studies reported the level of agreement between the PKG and clinical assessment/initial judgement; there was agreement in 54.5–90% of cases. 68,70,72 Hence, there appears to be considerable cross-study uncertainty in the consistency between PKG assessments and standard clinical assessments.

Six studies reported the proportion of patients for whom the PKG provided additional information leading to a change in the clinical management plan; this was the case in 31.8–79% of patients. 68,70,71,73–75 The most common treatment changes were the addition of at least one medication or a change in dosage while a small proportion of patients were referred for advanced therapy. This suggests that there will be a proportion of patients for whom PKG will lead to changes in management, but also a substantial proportion where management will be unchanged. There is considerable uncertainty concerning exactly how many patients will have changes to management if PKG is used. It was unclear from the publications how a decision to change, or not change, management related to patient symptoms, nor exactly what the changes were (such as how much levodopa dosage was adjusted).

One study assessed PKG use in virtual clinical appointments; 79% of virtual appointments were deemed successful (the outcome of the consultation was likely to have been the same as a face-to-face appointment). 69 Reasons for unsuccessful consultations included complex phase of disease, problems with the PKG, needing a blood pressure reading and speech problems.

Clinical outcomes

We identified 10 studies that reported on clinical outcomes related to PKG. As the original trial data were supplied for the Woodrow trial it is analysed in Woodrow individual participant data. The remaining studies are summarised in Other studies reporting results for clinical outcomes.

Woodrow individual participant data

Full data were made available for the Australian trial of PKG by Woodrow et al. 17 This was not randomised; rather, 12 centres were selected to either use PKG for the management of patients or use standard clinical practice. PKG clinics were generally those with existing experience of using PKG in practice. Patients were assigned to clinics based on location and convenience. Hence, the trial can be thought of as a quasi-randomised cluster trial. All patients wore the PKG smartwatch, and so were blind to which arm they were in. In total, 200 patients were recruited to the study. After withdrawals and patients with incomplete data were excluded, the EAG analysed 162 patients; 77 patients were managed using PKG, and 85 using standard care.

PKG measurements were taken for all patients, but only given to clinicians in the PKG arm. Patients were seen every 5 weeks, with PKG measurements taken before each visit until their PKG measurements were judged to be ‘in target’ (defined as bradykinesia score < 26 and dyskinesia score < 7 from PKG assessment), with a maximum of five consultations.

The risk of bias of the trial was assessed using the Cochrane risk of bias tool, and is reported in Table 8. Although Cochrane risk of bias is intended for RCTs, it was considered the most suitable tool for assessing the Woodrow trial. The main risk of bias in the trial was because it was not strictly randomised. Other aspects of the trial were judged to be at low risk of bias.

TABLE 8 - Risk-of-bias assessment for the Woodrow trial

Study	Woodrow17
Risk of bias arising from the randomisation process	High
Risk of bias due to deviations from the intended interventions	Low
Missing outcome data	Low
Risk of bias in measurement of the outcome	Low
Risk of bias in selection of the reported result	Low
Overall judgement of risk of bias	Some concerns

The supplied IPD was checked for potential bias problems, including imbalances across trial arms. When examining patient characteristics at recruitment, the EAG could not exactly match results presented in the trial publication,17 but inconsistences were small and most likely due to differences in how excluded patients were evaluated. The EAG found no substantial imbalance in patient characteristics between PKG and non-PKG patients, so, although the trial was not randomised, there does not appear to be any bias due to imbalance between arms. Missing data were largely confined to patients who were excluded or withdrew from the trial. There was no evidence of imbalance in missing data between arms.

The IPD included the following outcomes, which are reanalysed here:

• UPDRS (I, II, III, IV and Total):

  • part I covers non-motor aspects of daily living (e.g. depression and anxiety);

  • part II covers motor aspects of daily living (e.g. walking and eating);

  • part III is the full motor assessment; and

  • part IV covers motor complications (dyskinesia and ‘on–off’ times).

• Levodopa equivalent dose (LED).

• H&Y.

• Median bradykinesia score, active bradykinesia score and dyskinesia score.

• Montreal Cognitive Assessment (MOCA).

• NMSQuest.

• PDQ-39.

• The severity of predominantly non-dopaminergic symptoms in Parkinson’s disease (SENS PD).

• Percentage time with bradykinesia, dyskinesia or tremor.

• Percentage time inactive or immobile.

These are broadly the same outcomes reported in the trial publication, except time inactive was not reported in the paper so there is no clear evidence of reporting bias. Figure 2 shows the difference between PKG and non-PKG patients in terms of change in outcome from baseline, for all the outcomes reported above. The circles show the estimated difference between PKG and non-PKG arms, and the horizontal lines are 95% CIs. Results to the left of the red line indicate those favouring PKG. Results for LED are divided by 100, to fit on this plot. Full numerical results for this analysis are given in Appendix 5, Table 67.

FIGURE 2.

Results from the Woodrow trial: difference between PKG and standard care. BKS, bradykinesia score; DKS, dyskinesia score; SENS-PD, severity of predominantly nondopaminergic symptoms in Parkinson’s disease.

The results show that the use of PKG appears to improve UPDRS scores, particularly UPDRS III (by around 3.1 points) and IV (by around 1.2 points) and hence, total score. This is likely to be because PKG use is reducing time with bradykinesia (by 2.1 percentage points) dyskinesia (by 1.5 percentage points) and tremor (by 0.6 percentage points), although none of these reductions achieved statistical significance. Results for other outcomes are mostly in the direction of favouring PKG, but effect sizes are mostly small and CIs wide. Use of PKG appears to improve symptoms, without requiring substantial increases in levodopa dose.

The main exception to the general trend favouring PKG was for time inactive, which was higher in PKG patients than in non-PKG patients, by about 2.7% points. It is not clear why this discrepancy might occur. It was notably not reported in the original trial publication.

We examined models that adjusted for potential confounding factors, namely: age, sex, PD duration, UPDRS III at baseline and number of clinic visits during follow-up. This found that sex and age had no impact on results, but the other factors could alter outcomes. We reanalysed the data for all outcomes adjusting for PD duration, UPDRS III at baseline and number of clinic visits during follow-up. Results of this analysis are given in Appendix 5, Figure 11 and Table 67.

Overall, the results for the adjusted analyses were similar to the original analyses (Figure 2). The benefit of PKG was marginally reduced for some outcomes. For example, the benefit of PKG on UPDRS IV declined from 1.2 to 0.7 points. This might suggest that some of the observed benefit of PKG is because people in the PKG arm had more clinic visits.

Further analysis of the impact of both number of visits and baseline UPDRS III on outcomes was performed, by analysing outcomes separately for each number of visits and by quartiles of UPDRS III score at recruitment. These analyses are summarised in Appendix 5, Figures 12 and 13. These show that the benefits of PKG were mostly in patients who had poor UPDRS III scores initially, and those who required more visits before symptoms became in target. This suggests that PKG may be of most benefit to patients with more severe, and less manageable, symptoms. However, numbers of patients in each group were small (e.g. 48 patients had only one visit; 29 required four visits).

To match analyses performed in the trial publication, we also performed a subgroup analysis examining the impact of PKG on all outcomes according to whether patients were judged to be ‘in target’ or ‘out of target’ at baseline, using their PKG results (28 patients were ‘in target’ and 134 were ‘out of target’). The results of this analysis are summarised in Figure 3, and given in full in Appendix 5, Table 68.

FIGURE 3.

Impact of PKG in the Woodrow trial, by ‘in target’ status at baseline. BKS, bradykinesia score; DKS, dyskinesia score; SENS-PD, severity of predominantly nondopaminergic symptoms in Parkinson’s disease.

These results suggest PKG use predominately improves symptoms (particularly bradykinesia and UPDRS scores) in people who were not ‘in target’ and whose condition was not adequately controlled. For people whose condition was ‘in target’, there are no improvements in UPDRS or time in bradykinesia. However, for ‘in target’ patients on PKG, the percentage time in dyskinesia and LED were lower compared to non-PKG patients, although neither result was statistically significant because patient numbers were limited in this group.

This suggests that using PKG can be useful in improving UPDRS, by reducing bradykinesia in patients whose disease is not being adequately controlled, while possibly allowing for levodopa dose reduction and consequent reduction in dyskinesia in patients whose condition was already well-controlled.

The supplied IPD permitted analysis for three further dichotomous outcomes: change in medication, referral for device-assisted therapy (exact therapies were not reported) and ‘in target’ status. The first two were not included in the trial publication, but of relevance to this assessment. Odds ratios for these three outcomes are given in Table 9. Models adjusted for PD duration, UPDRS III at baseline and number of clinic visits during follow-up gave broadly similar results. The data were insufficient for analyses by target status at baseline.

TABLE 9 - Results for analysis of dichotomous outcomes in the Woodrow trial

Outcome	Number with PKG	Number with standard care	Odds ratio	95% CI
Change in medication	49	39	1.18	0.52	2.68
Referral for device-assisted technology	36	13	4.01	1.82	8.85
In target at follow-up	34	14	3.43	1.67	7.03

The results suggest that patients using PKG were substantially more likely to be ‘in target’ at follow-up and to be referred for device-assisted technologies. There was no clear evidence that patients using PKG were more likely to have a change in medication.

Patient and carer opinions

A total of eight studies reported patient or carer opinions on PKG. 68,69,71,73,78,91,93,94 The four studies reported in full also reported intermediate or clinical outcomes, so the quality assessment of these studies is reported in Table 6. 68,69,71,73 Four conference abstracts only reported survey results. 78,91,93,94 These have not been assessed for quality due to lack of information in abstracts.

The results are summarised in Table 11, with further data in Appendix 4, Table 63.

In surveys of patients and carers most agreed that PKG was generally easy to use. PKG appeared most useful to patients as a reminder to take medication, with between 75.3% (Joshi et al. )71 and 100% (Nahab et al. )73 agreeing that PKG was useful for that purpose. Results were more equivocal for whether PKG provided useful information on symptoms that the patients or carers could not themselves provide; between 59% (Dominey et al. )68 and 79% (Nahab et al. )73 of patients agreed this was the case. It is unclear whether this was because people felt the device was not providing useful information, or because their symptoms were such that the device did not provide extra information.

Patients were generally willing to use the PKG device again, although less so if required to cover the costs themselves (Nahab et al. ). 73 In the study that used PKG for the remote management of patients (Evans et al. ),69 patients were generally happy with the virtual clinic approach, suggesting that PKG could reasonably be used for fully remote monitoring and assessment.

Clinical opinions

A total of six studies reported clinicians’ opinions on the value of PKG. 71,73–75,87,90 These are summarised in Table 12. The studies reported in full also reported intermediate or clinical outcomes, so the quality assessment of these studies is reported in Table 6. Two conference abstracts only reported survey results. 87,90 These have not been assessed for quality due to the lack of information in the abstracts.

Clinicians’ opinions of PKG were generally less favourable than patient opinions. Most clinicians agreed that use of PKG improved dialogue with patients. 71,73,75 They were generally less convinced that PKG improved their assessment of symptoms or need for changes in therapy. Four of the studies found that PKG provided additional useful information in only 32–47% of patients. 71,74,87,90 Nahab73 was an outlier, where 89% of clinicians felt that PKG improved their ability to assess impact of therapy.

It should be noted that these clinical opinions somewhat contradict the actual evidence on changes in treatment (see Intermediate impact of monitoring and Table 7), where many patients did have a change in treatment as a result of PKG use.

Summary of Personal KinetiGraph evidence

Much of the published evidence for PKG consists of either diagnostic accuracy or association studies, mostly designed as proof-of-concept studies to show that PKG can usefully measure symptoms associated with PD. This evidence broadly suggests that PKG can accurately measure dyskinesia, bradykinesia and tremor. Sensitivity estimates (for bradykinesia and fluctuations in dyskinesia) were high (near 100%), but specificity estimates were generally lower (83–87.5%), suggesting a possibility that PKG may slightly overdiagnose symptoms. PKG appears to have lower accuracy for measuring sleep disturbance. Diagnostic accuracy studies also suggest that PKG has reasonable accuracy when diagnosing clinical factors, such as levodopa response and need for device-assisted therapy. It should be noted that the quality of diagnostic evidence is low and lacks replication, with most outcomes studied in only one study. This is partly due to the limitations of performing formal diagnostic accuracy studies in this field, as all outcomes are usually assessed by clinicians and patients, with no clear objective reference standards.

The results of the association studies generally supported those of the diagnostic accuracy studies; PKG bradykinesia, dyskinesia and tremor scores were generally moderately correlated with UPDRS scores, while the evidence for sleep and impulse control behaviours was less promising. The evidence on the impact of PKG on sleep disturbance, and other non-motor aspect PD, was extremely limited.

The studies investigating the intermediate impact of PKG were generally of good quality (5/8 studies had a low risk of bias). The evidence from those studies indicates that PKG is being used by clinicians to guide treatment decisions, primarily the addition of a new therapy or increase in treatment dose. PKG use is likely to lead to a change in management for a sizeable proportion of patients (plausibly 31.8–79% of patients). However, this also means that many patients will have no change in management after PKG use, although its use may still be helpful for confirming that no change is required. As these studies were not formal comparative studies there is a possibility that some medication changes made because of PKG results were unnecessary, or helpful medication changes may have been missed. The population of these studies was newly diagnosed PwP, those attending routine follow-up, clinically stable patients and those considered to benefit from continuous objective measurement, where stated. There was no comparison of different patient populations; therefore, no indication of which patients are more likely to have management changes as a result of PKG use.

The majority of the evidence on the clinical value of PKG came from a single trial of 162 patients (Woodrow et al. ). 17 This trial compared centres using PKG to those using standard assessment methods. It was not randomised but was otherwise at low risk of bias. The Woodrow trial demonstrated that patients managed with PKG appear to benefit more than those on clinical management alone, with improvements in UPDRS III and IV scores (by around 3.1 and 1.2 points, respectively), and plausibly reductions in bradykinesia and dyskinesia. This benefit seems to depend on whether patients were ‘in target’ (i.e. condition was under control with current treatment) before PKG use. Patients not ‘in target’ saw improved UPDRS scores, but those ‘in target’ did not. Patients ‘in target’ may, however, benefit from reduced levodopa dose and consequent reduction in dyskinesia, but this was inconclusive, as only 28 patients were ‘in target’. It should be noted that the Woodrow trial included multiple uses of the PKG device over a short period of time (once every 5 weeks); hence the clinical benefits that might occur if PKG is used less frequently might be different.

Other clinical evidence was more limited and drawn from non-comparative studies, although these studies generally had low risk of bias. This additional evidence supports PKG giving improvements in UPDRS scores; study populations varied, with some studies including clinically stable patients with well controlled symptoms, while others included uncontrolled patients. It also suggested improvements in outcomes in the majority of patients as assessed by patients and clinicians using PGI-I and CGI-I scales. As these studies were not comparative it is unclear how much of the benefit can be ascribed to PKG use specifically.

There was limited evidence on patient, carer or clinician opinions of PKG. Patients had generally favourable opinions as to the value of PKG, particularly valuing its ability to remind them to take medication. Patients often found that PKG helped with discussing or reporting symptoms with their clinician. Clinicians were more equivocal about the value of PKG, with many thinking that PKG provided no additional clinical information over their own judgement, although the majority agreed that PKG improved dialogue with the patient.

There was some evidence, from one study, that PKG can be used for remote management of PD. 69 In that study, clinicians judged that most remote assessments were successful, and patients were generally satisfied with a remote assessment of their condition.

Overall, the EAG concludes that there is a reasonable body of evidence to support the use of PKG in practice. PKG appears to be able to reliably measure bradykinesia and dyskinesia, and provides useful information that leads to changes in clinical management for at least some patients. There is some evidence that using PKG leads to genuine clinical improvements for patients when compared to standard management, in terms of reductions in UPDRS scores, but this is based on a single trial. However, multiple clinic visits and PKG assessments may be required before patients reach a controlled state.

Diagnostic accuracy

A total of eight papers reporting on diagnostic accuracy for STAT-ON were identified. 18,21,22,24,26,28,30,31 None of these explicitly identified the device used as STAT-ON; however, all were listed on the device website, so it is assumed that the device is the same as STAT-ON. Many of the papers overlap in authorship, and most report data from the REMPARK study, or related studies, supported by the manufacturer. All studies are reported here for completeness, but it should be noted that they are not independent and may have overlapping populations. One further conference abstract was found, which is not discussed here due to limited reporting of data (this study reported that no device-related adverse events were reported). 19

The quality assessment of the papers is given in Table 13.

As the STAT-ON diagnostic accuracy papers were mostly reporting the REMPARK study, and/or had overlapping authorship, the QUADAS-2 assessment was very similar across publications. Risk of bias for the index test was considered unclear as papers did not generally describe the index test directly, and in some it appeared to be derived as part of the data analysis itself. Hence, it was also unclear whether these index tests are part of the current device in commercial use. Although effort was made to have robust reference standards, such as by video-recording patients for clinical assessment, or adjusting patient diaries for possible errors, it remains unclear whether clinical or patient assessment represents a robust reference standard for the outcomes considered. Flow and timing risk was often rated as high or unclear due to lack of reporting as to exactly when index tests and reference standard assessments were performed, and whether all included patients were analysed.

Table 14 presents a summary of the diagnostic accuracy data from the STAT-ON publications. No studies were from the UK; studies were mostly conducted in Spain, or in various European countries as part of the REMPARK study. Sample sizes varied from 12 to 102.

Reporting of diagnostic accuracy data was limited, with only two studies reporting numbers of true positives, etc.,21,22 and most studies not reporting CIs. Four studies evaluated whether STAT-ON can diagnose ‘on–off’ times. 18,21,26,28 These all found that STAT-ON had high diagnostic sensitivity (between 90.3% and 97%) and high specificity (88–94%), and also high positive predictive value (PPV) and negative predictive value (NPV) (92% and 94%, respectively28). Two studies evaluated freezing of gait and found that STAT-ON had high sensitivity (88.1% and 91.7%), but lower specificity (80.1% and 87.4%). 24,31

One study found that STAT-ON had high accuracy for predicting bradykinesia,30 and one study found that STAT-ON had high accuracy for predicting strong trunk dyskinesia, but poor diagnostic accuracy for predicting dyskinesia in general. 22

Overall, these results suggest that STAT-ON is good at determining ‘on–off’ times, and symptoms in the lower limbs and trunk, such as freezing of gait and trunk dyskinesia.

Association outcomes

Three small cohort studies (n = 11–75) reported only association outcomes. 23,27,29 One conference abstract that reported patient satisfaction/usability also reported limited association outcome data, but is not considered further here (see Appendix 4, Table 55 for data). 20 A summary of the association data reported in the included studies is shown in Table 15.

We note that the study by Perrote did not strictly report correlation between STAT-ON and patient diaries; however, the study meets inclusion criteria for the review and fits better within ‘association outcomes’ than any of the other outcomes of interest. 23

In one study, there was a significant correlation between STAT-ON and clinical assessment using the Unified Dyskinesia Rating Scale (UDysRS) (0.70, 95% CI 0.33 to 0.88), which was higher for trunk and legs scale subitems (0.91, 95% CI 0.76 to 0.97), since the sensor is located on the waist. 29 In the largest study (n = 75) there was moderate correlation between UPDRS III and STAT-ON outputs (rho −0.56); correlation between STAT-ON outputs and the gait item in the UPDRS III was good (rho −0.73). 27 The other study reporting association data on STAT-ON simply reported that mean hours monitored was greater by STAT-ON than that recorded by the movement diary (60 ± 9.89 vs. 40 ± 16.4, p < 0.001) and that reporting of freezing of gait episodes and hours in the ON state were higher with STAT-ON compared to the patient movement diary. 23

Clinical outcomes and intermediate impact of monitoring

No studies reporting either intermediate impact or clinical outcomes were identified for STAT-ON.

Patient-, carer- and clinician-reported opinions

One paper18 and one conference abstract25 reported patient or carer opinions on STAT-ON and one paper32 and two conference abstracts20,25 reported clinicians’ opinions.

Due to the limited reporting of these studies, quality assessment was not performed. Results of the studies are summarised in Table 16, and are given in full in Appendix 4, Table 64.

Based on the limited evidence available, patients and caregivers generally found STAT-ON easy to use and were satisfied with the device (over 75% in two studies). 18,25 The majority of neurologists considered the device/information ‘quite’ to ‘very useful’ and considered it a useful tool for identifying patients with advanced PD symptoms. 32

One small cohort study (n = 39) reported that patient satisfaction with STAT-ON was high (assessed using Quebec User Evaluation of Satisfaction with Assistive Technology) and the system was found to be easy to use (assessed using System Usability Scale). 20

Summary of STAT-ON evidence

While a substantial body of evidence on STAT-ON has been published, this evidence is largely restricted to diagnostic accuracy studies. The diagnostic accuracy studies were of uncertain risk of bias, largely because of lack of clarity in the exact index tests being used, and difficulties in establishing a robust reference standard. In general, the studies suggest that STAT-ON can accurately determine ‘on–off’ times and bradykinesia. STAT-ON may have particular benefits in identifying trunk and lower limb symptoms. It appears to have reasonably good accuracy for diagnosing freezing of gait, and possibly trunk dyskinesia, but not dyskinesia elsewhere. However, most of these results are based on very small numbers of patients. A larger study reporting association outcomes suggests that there is a moderate correlation between STAT-ON outputs and the UPDRS III; there was good correlation between STAT-ON outputs and the gait item in the UPDRS III. 27

A key limitation in the STAT-ON evidence base is that there is currently no evidence on the intermediate impact of STAT-ON (i.e. whether its use changes clinical management of patients) or on the clinical impact of STAT-ON (such as whether it leads to improved UPDRS scores). It is therefore unclear whether STAT-ON use will lead to treatment modification, and whether patients will see a benefit in terms of reduction in bradykinesia and dyskinesia, and improvements in UPDRS scores and quality of life.

There was limited evidence on patient, carer or clinical opinions of STAT-ON; most was generally favourable.

Overall, the EAG considers that the diagnostic accuracy evidence suggests that STAT-ON is a promising technology, that is likely to be able to identify ‘on–off’ periods, bradykinesia and freezing of gait. As it is worn on the waist it may be particularly useful for detecting lower limb and trunk symptoms. However, the lack of clinical evidence for STAT-ON means that it is not currently possible to determine if its diagnostic value will translate into a real clinical value for patients. The EAG considers that, because STAT-ON is a waist-worn device and because diagnostic accuracy data relates mostly to ‘on–off’ times and freezing of gait, it cannot be safely assumed that the possible clinical benefits observed when using PKG will also apply when using STAT-ON.

Diagnostic accuracy

One study reported diagnostic accuracy for a technology judged to be equivalent to Kinesia 360. 35 It should be noted that the device used was not explicitly named as Kinesia 360, and so may not be exactly the same as the true Kinesia 360 motor assessment system. The QUADAS-2 assessment is presented in Table 17, and the results are summarised in Table 18.

No diagnostic accuracy studies for KinesiaU were identified.

The risk of bias for the study was generally rated as low because the index test was clearly described, although it was not explicitly named as Kinesia 360. The index test and reference standard were assessed in a blinded fashion.

Kinesia 360 was found to have moderate to good diagnostic accuracy to detect bradykinesia, dyskinesia and tremor, with AUCs ranging from 82% for bradykinesia to 89% for tremor.

Association outcomes

There were no studies reporting association outcomes for Kinesia 360 or KinesiaU.

Clinical outcomes and intermediate impact of monitoring

Two studies of Kinesia 36033,34 and one study of KinesiaU36 were eligible for inclusion. The studies were assessed for quality using tools appropriate to their study design; the two RCTs were assessed using the Cochrane risk of bias tool and the cohort study was assessed using a tool developed for the review using relevant criteria. The quality assessment results are summarised in Tables 19 and 20.

Each of the included studies had a high risk of bias or some concerns regarding bias, limiting the reliability of their results.

A summary of the results of studies reporting clinical outcomes related to Kinesia 360 and KinesiaU is reported in Table 21.

A small pilot RCT (n = 39) compared Kinesia 360 to supplemented standard care versus standard care alone to titrate the optimal rotigotine dosage in PwP with insufficiently controlled motor symptoms, prescribed transdermal rotigotine. 33 Mean rotigotine dose, mean rotigotine dosage increase and mean number of dosage changes were higher in the Kinesia 360 arm than the standard care arm. At week 12 there was a statistically significant improvement in UPDRS II in the Kinesia 360 arm compared with a slight worsening in the standard care arm (−2.1 vs. 0.5, p = 0.004). The difference in improvement in UPDRS III was not statistically significant between groups (−5.3 vs. −1.0; p = 0.134). There was no significant change in PDQ-39 or PAM-13 score at week 12 in either study group.

A small RCT that was suspended due to COVID-19 (n = 25) compared telehealth follow-up care using data from Kinesia 360 with usual in-person follow-up care in PwP with bothersome tremor or dyskinesia identified as a treatment target at their most recent visit. 34 The average change in PDQ-39 summary index score from baseline to completion (primary outcome) was −4.7 points in the telehealth group (95% CI −10.2 to +0.7) and +0.9 (95% CI −3.6 to +5.5) in the usual care group (mean baseline PDQ-39 score was 29 in the telemedicine group and 25 in the usual care group). Secondary outcomes were not significantly different between groups. Repeat measurement of MDS-UPDRS Part III was not completed due to suspension of face-to-face clinical and research visits, due to COVID-19.

A small cohort study (n = 16) assessed KinesiaU in PwP undergoing therapy changes. 36 Fourteen patients successfully used the KinesiaU system, while two did not complete the recordings due to user difficulty or technical issues. Thirteen of the patients who successfully used the KinesiaU system returned for a follow-up visit; the clinician reviewed the KinesiaU report with each patient and made a therapy recommendation based on the report and clinical judgement. Eight patients demonstrated improvements with their new therapy and were instructed to continue with it while five were instructed to return to their previous therapy.

Patient-, carer- and clinician-reported outcomes

One paper reported patient opinions of Kinesia 360, and one paper reported patient opinions of KinesiaU. The quality assessment results of these studies are presented in Tables 19 and 20. Table 22 summarises the patient opinion data from the two studies.

In the RCT of telehealth follow-up care with data from in-home Kinesia 360,34 54% of telehealth patients reported feeling comfortable or very comfortable using motion sensors and 8% were uncomfortable or very uncomfortable using motion sensors. Forty-six per cent of telehealth patients would have preferred to be in the usual care group, 8% would not and 46% were undecided. Eight per cent of usual care patients would have preferred to be in the telehealth group, 67% would not and 25% were undecided.

In the cohort study of KinesiaU,36 88% of patients agreed or strongly agreed that the KinesiaU system was easy to understand and use, while 12% disagreed or strongly disagreed. Only 32% of patients agreed or strongly agreed that they looked at the KinesiaU reports often and only 38% agreed or strongly agreed that they were useful to look at. Forty-four per cent of patients agreed or strongly agreed that they would continue to use the system if it was available to them, 25% were neutral and 31% disagreed.

Summary of Kinesia 360 and KinesiaU evidence

The evidence on Kinesia 360 was mostly from two small RCTs (64 patients in total). These suggested favourable results when using Kinesia 360, with reductions in UPDRS scores, and improvements in quality of life comparable to those seen for PKG. However, neither study used the device during routine clinical visits for PwP. In one study patients with insufficiently controlled motor symptoms were being assessed to titrate the optimal rotigotine dosage,33 and in the other study Kinesia 360 was used in remote telehealth assessments. 34

One diagnostic accuracy study found that Kinesia 360 had moderate to good accuracy for diagnosing dyskinesia, bradykinesia and tremor. 35

Given the limited evidence base, the EAG considers that Kinesia 360 is a promising technology, but there is too little evidence at present to be confident about its clinical value.

Evidence on KinesiaU was limited to one small study (16 patients). 36 The EAG considers this to be too little evidence to draw any conclusions on the clinical value of KinesiaU. Patient opinion of the KinesiaU system was not particularly favourable.

Diagnostic accuracy and association outcomes

One small cohort study (n = 30), reported only as an abstract, reported that PDMonitor accurately detected and estimated the severity of arm bradykinesia, dyskinesia, gait impairment, wrist tremor, leg tremor and freezing of gait compared with a clinical assessment by a PD expert physician using UPDRS III and AIMS. 37

Clinical outcomes and intermediate impact of monitoring

One case series described two cases where PDMonitor revealed information leading to medication changes and improvement in symptoms. 95 As this study only included two patients, the EAG do not think it represents useful evidence for PDMonitor and do not consider it further.

Patient-, carer- and clinician-reported outcomes

No studies reporting patient, carer or clinician opinion for PDMonitor were identified.

Summary of PDMonitor evidence

As there are currently no fully published studies of PDMonitor, the EAG considers that no conclusion as to the clinical value of PDMonitor can be drawn. It cannot be safely assumed that any possible benefits of PKG (or other technologies) will apply if PDMonitor is used.

Summary of key results

A full summary of the EAG’s conclusions for each technology is reported in the sections of the report for each technology (see Personal KinetiGraph, STAT-ON, Kinesia 360 and KinesiaU motor assessment systems and PDMonitor). Here we present a broad summary of all evidence.

Overall, the EAG considers that only PKG has a body of research evidence sufficient to properly assess its clinical potential. PKG appears to accurately measure dyskinesia and bradykinesia, with very high sensitivity and reasonably high specificity. Diagnostic accuracy was also reasonably high for measuring tremor and treatment-related outcomes. However, its accuracy for measuring sleep disturbance was lower.

PKG is being used by clinicians to guide treatment decisions, primarily the addition of a new therapy or increase in treatment dose; PKG use led to a change in management for 32–79% of patients. The majority of the evidence on the clinical value of PKG came from a single non-randomised trial of 162 patients by Woodrow et al. 17 The trial demonstrated that patients managed with PKG appear to benefit more than those on clinical management alone, with improvements in UPDRS III and IV scores. This benefit seems to depend on whether patients were ‘in target’ (i.e. condition was under control with current treatment) before PKG use. Patients not ‘in target’ saw improved UPDRS scores, but those ‘in target’ did not (but only 28 patients were ‘in target’). Also, PKG was used frequently (every 5 weeks) in the trial, and benefits may be different if PKG is used less frequently.

Therefore, there is some evidence that using PKG leads to genuine clinical improvements for some patients. However, multiple clinic visits and PKG assessments may be required before patients reach a controlled state.

For STAT-ON, evidence is almost entirely limited to diagnostic accuracy studies. These suggest that STAT-ON can accurately diagnose ‘on–off’ times and bradykinesia. STAT-ON also seems to have reasonably good accuracy for diagnosing freezing of gait and possibly trunk dyskinesia (as a waist-worn device), but not dyskinesia elsewhere.

There is currently no evidence on the intermediate impact of STAT-ON or on the clinical impact of STAT-ON; therefore, it is unclear whether STAT-ON use will lead to treatment modification and subsequent improvements in symptoms and quality of life. Overall, the EAG concludes that while STAT-ON is a promising technology, the lack of clinical evidence means that it is not currently possible to determine if its diagnostic value will translate into real clinical value for patients.

Two small RCTs suggest favourable clinical outcomes with Kinesia 360 use in populations receiving rotigotine or when Kinesia 360 was used for remote telehealth assessments. In view of the extremely limited evidence base, the EAG considers that Kinesia 360 is a promising technology in certain situations, but that there is currently too little evidence to be confident about its clinical value.

Evidence on KinesiaU and PDMonitor are too limited to draw any conclusions on their clinical value.

Almost all the available evidence for all devices focused on motor symptoms (bradykinesia, dyskinesia and tremor). Evidence on sleep disturbance and other non-motor aspect PD was extremely limited.

The EAG considers that, because of the very different natures of the technologies assessed, such as the specific symptoms they measure, the position of the sensors on the body and the characteristics of PwP the devices have been assessed in, it should not be assumed that any clinical benefits observed for PKG would also be found with the other technologies.

There were no studies that directly compared one remote continuous monitoring device against another. In addition, there was limited evidence on the use of remote monitoring devices in different patient subgroups. Therefore, it is unclear which patients are more likely to have management changes and subsequent improvements in clinical outcomes as a result of their use. The only evidence relating to adverse events was that there were no device-related adverse events reported (PKG and STAT-ON).

Generalisability of results

The EAG considers that the results observed in the included studies are likely to be broadly generalisable to PwP in the UK. Although few studies were conducted in the UK itself, most were conducted in Europe, North America or Australia, and so their results are likely to be broadly applicable to the UK. Most studies recruited PwP without any further limitations on inclusion criteria, and without focus on specific subgroups. This would suggest that the results will be generally applicable to other PwP.

A key generalisability concern is that almost all studies were conducted in patients receiving pharmacological therapy, primarily levodopa. The clinical evidence for PKG is largely focused on how PKG use can modify levodopa therapy, and the clinical impacts of those therapy changes. Consequently, there is little to no evidence on the possible benefits of the technologies in other types of patients, such as those receiving non-pharmacological therapy, or on more advanced therapies such as DBS. The EAG does not think it safe to assume that any benefits of PKG, or other technologies, will necessarily apply to these other patient groups.

There was no evidence relating to most patient subgroups of interest. In particular, there was no evidence specifically related to people with communication barriers or difficulties, and no evidence for specific ethnicities, or by socioeconomic status. It is unclear how PKG, or other technologies, might work in those key populations.

As noted elsewhere, because of the very different natures of the technologies assessed, the EAG does not consider that any clinical benefits observed for PKG would also be found with the other technologies.

Strengths and limitations of analysis

This is the first complete systematic review of all available diagnostic and clinical evidence for PKG, Kinesia 360, KinesiaU and PDMonitor (a review of STAT-ON was published while this project was underway). This review used extensive database searches to identify all published evidence on the included technologies and followed rigorous recommended review methods to identify relevant publications, assess their risk of bias and undertake a narrative synthesis of the results. As such, this is the first fully rigorous review of these technologies, and also the first to compare the technologies in one review.

The review was strengthened by the provision of individual-level data for the key clinical trial of PKG, which permitted a more thorough examination of the clinical impact of PKG than would otherwise have been possible.

A key limitation of the analyses is due to the lack of replication across studies. In general, most outcomes were reported in only one or two studies, or outcomes were reported in inconsistent ways across studies. This meant that no meta-analysis was possible for any included studies, and the narrative synthesis was severely limited by the consequent difficulties in comparing different studies. This lack of replicability raises some concerns as to how robust the findings of the review are. It should be noted that many of the review conclusions are based on individual studies.

A further limitation is the low quality of much of the evidence, particularly for diagnostic accuracy. This casts some doubt on the validity of the diagnostic accuracy evidence. It should be noted that few studies were formal diagnostic accuracy studies, and there are innate difficulties in this field in robustly assessing diagnostic accuracy, given the lack of clear reference standards, and lack of clarity over the exact algorithms used to convert device output into diagnostic assessments.

Searches

The bibliographic search detailed in the Methods for reviewing clinical effectiveness was used to identify studies reporting on the cost-effectiveness of PKG, Kinesia 360, KinesiaU, PDMonitor and STAT-ON devices.

Selection process

The review considered a broad range of economic studies including trial-based economic evaluations, modelling studies and analyses of administrative databases. The inclusion criteria considered were full economic evaluations comparing two or more alternative interventions in terms of both costs and consequences (i.e. cost-minimisation, cost-effectiveness, cost–utility and cost–benefit analyses).

The protocol for the selection of relevant studies defined two selection stages: (1) assessment and screening for possible inclusion of titles and abstracts identified by the search strategy, and (2) acquisition and screening for inclusion of the full texts of potentially relevant studies. Two researchers (RW and NM) independently screened the titles and abstracts of all reports identified by the bibliographic searches. Full-text papers were subsequently obtained for assessment and screened by at least two researchers, with any disagreement resolved by consensus.

Lynch et al.

Lynch et al. was a cost-consequence analysis considering the cost savings and QALY gains in people whose oral therapy was adjusted following assessment with the PKG and did not consider any comparator. 88 The analysis was based on a before-and-after study, enrolling 103 PwP in Australia, all of which were evaluated using the PKG. The economic evaluation focused specifically on a subgroup of 33 of these patients who had their oral therapy adjusted. In this subgroup, patients were observed to have experienced reductions in UPDRS II and UPDRS III scores and improvements in the PDQ-39 quality-of-life instrument.

No formal modelling was implemented as part of the economic analysis. Instead, the UPDRS domain scores were used to directly estimate reductions in costs and QALYs using evidence from published sources considering a 1-year time horizon. Cost savings were estimated using an algorithm reported in McCrone et al. , which suggested that a one-point reduction in UPDR II is associated with a $430 cost saving. 97 Based on reported changes in UPDRS II, this resulted in cost savings of $1719.42 per patient (exclusive of the cost of the PKG monitoring device). Benefits were estimated using regression models reported in the NICE PD guideline, which linked EQ-5D with several clinical measures. 98 Using reported improvements in UPDRS II, UPDRS III, per cent over target and PDQ-39, QALY gains of between 0.10 and 0.12 were estimated.

Rao et al.

Rao et al. sought to assess the impact of PKG on pharmacological treatment and decisions to initiate advanced treatment. 89 The study reports on 37 PwP attending a movement disorder clinic and who were clinically assessed as requiring advanced treatment. Following PKG, five of these were considered to need advanced treatment with the remaining five patients managed via dose adjustments to pharmacological care.

On the basis of this finding, the study estimated potential cost savings associated with postponing advanced treatment. This assumed transition to apomorphine either administered by pump or by pen. Cost savings associated with APO-go pump were estimated as £5400 per pump per year leading to a saving of £27,000 per year for five patients. For APO-go Pen treatment, equivalent cost savings were estimated as £3200 per year leading to a saving of £16,000 per year. Sources used to inform the costs were not stated.

Chaudhuri et al.

Chaudhuri et al. explored the cost-effectiveness of the PKG and clinical assessment in the management of PD compared to standard of care (SoC) in the context of the UK NHS. 96 A cost–utility model was developed using a Markov model structure. The model comprised three health states: (1) uncontrolled, (2) controlled and (3) death, with the ability to transition bidirectionally between the uncontrolled and controlled health states. The uncontrolled and controlled health states were based on the MDS-UPDRS domain scales of II (motor experiences of daily living) and III (motor examination). All individuals entering the model start in an uncontrolled state. The transitions between the uncontrolled and controlled health states are dependent on the improvement in MDS-UPDRS II and III with either the PKG, in addition to clinical assessment, or with clinical assessment alone (considered to represent SoC). The transition to the death state was based on UK all-cause mortality rates, with an elevated mortality risk of 2.22 associated with PD-specific relative mortality risk (based on Xu et al. , 2014). 99 The Markov model had a cycle length of 1 year and a lifetime horizon of 22 years in the base-case analysis. A discount rate of 3.5% was used for both costs and effects, in line with NICE recommendations. 100

The clinical efficacy of the PKG and clinical assessment compared with SoC was based on the study by Woodrow et al. 17 Full details of this study are described in the section Clinical outcomes of this report. In summary, the study is based on a population in Australia and evaluated comparative outcomes in a PKG+ arm (use of the PKG and clinical assessment) compared to a PKG− arm (clinical assessment without the use of the PKG). At the first screening visit, individuals were assessed to decide whether their PD motor features were ‘in target’ (no further treatment required) or ‘out of target’. In the latter case, a plan for changing treatment was provided until the next consultation 5 weeks later. The same assessment protocol was followed until the PD motor features were ‘in target’. A maximum of five visits were permitted, inclusive of the first screening visit. The primary study outcome was the difference in MDS-UPDRS total score from baseline to the end of the study. Woodrow et al. provide the outcomes of Total MDS-UPDRS score, as well as the subscores for domains I–IV, at the first and last study visit in the PKG+ and PKG− arms. 17 In the PKG+ arm, MDS-UPDRS total score and MDS-UPDRS domain scale III significantly improved by 8.5 points and 6.4 points, respectively, between the first and last study visit, while in the PKG− arm the change in MDS-UPDRS total score and MDS-UPDRS III failed to reach statistical significance.

The patient population used in the model by Chaudhuri et al. is based on the Woodrow et al. study. 17,96 The baseline patient characteristics for age, gender, levodopa equivalent daily dose (LEDD), MDS-UPDRS domain scales II and III and H&Y stage are the same as those reported in Table 1 of Woodrow et al. 17 For each intervention strategy in the model (i.e. PKG and clinical assessment, represented by PKG+ in Woodrow et al. , or SoC, represented by PKG− in Woodrow et al. ), individuals in the uncontrolled health state were assumed to have a MDS-UPDRS score in line with the MDS-UPDRS score obtained at the first screening visit in the corresponding arms of Woodrow et al. , while individuals in the controlled health state were assumed to have a MDS-UPDRS score in line with the final study visit in Woodrow et al. The probabilities of transitions between the uncontrolled and controlled health states in the first 6 months of the model were derived from the proportion of patients who were identified as controlled after the initial use of the PKG in Woodrow et al. After 6 months, transition probabilities were estimated using a bootstrap approach, but no details on the methods used are provided in Chaudhuri et al. The model did not include adverse events.

The treatment effectiveness was assumed to be maintained for a period of 5 years. This was based on a systematic literature review on the impact of LCIG in patients with advanced PD, where the results of the review suggest that LCIG extends the benefits of levodopa (in terms of reduced ‘off-time’) for at least 2–5 years. 101 From 6 years onwards, a long-term gradual waning of effectiveness was assumed in line with the natural disease progression of PD. The authors acknowledge that long-term progression rates are heterogeneous as they vary from person to person. Therefore, the authors used two alternative estimates for rates of progression: (1) a rate of progression based on a bootstrap analysis of published UPDRS III progression data based on a study by Holden et al. ,102 which equates to an average annual rate of progression of 10.9%, and (2) a published annual rate of progression of 2–7% based on a prospective study by Schrag et al. 103

The base-case analysis was modelled to mitigate the benefit observed in the Woodrow et al. study to 75%. This is because participating clinicians assigned to the PKG+ arm in Woodrow et al. received an additional 1-day training in interpreting the PKG for the assessment of PD compared to participating clinicians who were not assigned a PKG. The base case also assumed that PKG+ patients who are controlled will use two PKGs per year, while of the uncontrolled patients, 50% will use three PKGs per year and the other 50% will use four PKGs per year.

To derive HRQoL, the MDS-UPDRS domain scores for II and III obtained from the Woodrow et al. study were used in a published mapping algorithm to predict EQ-5D index values. The mapping algorithm was derived from a study population of 121 patients with idiopathic PD from study centres in Hessia, Germany. 104 The EQ-5D values were derived based on weights from a European population, valued by a visual analogue technique (European index) for the base-case analysis. Alternative values for a scenario analysis were based on weights derived from a German population, valued with the time trade-off approach (German index). The authors also conducted a scenario analysis using utility values derived from a model developed for the NICE PD guidelines, which estimated that HRQoL increases by 0.04 for every point reduction in UPDRS II and 0.02 for every point reduction in UPDRS III. 98 The resulting health state utility values from the alternative sources and approaches differed significantly.

PKG costs were based on the manufacturer’s list price of £225, equating to a total cost of £450 per year for controlled patients (two PKGs per year), and £787.50 per year for uncontrolled patients (50% with three PKGs per year and 50% with four PKGs per year). An outpatient and telephone visit with a movement disorder specialist was costed at £384 and £60 per year, respectively. Health state costs associated with PD progression were based on direct medical and non-medical costs associated with the H&Y scale from the NICE PD guidelines. 98 The authors state that the MDS-UPDRS scores in the model were applied to derive average annual costs by H&Y stage (see Table 23), but no details are provided. 96

The cost-effectiveness results from Chaudhuri et al. showed that the intervention of PKG and clinical assessment is associated with lower total costs compared to SoC (£141,950 vs. £159,312) and improved total QALYs (7.88 vs. 7.61) over a lifetime horizon, which resulted in an incremental difference of £17,362 and 0.267 QALYs per patient and an ICER of −£64,979 per QALY. Sensitivity analysis by the authors indicated that the ICER was most sensitive to the annual cost of H&Y stage 4.

Critique of Chaudhuri et al.

Chaudhuri et al. 96 represents the first cost-effectiveness analysis of the PKG remote continuous monitoring system in PD. The study appears to be well conducted and has been performed from the perspective of the UK NHS, which makes it directly relevant to the decision problem addressed in this report. The study accounts for direct medical costs only and QALYs are accumulated over the lifetime of an individual with PD. The population is with moderate disease based on the inclusion criteria of Woodrow et al. (idiopathic PD of ≥ 4 years or taking ≥ 4 doses of levodopa per day because of a much greater likelihood of being out-of-target at baseline; aged 59–75 years because they are less likely to be candidates for DBS and less likely to have a high incidence of contraindications to increasing dopaminergic therapy; not receiving treatment with, or under consideration for, device-assisted therapy), while the intervention’s effectiveness is based on outcomes from the only comparative study for the PKG, where the PKG, in addition to clinical assessment, is compared to clinical assessment alone in Woodrow et al. In the absence of alternative sources of data for clinical efficacy, the data source, population and intervention strategies used by the authors are considered appropriate; however, it is not clear that individuals suitable for PKG in the UK NHS would use as many PKGs as assumed in Chaudhuri et al. (i.e. PwP classified as controlled and in-target are assumed to use two PKGs and have two movement disorder specialist visits per annum, while PwP classified as uncontrolled and out-of-target are assumed to use 3–4 PKGs with 3–4 visits per annum).

The clinical effectiveness of the PKG is based on the MDS-UPDRS domain scores II and III, while the primary end point of the Woodrow et al. study was total MDS-UPDRS. 17 Chaudhuri et al. justified the choice of domains II and III based on the availability of published mapping algorithms that use domains II and III to predict EQ-5D utility values for the model and on the basis that these domains constitute the largest components of the total MDS-UPDRS score. The EAG notes that this means that the results of Chaudhuri et al. do not reflect the impact of the PKG on non-motor experiences of daily living (MDS-UPDRS I) or on the most severe motor complications (MDS-UPDRS IV) associated with PD.

The model considered a lifetime horizon of 22 years in the base-case analysis. The study justified the choice of 22 years to approximate lifetime treatment and to capture the long-term costs and health effects of treatment. While the EAG acknowledges that a lifetime horizon is an appropriate time horizon to capture the effect of treatment on the long-term progression of PD, the EAG also has a major concern about the use of a lifetime horizon to capture the difference in costs and health effects of PKG compared to SoC when there is an absence of evidence to suggest that the benefits of PKG equates to changes in treatment that are sustained over the long-term, into a future of 22 years, particularly in recognition of the fact that there are no disease-modifying treatments available for PwP. The model by Chaudhuri et al. makes a number of strong assumptions about sustaining the 6–12 months of benefits observed in Woodrow et al. over the long-term:

• In the model by Chaudhuri et al. the change in the MDS-UPDRS II and III scores between the first (uncontrolled) and last study visit (controlled) for the PKG+ arm compared with the PKG− arm in Woodrow et al. is used to capture a 6–12 months treatment effect. Although the authors mitigate this benefit to 75% to account for additional support provided to clinicians in the PKG+ arm and the effect of participating in a clinical trial, the benefits are assumed to be sustained in full (i.e. without any waning effect) for a period of 5 years. This assumption is not supported by any treatment data in patients with moderate PD and only justified by the authors as a proxy on the basis of data for the use of LCIG in patients with advanced PD.

• It is only from year 6 onwards in the model that a long-term waning of treatment effect is included. The waning effect is assumed to gradually decline in line with the natural history of disease progression for PD; however, the authors acknowledge that the long-term progression rate of PD is heterogeneous and highly uncertain. An average progression rate of 10.9% per annum was used by the authors based on the progression of MDS-UPDRS III scores over 5 years from the Parkinson’s Progression Markers Initiative, which is an international, multicentre study in patients with de novo PD (early, initially untreated PD).

• The transition probabilities representing the bidirectional movement between the uncontrolled and controlled health states in the model over time were based on the proportion of patients who were identified as controlled at the first visit in the Woodrow et al. study. The exact proportion used in the model at 6 months does not appear to be reported in Woodrow et al. but, more importantly, this proportion at 6 months, with an estimate of uncertainty, is used to establish the transition probabilities between the health states over a lifetime horizon.

The authors state that their modelling approach is conservative because they have mitigated the benefit from Woodrow et al. to 75% and have included the benefit of using the PKG for 1 year, whereas the cost of provision of the PKG for the following years is included in the model. However, the EAG notes that the benefits of using the PKG in the first year are sustained in full for a period of 5 years; it is simply that no additional ‘top-up’ benefit from using PKGs in the following years is included in the model. Furthermore, the changes in MDS-UPDRS scores from first to last visit in Woodrow et al. are based on up to five consultation visits, where a PKG was worn prior to each visit in the PKG+ arm. At each visit (5 weeks apart), a plan for changing treatment was provided until the motor features of the disease were considered to be in-target and no further treatment required. The EAG considers it unlikely that UK clinical practice would follow a similar protocol.

The movement between the uncontrolled and controlled health states over time, based on the MDS-UPDRS II and III scores from the Woodrow et al. study, affects the costs and utility values at each model cycle over the lifetime horizon of the model. The MDS-UPDRS II and III scores from Woodrow et al. are used in a mapping algorithm to predict EQ-5D index values for the uncontrolled and controlled health states in the model. The mapping algorithm by Dams et al. is based on a small study population of 121 PwP in Germany, where the health states identified by the EQ-5D were converted into EQ-5D indices employing weights from a pooled European population valued by a visual analogue technique. 104 The EAG notes that this method does not align with the NICE Reference case, where the valuation of HRQoL measured by patients (or their carers) should be based on a valuation of public preferences from a representative sample of the UK population using a choice-based method. 100

The mapping algorithm in Dams et al. is also based on UPDRS scores, rather than MDS-UPDRS scores. Chaudhuri et al. do not provide details on whether any conversion was made between UPDRS and MDS-UPDRS scales to derive the EQ-5D values. Furthermore, the resulting estimates of health state utility values differed substantially from other alternative sources and approaches considered by the authors. Therefore, even despite the concerns about the long-term extrapolation of benefits in the model, the health state utility values themselves are uncertain. This means that the accumulated total QALYs over a lifetime horizon are subject to considerable uncertainty.

The health state-related costs used in the model were based on MDS-UPDRS scores from Woodrow et al. and costs associated with the H&Y scale. The authors refer to the costs reported in the NICE PD guidelines Appendix F for the H&Y scale; however, despite a thorough review of the NICE PD guidelines,98 the EAG was unable to identify or validate the costs reported in Chaudhuri et al. No details are provided on how the annual costs by H&Y stage were derived in the model. Importantly, it is unclear how the MDS-UPDRS scores were converted onto the H&Y scale. The costs associated with the H&Y stages and the proportion of individuals in each stage by intervention strategy is a key driver of the total costs over a lifetime horizon. This is evident from Table 23, where the distribution of patients by intervention (PKG+) and SoC (PKG−) shows a greater proportion of patients in the less severe H&Y stages 1 and 2 and a smaller proportion in the higher stages of 3–5 for PKG+, whereas a smaller proportion are in stages 1 and 2 and a higher proportion in stages 3–5 for SoC. The annual cost associated with H&Y stages 1 and 2 is £3918 and £7417, respectively, whereas the annual cost associated with H&Y stages 3–5 varies from £14,150 to £53,335. Given that the annual health state costs are considerably larger than the annual cost of PKG and consultation visits (by an approximate order of magnitude of 10 for the lower H&Y stages and 100 for the higher stages), it is expected that the key driver of total costs in the model is the difference in the proportion of individuals in the H&Y stages by intervention over a lifetime horizon.

The base-case results suggest that PKG and clinical assessment compared to clinical assessment alone reduces the total costs to the NHS by £17,362 per patient. The cost savings for PKG+ are driven by a lower likelihood, on average, that a patient will end up in one of the more severe H&Y stages 3–5 (36.9%) compared to SoC (48%) over a lifetime horizon. The base-case results also suggest that PKG+ increases the total QALYs by 0.267 QALYs per patient compared to SoC. This is driven by a higher increment in utility for the controlled versus uncontrolled health states for PKG+ (0.036) compared to the increment in utility for the controlled versus uncontrolled health states for SoC (0.021), and a greater likelihood of being in the controlled health state versus uncontrolled health state for PKG+ compared to SoC over a lifetime horizon. Results of sensitivity and scenario analyses by the authors showed that the cost-effectiveness results were robust to the base-case conclusions; however, the EAG notes that the majority of these analyses did not address the key structural assumptions affecting the long-term efficacy, HRQoL and costs over a lifetime horizon in the model.

Searches

The bibliographic search detailed in Methods for reviewing clinical effectiveness was used to identify relevant studies.

Study selection

Cost-effectiveness studies using decision-analytic modelling published after the year 2000 were considered for inclusion. Only cost-effectiveness, cost–utility and cost–benefit analyses were considered eligible. The population of this review was defined as people with diagnosed PD. Studies in people with other neurological disorders were excluded. The inclusion criteria further specified that only titles in English would be considered eligible. Titles that were books, editorials, letters to the editor and reviews that did not include a de novo model were excluded from the review.

Two researchers (EC, THP) conducted the two-step selection process consisting of first screening studies for inclusion based on the titles and abstracts of studies identified by the bibliographic searches, and then reviewing the full-text articles identified at the previous step as potentially relevant.

Fundament et al.

Fundament et al. developed a Markov model to represent the progression of PD as rated using the UPDRS over time in patients with an early onset of motor complications. 137 The model was developed to evaluate the cost-effectiveness of DBS, compared to best medical treatment (BMT), for PD with early motor complications from a UK payer perspective. The study population was based on baseline characteristics for patients included in the Controlled Trial with DBS in Patients with Early Parkinson’s Disease (EARLYSTIM) trial, who all were assumed to have motor complications at model entry. The base-case analysis used a 15-year time horizon to capture the long-term progression to more advanced stages of PD.

Health states in the Markov model were based only on the treatment interventions (DBS and BMT) and death, but disease progression over time was modelled according to changes in UPDRS domain scores I–IV by treatment. Patients with DBS were assumed to continue therapy until withdrawal, after which they would continue with BMT until the end of the modelled time horizon or death. For the first 2 years of the model, clinical efficacy estimates in terms of changes in UPDRS scores for each intervention were derived from the EARLYSTIM trial, which collected UPDRS domain scores at treatment initiation and at 5, 12 and 24 months follow-up. These were used to calculate the percentage change in UPDRS from baseline in each domain for each intervention. Rates of disease progression beyond 2 years were modelled to be the same for both interventions (with the exception of UPDRS domain score IV) due to a lack of consistent long-term data on UPDRS outcomes by treatment. Annual rates of progression for each UPDRS domain were based on data pooled from various studies because the authors did not identify a single study that reported all domains of the UPDRS consistently. Supplemental appendix, table S1 in the paper provides the annual rates of progression in UPDRS scores by treatment group.

Mortality was incorporated into the model using a two-step approach. Firstly, age- and gender-specific baseline mortality was applied using UK all-cause mortality rates based on the baseline characteristics of patients in the EARLYSTIM trial. In the second step, a hazard ratio for mortality risk of 1.31 was applied to the baseline mortality for patients with more advanced disease. Using data from studies reporting on the relationship between UPDRS III and mortality, a per 10-point increase in UPDRS III score was associated with an increased risk of mortality, which was applied only in patients with a UPDRS III score > 15 to reflect the increased mortality associated with more advanced disease.

The model accounted for both treatment-specific and disease-related adverse events. The incidence of adverse events reported in the EARLYSTIM trial was used to inform the treatment-specific rates. For the disease-related adverse events, the authors highlighted that PD progression is associated with increasing postural instability, which may lead to falls and serious injury in some patients. To reflect the incidence of falls and associated hospitalisation rates, the authors pooled data from a series of studies to define a baseline proportion of patients falling per year of 42.8% based on patients with a UPDRS III score of 12 points. An odds ratio of 1.07 for each point increase in UPDRS III score was then applied to the baseline risk based on three studies reporting incidence of falls. 141 Of these falls, 50.9% were assumed to require hospitalisation.

Health-related quality of life was accounted for, in the short term, using a published mapping algorithm to map the PDQ-39 data from the EARLYSTIM trial to the EQ-5D, since EQ-5D was not collected in the trial. The corresponding utility weights by intervention were applied in the first 2 years of the model. For the longer term, after 2 years, the authors developed a new algorithm to link UPDRS scores to EQ-5D. The authors used an iterative approach to identify a statistical model that could accurately predict the EQ-5D index from the explanatory variables available from the EARLYSTIM trial (including UPDRS domain scores). The resulting EQ-5D function was given by Fundament et al. :

This algorithm was applied to changes in UPDRS scores over time in the model (beyond 2 years) to estimate the HRQoL by treatment group.

For resource use and costs used in the model, data were sourced from price lists, national drug prices, hospital payment tariffs and social care cost data. Due to a lack of standardised drug protocols for the management of PD, the authors analysed data from the UK Clinical Practice Research Datalink (CPRD) of 297 patients with PD for the period April 2003–March 2012 to derive information on drug formulations administered. Dosing information and the number of patients receiving each drug was combined with unit costs from the British National Formulary to calculate a drug cost per day for each treatment option of £4.16 for BMT and £2.28 for DBS. The costs of regular neurology outpatient appointments were accounted for by assuming four visits per year in the first year of treatment with DBS and two visits per year thereafter, while patients on BMT were assumed to have two visits per year. Home visits by a PD nurse were applied in both treatment groups with the same frequency as the neurology outpatient appointments. The unit costs associated with hospitalisation for falls and follow-up appointments were based on hospital payment tariffs and social care cost data. Supplemental appendix, table S1 in the paper provides a list of the unit costs used in the model.

The results of the cost-effectiveness analysis indicate that DBS leads to improved QALY outcomes and increased costs compared with BMT in patients with early onset of motor complications from a UK NHS perspective, leading to an ICER of £19,887 per QALY gained. Device costs made up the largest percentage of costs in the DBS treatment group, while in the BMT group, drug therapy and management of comorbidities were the main cost drivers. The model results were most sensitive to the time horizon used to model long-term costs and outcomes; when this was limited to 5 years, the ICER increased beyond £30,000 per QALY gained for DBS. The key uncertainty in the model was the lack of long-term follow-up data beyond 5 years for the interventions.

Chandler et al.

Chandler et al. developed a modelling framework to facilitate the estimation of long-term health and economic outcomes in PD. 140 The model was structured to simulate the long-term progression of PD from diagnosis through to a lifetime horizon, capturing both motor and non-motor symptoms and associated outcomes. The model was developed to support the cost-effectiveness evaluation of new disease-modifying drugs in PD when they become available. The authors recognise that there are currently no disease-modifying therapies available for the treatment of PwP, but new therapies are being studied and results from clinical trials are expected in the future.

The objective of the study by Chandler et al. was to develop and validate a novel model that addresses the unmet need to simulate the disease progression from diagnosis over a lifetime horizon. The authors also recognised the limitations of previous models in PD, where most have typically modelled transitions between health states defined based on the H&Y staging system and very few have tracked disease progression using UPDRS domain scores. The authors emphasised the need to reflect non-motor aspects of PD and therefore the need to develop models using the revised version of the original UPDRS (MDS-UPDRS). The model by Chandler et al. represents the first novel model in PD to track disease progression over a lifetime horizon using MDS-UPDRS domain scores (amongst others).

The model by Chandler et al. provides a conceptual modelling framework of long-term disease progression in PD, but it also simulates the progression of disease along the MDS-UPDRS and UPDRS subscales over time using predictive equations that capture the intercorrelation between MDS-UPDRS/UPDRS subscales and baseline population characteristics. The predictive equations for the simulation of disease progression for newly diagnosed PwP were developed from an analysis of longitudinal, observational data from the Parkinson’s Progression Markers Initiative, which is an international multicentre study that follows individuals from diagnosis (treatment naïve, age > 30 years) up to 6 years of follow-up. The data were analysed using a mixed-effect repeated measures (MMRM) model to predict change from previous values of MDS-UPDRS I (non-motor experiences of daily living), MDS-UPDRS II (motor experiences of daily living) and MDS-UPDRS III (motor examinations). In the MMRM model, the initiation of treatments for PD symptoms was identified as a predictor of MDS-UPDRS II and MDS-UPDRS III scores. For people treated with levodopa and/or dopamine agonists, MDS-UPDRS III was assessed both pre- and post-dose in the Parkinson’s Progression Markers Initiative data set, while for other medications (e.g. monoamine oxidase-B inhibitors) MDS-UPDRS III was assessed post dose. Additional predictors in the model included age, gender, disease duration, time, baseline MDS-UPDRS scores and prior MDS-UPDRS scores.

The form of the predictive equations was a general linear model. Supplementary Table 2 of Chandler et al. provides the mean predictor coefficients (with standard error) for change within each MDS-UPDRS subscale (I–III) from a prior visit for people with newly diagnosed PD based on baseline values for age, gender, disease duration and MDS-UPDRS scores, time from previous visit in days, MDS-UPDRS scores at a prior visit and PD medication. The mean scores within each MDS-UPDRS subscale increased year on year, which was indicative of greater levels of severity in both motor (subscales II and III) and non-motor (subscale I) symptoms in newly diagnosed PD. The authors also noted a significant difference in MDS-UPDRS II and III scores between people on no PD medication, people treated with levodopa and/or dopamine agonists, and other PD medications alone (e.g. monoamine oxidase-B inhibitors).

In addition to the predictive equations for newly diagnosed PD, the model by Chandler et al. also simulates disease trajectories when people initiate treatments for PD symptoms via a separate set of predictive equations for changes in UPDRS subscales. This second set of predictive equations was developed from an analysis of the National Institute of Neurological Disorders and Stroke Exploratory Trials of Parkinson Disease Long-term Study 1 (NET-PD LS-1), which was a large multicentre, randomised placebo-controlled efficacy trial of creatine, where eligible participants were within 5 years of PD diagnosis and treated with levodopa and/or dopamine agonists. The data from this study (1720 PwP with 5 years of follow-up) were used to develop equations to predict changes over time in UPDRS subscales I–IV, LED, and per cent off-time. Supplementary Table 3 of Chandler et al. provides the mean predictor coefficients (with standard error) for change within each UPDRS subscale (I–IV) from a prior visit based on baseline values for age, gender, disease duration and UPDRS scores, LED, time from the previous visit in days, off-time, UPDRS scores at prior visit, rate of change in prior UPDRS scores and interactions between variables. In the predictive model, LED is predicted to increase over time and impact on the UPDRS subscores I–IV.

Using the predictive equations for disease progression, Chandler et al. developed an individual patient-level simulation model to represent the heterogeneity observed in progression rates and capture the potential long-term benefits of new disease-modifying drugs when they become available, as distinct from symptomatic improvements. Disease progression was characterised in terms of changes in MDS-UPDRS, UPDRS and H&Y stages from diagnosis through to a lifetime horizon. Through the prediction equations, the simulation captures the benefits of starting symptomatic treatments, dose adjustments, increases in off-time and the associated complications of dopaminergic therapies.

A target population of newly diagnosed PwP was analysed by the authors to understand the potential value of treatment with new disease-modifying treatments. A profile of risk factors to predict disease progression, treatment changes and mortality was generated for each simulated individual in the model using the Parkinson’s Progression Markers Initiative data set. When individuals progressed to MDS-UPDRS or UPDRS subscale III, a UPDRS III threshold was used to predict the time to H&Y stage 3. H&Y was not collected in NET-PD LS-1; therefore, the longitudinal PRECEPT and PostCEPT studies in early-stage PD were used to predict time to H&Y stage 3. Conversions between the UPDRS and MDS-UPDRS subscales were based on the linear relationship published in Goetz et al. 142 The progression between H&Y stages 3 and 5 was based on published transition probabilities from a previous economic model in PD. 108 General population mortality estimates were adjusted to capture the impact of PD on mortality by applying hazard ratios by H&Y stage.

The model was validated in two ways: (1) the predictive equations developed for the MDS-UPDRS and UPDRS subscales were used to compare the predicted and observed scores each year post-baseline in the corresponding Parkinson’s Progression Markers Initiative and NET-PD LS-1 data sets, and (2) the simulated outcomes from the model with the predictive equations implemented within the model were compared to the observed data in the data sets. Supplemental Figures 1 and 2 demonstrated good validity between the simulated disease progression of UPDRS scores predicted over time and observed outcomes.

Chandler et al. also used their model to conduct an economic analysis for a new hypothetical Disease-modifying therapy (DMT), in addition to the current SoC compared to SoC alone. The analysis was conducted from a UK NHS perspective. The model was used to simulate newly diagnosed patients starting a DMT in addition to SoC. For the DMT, a 50% change in MDS-UPDRS progression compared to the natural history without the DMT was assumed, while a 5% discontinuation rate was included in the first year and a 2.5% in subsequent years. Patients who discontinued DMT were assumed to experience a gradual loss of treatment effect over 2 years, reverting to the natural history of progression on SoC.

For HRQoL, a MMRM model was developed by the authors to predict EuroQol-5 Dimensions, three-level version (EQ-5D-3L) utility values based on an analysis of data collected in the NET-PD LS-1 study and using the UK preference weights. This study had up to 6 years of follow-up with EQ-5D-3L index scores of 1741 observations at baseline and 330 at year 6. The statistical model was used to identify predictors of the association between EQ-5D utility values and PD severity; it found that gender, non-motor and motor aspects of PD (i.e. all four UPDRS subscales) were all significant predictors of EQ-5D-3L utility value. Supplemental appendix, table 4 in the paper provides the mean values (with standard error) for the coefficients of gender and each of UPDRS I–IV subscales to predict EQ-5D-3L index scores, while Supplemental figure 3 demonstrates that the utility equation performs well when validated with the observed data. In the authors’ hypothetical economic analysis for a new DMT, the predictive equation was used to derive utility values up to H&Y stage 3. Once individuals transition to H&Y stage 3, utility values are assigned conditional on H&Y stage and off-time using an approach adopted in a previous economic model in PD. 129 Caregiver disutilities were also assigned conditional on H&Y stage from a study reporting caregiver decrements by H&Y while controlling for age and gender. 130

For resource use and costs, the initiation of medications (levodopa and/or dopamine agonists or other PD medications such as monoamine oxidase-B inhibitors) for newly diagnosed PD was based on data observed in Parkinson’s Progression Markers Initiative for the likelihood of starting PD treatments over time. This was supplemented by symptomatic treatment costs from a study by Kalilani et al. who identified treatment patterns in the UK CPRD database for the period 2004–15 based on 7775 PD patients. 143 The most common PD medications included in the CPRD database (levodopa, 43%; pramipexole, 30.34%; ropinirole, 21.52%; and bromocriptine, 4.78%), the average cost per milligram and LED conversion factors for each treatment were used to derive a weighted average annual cost per milligram LED of £5.01. The percentage of patients receiving advanced therapies and associated treatment durations were based on the OBSERVE-PD (observational, cross-sectional Parkinson’s disease) study,144 while direct medical and non-medical costs per annum by H&Y stage were derived from a UK healthcare survey published by Findley et al. 145

The results of the cost-effectiveness analysis for the hypothetical DMT indicate that over a lifetime horizon, DMT in addition to SoC leads to improved discounted (at 3.5% per annum) QALY outcomes of 1.1 QALYs compared to SoC alone. Patients treated with SoC were projected to incur discounted costs of £232,619 over a lifetime (26% related to treatment costs and 55% from non-medical costs including respite care and nursing home costs). For the hypothetical DMT, without including acquisition and administration costs of the DMT, the total discounted costs were predicted to be lower than SoC by 16% due to reductions in hospitalisations, nursing home admissions and at-home care.

Patient population

The target population in the model consists of patients in the maintenance phase of PD, where the management of symptomatic motor and non-motor features of the disease is routinely required. Patients in the model consist of the average characteristics of participants enrolled in the Woodrow et al. study, given that this study represents the largest comparative assessment of clinical effectiveness for remote continuous monitoring devices. Table 24 summarises the values used for the baseline patient characteristics in the model. Disease duration, levodopa daily dose, H&Y stage and baseline MDS-UPDRS scores all constitute key characteristics in determining baseline disease progression in the model. Sex and age characteristics establish disease progression, patient’s HRQoL and mortality risk.

Expert clinical advice obtained by the EAG indicates the Australian study setting does not preclude its relevance to the UK context. Expert clinical advice sought by the EAG did not preclude the relevance of this population to the UK context. However, compared to the patient population in the UK, the trial population may represent a younger and more sex-balanced cohort than seen in UK clinical practice with the average age of Parkinson’s diagnosis in the UK occurring between 70 and 79 years of age and prevalence is notably higher amongst men. 4,148 The model population aims to reflect the average characteristics of patients with maintenance stage PD; however, the study by Isaacson et al. represents a more severe subpopulation of those experiencing clinically significant motor symptoms insufficiently controlled by current therapy. The EAG acknowledges these limitations.

Monitoring schedules and settings

Patients with PD require regular follow-up care. Appointments serve as an opportunity for healthcare professionals to review, test and treat a wide range of Parkinson’s-specific and age-related symptoms, events and complications (e.g. motor issues, behavioural abnormalities, falls, dementia, blood pressure, constipation, etc.). In UK clinical practice, appointments are conducted with a consultant or community PD nurse specialist, either via face-to-face consultations or remotely using phone or video appointments. At present, no standard template exists for monitoring within current care pathways with or without the introduction of remote monitoring devices. Remote monitoring devices have the potential to be introduced in a variety of ways, including on a targeted basis, where clinicians believe specific circumstances warrant its application; as a clinical aid used across all follow-up periods or as an intermediary tool to improve clinical understanding while facilitating targeted follow-up (only assessing those in need). After consultation with experts, reviewing real-world applications of remote monitoring devices and considering company responses, the EAG modelled one-time and routine applications of remote monitoring devices, with an alternative recurrent use assessed in a scenario analysis. The assumed schedules and settings for SoC and for the alternative remote monitoring strategies are detailed below.

Patient health-related quality of life

As previously discussed, health benefits from monitoring devices are assumed to be as a consequence of better symptom control and improved management of the disease. This is captured in the model via changes in UPDRS domain scores. To estimate HRQoL benefits associated with the monitoring devices, changes in UPDRS domain score were linked to HRQoL improvements using an algorithm reported in Chandler et al. 140 The Chandler et al. algorithm was based on a regression analysis using EQ-5D-3L data from the NET-PD LS-1 database, which includes 1741 PwP from the USA and Canada followed-up for maximum of 6 years. Data were analysed using UK preference weights for the EQ-5D-3L responses and a MMRM model to account for repeat measurement and included gender and UPDRS domain covariates (having tested for the impact of other patient covariates).

The EAG also considered two alternative sources of HRQoL data: NICE CG71 and Fundament et al. 98,137 Both alternative value sets were, however, based on data from patients with advanced disease and therefore did not reflect the modelled population. In the case of Fundament et al. , there were also specific methodological concerns with how the value set was generated (see Results of the systematic review of decision models evaluating interventions in Parkinson’s disease). The EAG, therefore, deemed Chandler et al. ’s algorithm the most appropriate source for estimating HRQoL within the UK decision-making context for management-phase PD. Table 28 reports the modelled coefficients.

The total incremental QALY gains for the remote continuous monitoring devices compared to SoC are derived from the magnitude of treatment effect (see Table 26), persistence of effect over time (see Efficacy) and HRQoL gains (see Table 28) associated with the UPDRS differentials between the alternate monitoring strategies. Differentials in UPDRS and associated HRQoL are estimated on a per cycle basis in the model (see Progression of disease). The average HRQoL utility values in the enhanced maintenance health state (using the restricted approach for estimating the efficacy of PKG) is displayed in Figure 9.

FIGURE 9.

Modelled HRQoL in the standard maintenance and in the treatment-specific enhanced maintenance health states for the remote monitoring devices.

Although PDQ-39 summary scores were available in Woodrow et al. , the omission of dimension-level individual participant data precluded the EAG from generating treatment-specific utility values by mapping the PDQ-39 onto EQ-5D. 152

Carer quality of life

Caring for PwP can place a burden on informal caregivers, negatively impacting their (health-related) quality of life. 153 Studies have shown that functional status is an important determinant of carer quality of life, with several studies emphasising mobility and cognitive impairment as drivers of carer quality of life. There is, however, no direct evidence linking the use of remote monitoring devices to carer quality of life. Moreover, the broader literature does not show any consistent relationship between relevant clinical outcomes such as UPDRS. 154 It was therefore not possible to account for any impacts on carer quality of life within the economic analysis. This may represent an uncaptured benefit given the observed improvements in UPDRS and other clinical outcomes.

Remote monitoring device costs

The costs of PKG, Kinesia 360, STAT-ON, KinesiaU and PDMonitor devices were based on company responses to the NICE request for information. The alternative devices had three types of payment mechanism: (1) pay per use, (2) subscription model or (3) outright purchase of the device. VAT was not applied to device costs as outlined in the NICE Diagnostics Assessment Programme manual. 100 The costs of the devices are reported in Table 29.

The PKG device requires a payment of £225 per application. The cost is inclusive of the postage of the data logger to the patient, postage back to Global Kinetics Corporation (GKC) and the PKG report made available via the online portal. Kinesia devices use a subscription service cost model with monthly fees of £224 and £64 for Kinesia 360 and KinesiaU products, respectively. KinesiaU comprises patient-level costs for access to the company’s smartphone app (£5 per month) and clinician-specific costs for access to the KinesiaU portal (£59 per month). STAT-ON uses a subscription model with an annual licence fee (£1600). This grants the user(s) a device, charger kit and adjustable belt with clinical and technical support as well as a 2-year warranty. Kinesia 360, STAT-ON and PDMonitor allow multiple users to access the subscribed or purchased devices (albeit with new straps required). Besides PKG, it is unclear whether broader costs for the delivery and management of devices are included in the company costs (e.g. device delivery, administration, relevant support, etc.). The costs of potential loss or damage to devices is not stated by the companies and for simplicity is not included in the model.

The EAG assumes subscription models (Kinesia 360, KinesiaU and STAT-ON) continuously run over the course of the model time horizon for routine strategies and for the recurrent monitoring scenario. PDMonitor is treated as a one-time up-front cost, with no additional intervention costs for repeated use. The EAG believe this to be reasonable on the basis that the model time horizon falls within the company stated lifetime of the device (approximately 7 years). For one-time use remote monitoring strategies, it was assumed that a 3-month subscription was required for Kinesia products (in line with the 12-week follow-up in Isaacson et al. 33), and a 1-year subscription for STAT-ON. The EAG acknowledges that the one-time monitoring strategy does not align with the company’s positioning of purchased (PDMonitor) or subscription-based services (Kinesia 360, KinesiaU and STAT-ON) and may incur further administrative burden and implementation costs relative to one-time PKG use.

Implementation costs

The costs required to successfully implement remote monitoring strategies into service pathways were divided into fixed implementation costs (those irrespective of device use) and variable costs (those specifically incurred from using a remote monitoring device in clinical practice).

Fixed implementation costs were calculated in accordance with staff training times noted in company responses, with the mid-point selected within ranges (PKG and STAT-ON: 90 minutes; Kinesia 360, KinesiaU and PDMonitor: 30 minutes). This equated to a £56 cost per patient, assuming clinician-level fixed costs are distributed over eight patients. 69,155 Implementation costs may include clinician training (participating clinicians in the Woodrow study received a day of training in interpreting the PKG) and a variety of process factors (e.g. administration, procurement, etc.).

Variable implementation costs may exist given the potential for remote monitoring devices to require additional patient support and healthcare professional time required to arrange its application and review findings. The base-case analysis applies zero variable implementation costs, assuming consultation costs sufficiently cover potential broader service factor costs. Scenario analyses considered the removal of implementation costs and the addition of variable costs (an additional £39 cost per consultations using a remote monitoring device, equivalent to 15 minutes of general practitioner time).

Consultation costs

In line with Woodrow et al. , it was assumed that several initial face-to-face consultations would be required. For all remote monitoring strategies patients were assumed to undertake 2.57 initial visits, while patient receiving SoC were assumed to receive 2.17 visits.

For subsequent visits, the consultation setting and their associated costs are dependent on the monitoring strategy (i.e. one-time use or routine monitoring). As described previously (see Monitoring schedules and settings), patients receiving SoC are assumed to undertake review appointments every 6 months, with 55% of consultations conducted face to face and 45% remotely. This split was informed by activities reported in the 2019–20 NHS reference costing schedule (from 158,768 recorded specialist Parkinson’s and Alzheimer’s nursing/liaison activities). 149 Face-to-face and remote consultations were assumed to cost the NHS £81.41 and £56.41 respectively, in line with NHS reference costs 2019/20. 149

The one-time use remote monitoring strategies were assumed to align to the setting and follow-up schedule received with SoC. Relative to SoC and one-time use strategies, the routine and recurrent remote monitoring strategies are assumed to increase the proportion of consultations that can be conducted remotely based on evidence from Evans et al. 69 To incorporate this cost-saving, the routine and recurrent (scenario analysis) strategies assume 79% of consultations are conducted remotely (compared to 45% in SoC). The recurrent scenario strategy also allows 79% of patients to avoid an interim review between annual appointments (with the remaining 21% requiring face-to-face consultation). To consider the potential for remote monitoring strategies to mitigate the need for consultation, alternative consultation savings were explored in sensitivity analysis. Table 30 provides an overview of the costs and patient consultation settings for SoC and each monitoring strategy.

Medication costs

Medication costs in the model were based on the average cost per milligram of LEDD regimens reported in Chandler et al. 140 The study calculates the average cost per milligram of LED in the UK to be £5.01 per year, from the medication composition published in an analysis of the UK Clinical Practice Research Datalink database between 2004 and 2015 (levodopa 43%, pramipexole 30.34%, ropinirole 21.52% and bromocriptine 4.78%) and drug costs from the Monthly Index of Medical Specialties database. 143

Across all remote monitoring strategies, patients within the enhanced maintenance health state in the model were prescribed an additional 22 mg LED, in line with the EAG adjusted estimates of the differences in LEDD consumption from the Woodrow et al. study (see Woodrow individual participant data). Based on previous economic evaluations, it was assumed LED doses increase at a rate of 10% per annum. 156–158 Differentials in LED between remote monitoring strategies diminish at the assumed rate of decay in treatment benefit, whereby differences are maintained over the model time horizon in the base-case routine and recurrent remote monitoring strategies (assuming no efficacy decay), and diminish at an assumed positive rate in the one-time remote monitoring strategies (50% base case).

Base-case analysis

The base-case analyses present deterministic and probabilistic pairwise comparisons between PKG and SoC and Kinesia 360 and SoC for one-time use and routine remote monitoring strategies. Cost-effectiveness analyses of PKG were presented both with restricted and unrestricted efficacy (i.e. restricted to only assessing UPDRS domains III and IV). A base-case analysis with an incremental comparison of SoC, PKG and Kinesia 360 was also presented. The base-case parameters and their associated assumptions and sources are detailed in Table 31.

The core structural assumptions underlying the economic analysis are detailed in Table 32.

Scenario analyses

A number of scenario analyses are considered in which the alternative strategies and assumptions are input into the economic model and results compared to base-case findings. These analyses are undertaken to assess the robustness of the base-case results to key uncertainties. Details of each scenario, including applicable model element, the relevant position taken in the base-case analysis, which strategies the scenario is applicable to and the alternative assumption applied is presented in Table 33.

Model validation

The model was developed in Excel by EC and validated by a second analyst (RH). As part of an overall quality assurance process, the internal validity of the model was assessed by extensively exploring logical consistency in the model results.

Clinical effectiveness

Overall, the EAG considers that only PKG has a substantial body of research evidence. PKG appears to accurately measure dyskinesia and bradykinesia, with very high sensitivity and reasonably high specificity. Diagnostic accuracy was also reasonably high for measuring tremor and treatment-related outcomes. However, its accuracy for measuring sleep disturbance was lower.

PKG is being used by clinicians to guide treatment decisions, primarily the addition of a new therapy or increase in treatment dose; PKG use led to a change in management for 32–79% of patients. Patients managed with PKG appear to benefit more than those on clinical management alone, with improvements in UPDRS III and IV scores. This benefit seems to depend on whether patients were in-target (i.e. their condition was under control with current treatment) before PKG use. Patients not in-target saw improved UPDRS scores, but those in-target did not.

For STAT-ON, evidence is almost entirely limited to diagnostic accuracy studies. These suggest that STAT-ON can accurately diagnose on–off times and bradykinesia. STAT-ON also seems to have reasonably good accuracy for diagnosing freezing of gait and possibly trunk dyskinesia (as a waist-worn device), but not dyskinesia elsewhere.

There is currently no evidence on the intermediate impact of STAT-ON or on the clinical impact of STAT-ON. Therefore, it is unclear whether STAT-ON use will lead to treatment modification and subsequent improvements in symptoms and quality of life.

Two small RCTs suggest favourable clinical outcomes with Kinesia 360 use in populations receiving rotigotine or when Kinesia 360 was used for remote telehealth assessments. However, the EAG considers that there is currently too little evidence to be confident about its clinical value.

Evidence on KinesiaU and PDMonitor are too limited to draw any conclusions on their clinical value.

The only evidence relating to adverse events was that there were no device-related adverse events reported (PKG and STAT-ON). Almost all the available evidence for all devices focused on motor symptoms (bradykinesia, dyskinesia and tremor). Evidence on sleep disturbance, and other non-motor aspect PD was extremely limited.

Cost-effectiveness

The base-case cost-effectiveness results for one-time use and routine PKG and Kinesia 360 remote monitoring strategies found ICERs exceeding £30,000 per QALY gained. The EAG were not able to evaluate the cost-effectiveness of STAT-ON, KinesiaU or PDMonitor. Over a 5-year time horizon, modelled device costs for routine use were lowest for PKG, provided three devices or less were ordered per annum, followed by KinesiaU, STAT-ON, Kinesia 360 and PDMonitor. Base-case QALY gains from PKG remote monitoring strategies were highly sensitive to the inclusion of associated small, unfavourable and statistically insignificant changes on the UPDRS II domain.

Scenario analyses considered an additional monitoring strategy and alternative model assumptions from those used as part of the base-case analysis. Cost-effectiveness results were largely robust to changes in consultation setting (face to face vs. remote), fixed implementation costs and potential consultation savings. The scenarios were also used to identify the main drivers of cost-effectiveness. The key drivers identified were: (1) the direction and magnitude of changes on the UPDRS associated with remote monitoring strategies, (2) the persistence in changes to UPDRS (treatment waning) and (3) the number of devices requested (PKG).

The EAG was not able to evaluate the cost-effectiveness of STAT-ON, KinesiaU or PDMonitor due to a lack of comparative clinical effectiveness evidence. In a cost comparison (assuming a 5-year time horizon), modelled device costs were lowest for PKG, provided three devices or less were ordered per annum, followed by KinesiaU, STAT-ON, Kinesia 360 and PDMonitor.

Strengths

This is the first complete systematic review of all available diagnostic and clinical evidence for PKG, Kinesia 360, KinesiaU, STAT-ON and PDMonitor. This review used extensive database searches to identify all published evidence on the included technologies and followed rigorous recommended review methods to identify relevant publications, assess their risk of bias and undertake a narrative synthesis of the results. As such, this is the first fully rigorous review of these technologies, and also the first to compare the technologies in one review.

The review was strengthened by the provision of individual-level data for the key clinical trial of PKG, which permitted a more thorough examination of the clinical impact of PKG than would otherwise have been possible.

This is also the first study to review and estimate the cost and cost-effectiveness of PKG, Kinesia 360, KinesiaU, STAT-ON and PDMonitor technologies relative to SoC and each other (where possible). A de novo economic model was developed to assess the costs and consequences associated with one-time and routine applications of each technology within management-stage PD. The model made the best use of systematically identified evidence of clinical effectiveness, considered a broad and sensitive measure of PD severity, progression and symptomatic benefit, and utilised contemporary evidence of disease progression, HRQoL and NHS service utilisation associated with PD and remote monitoring.

Limitations

There was a lack of replication across studies. In general, most outcomes were reported in only one or two studies, or outcomes were reported in inconsistent ways across studies. This meant that no meta-analysis was possible for any included studies and the narrative synthesis was severely limited by the consequent difficulties in comparing different studies. This lack of replicability raises some concerns as to how robust the findings of the review are. It should be noted that many of the review conclusions are based on individual studies.

A further limitation is the low quality of much of the evidence, particularly for diagnostic accuracy. There were many low-quality case-control studies, and in most studies it was unclear whether the reference standard (usually clinical opinion) was robust. This casts some doubt on the validity of the diagnostic accuracy of evidence. It should be noted that few studies were formal diagnostic accuracy studies, and there are innate difficulties in this field in robustly assessing diagnostic accuracy given the lack of clear reference standards, and lack of clarity over the exact algorithms used to convert device output into diagnostic assessments. There were also quality concerns for the clinical evidence, as most studies were not comparative, and no randomised studies have been performed for any device.

Cost-effectiveness results were limited for a number of reasons and should be interpreted with caution. The EAG identified no evidence to reliably establish the clinical value of STAT-ON, KinesiaU or PDMonitor, thereby restricting any meaningful assessments of cost-effectiveness for these devices. The evidence used to inform the clinical effectiveness of Kinesia 360 is extremely limited and unlikely to be comparable with that used for PKG, making comparisons problematic. Conversions made between MDS-UPDRS scores and UPDRS are at risk of bias. Cost-effectiveness results were highly sensitive to the persistence in initial clinical improvements, a variable that could not be informed with current evidence. Furthermore, cost-effectiveness assessments were confined to management-phase PD assessments only. Remote monitoring devices are applicable to a wide variety of potential schedules, contexts and settings. The consequences, implementation costs (e.g. healthcare professional time and administration) and risks (e.g. loss, damage or theft of devices) associated with alternative delivery model configurations is unknown and likely to impact cost-effectiveness.