An app-, web- and social support-based weight loss intervention for adults with obesity: the HelpMeDoIt! feasibility RCT

Goal-setting and self-monitoring (intrapersonal processes)

The important role of goal-setting and self-monitoring is well established in behaviour change research. In a meta-analysis of behaviour change interventions for physical activity and healthy eating, more effective interventions were shown to combine self-monitoring with at least one other technique derived from control theory (e.g. intention formation, specific goal-setting). 17 Goal-setting and self-monitoring are two of the most commonly used behaviour change strategies in weight loss interventions, and are typically used in conjunction. 20,21 A recent meta-analysis of behaviour change techniques in 48 weight loss interventions found that goal-setting and self-monitoring were the most effective components of the interventions. 21 Other intrapersonal processes, which we planned to address within our intervention, included intrinsic motivation, self-efficacy, action-planning22 and implementation intentions23 (described further in Chapter 2).

Social support (interpersonal process)

Social support and its relation to health behaviour change is undertheorised. This is partly because social support is a broad and somewhat loosely used concept; for example, it is an element in the widely used terms of ‘social capital’ and ‘social networks’, which are used to frame ideas about social support. 24,25

Social support is multifaceted: there are various types of social support (e.g. emotional, instrumental, informational)26,27 and there are different kinds of support-giving/-receiving behaviours (e.g. reinforcement, encouragement, motivation, feedback, empathy, role-modelling). 28 Although it is generally used as a positive construct, social support may also have negative elements, such as bullying or co-dependency. 24,29 Minimising negative support, as well as promoting positive support, should be a consideration of behaviour change interventions. 29

Social support is also conceptualised in varied ways in terms of who provides the support. Family,30 friends and colleagues,31 influential people within existing social networks,32 and fellow members of groups with a shared behavioural goal (e.g. weight loss, exercise)33 have been found to be effective in supporting behaviour change in alcohol consumption, smoking prevention and cessation, physical activity, diet and sexual behaviour.

There is evidence that using a ‘buddy’ or ‘helper’ can be effective for weight loss. One trial31 found that when people came with a friend to a weight loss programme they were more likely to adhere to the programme, have greater weight loss and maintain weight loss than those who came alone. Another study34 exploring predictors of adherence to a weight loss programme found that having a buddy (family member or friend) led to an increase in programme adherence from 79.9% to 96.1%. In a study35 of 704 participants in a 15-week online weight loss programme, 54% of participants chose to use a buddy, and they lost more weight than those who did not have a buddy. The same level of effectiveness was found whether the buddies were romantic or non-romantic. A systematic review36 of 21 studies supported these findings, reporting that spousal support could be effective for weight loss. These findings identify the important role that a buddy, or helper, could have in supporting weight loss.

Individuals draw on different types of support from different people in their network. For example, they may derive emotional support from a close friend in their network but may choose to recruit a more distant friend for that person’s expertise in a particular area. Family and friends are significant social influences on health behaviour as a result of factors such as intimacy, influence and proximity to day-to-day health behaviours. They are also immediately accessible to participants because this type of support does not entail joining any kind of formal group.

Social support is important in the initiation and longer-term maintenance of behaviour change,13,31,37 and is typically employed and theorised as one of several elements of behaviour change interventions. 38 In reviews13,17 it has been identified as one contributing factor to effectiveness, alongside goal-setting and self-monitoring. Common intervention elements theorised to operate in conjunction with social support are self-efficacy,39 perceived control39,40 and social norms. 41 Ferranti et al. 42 found that social support is positively correlated with healthy diet. Social support is also associated with increased physical activity40,43 and can improve weight loss maintenance,31 encourage health-promoting behaviours and promote well-being. 44 There is also evidence that unhealthy lifestyle behaviours are correlated with less social support. 15 For many of us, a significant proportion of our social contact is now via digital technologies, and therefore an intervention using this medium to facilitate social support may be useful. In one study45 of an online intervention, despite social support not being specifically promoted as part of the intervention, findings demonstrated that perceived social support from existing social networks and the use of self-regulatory behaviours were strong predictors of improved physical activity and nutrition behaviour. Similarly, Neuhauser and Kreps46 argue that communication that is interpersonal, affective (not just rational), interactive, individually tailored and set within an individual’s social context is more likely than other forms of communication to be effective in changing health behaviour, and that this should be incorporated within new technology- and internet-based interventions. This type and quality of social support would be better facilitated through contact with family and existing friends, rather than anonymous online groups. However, social support from friends and families tends not to be incorporated into the formal design of online behaviour change interventions.

The growing accessibility of internet and smartphones

Technology offers opportunities to deliver behaviour change interventions that can reach a large proportion of the population at a low cost. 56 In particular, smartphone apps and web-based interventions can be effective in reaching large numbers of people. 47,57,58 In 2017, internet access was available in 88% of UK households. 59 Smartphones were owned by 85% of adults, of whom 55% reported checking their phone within 15 minutes of waking. 60 Interventions delivered via these technologies have the potential to reach large numbers of people, including ‘Silver Swipers’ (those aged 55–75 years), who were the fastest-growing adopters of smartphones in 2017;60 and those from lower socioeconomic groups, with 73% of people living in Scotland’s 20% most deprived areas having access to the internet. 61

Previous research

There are many smartphone apps (and accompanying websites) for weight loss available that incorporate some or all of the key behaviour change features of goal-setting, self-monitoring and social support [e.g. StickK (Brooklyn, NY, USA), www.stickk.com (accessed 1 June 2016);62 MyFitnessPal (Under Armour, Inc., Baltimore, MD, USA), www.myfitnesspal.com (accessed 1 June 2016)63]. However, a systematic review64 of the most popular apps for weight loss (n = 28) found that the majority were of inadequate quality, lacked evidence-based information on weight loss and lacked appropriate behaviour change techniques. Furthermore, although evidence from research-based interventions using new technologies suggested that they could be effective,47,57,58 the interventions were often simplistic and not based on the best evidence and theory of effective behaviour change. 52 The effectiveness of these interventions could be enhanced by incorporating well-evidenced behaviour change techniques and promoting support through an individual’s social network (family, friends and colleagues).

There is a need to improve our understanding of how interventions involving new technologies effectively facilitate changes, for example factors such as optimal website design, how to maximise exposure to websites or what type of prompting works best are areas that still require development. 65 Particular aspects of new technologies may enhance interventions, such as through higher intensity (e.g. more frequent contact with people in an individual’s social network). This may increase the success of an intervention13 but at a lower cost than traditional methods. Some applications may also allow for more personalisation or individual tailoring of an intervention to suit individual needs, which may also improve success rates. 66 However, the evidence base is limited and, to date, somewhat mixed. 67 Therefore, although there have been promising signs in this emerging field, such as the potential for high reach and for engaging hard-to-reach groups, there is a need to address research gaps in understanding how new technologies might support or enhance known health behaviour change mechanisms.

Currently available websites or apps for weight loss use various strategies for behaviour change, including elements such as monetary incentives or prizes, competing with others and behavioural goals. Some apps provide an element of social support, such as a chat forum. 63 There is some evidence to suggest that online social networks can have a positive impact on health behaviour change. 68 However, online users are typically not known to each other and the apps are not designed to harness the ‘offline world’ and the immediate support of family, friends and colleagues. Evidence indicates that support from key individuals in a person’s life is more effective than that provided by anonymous online contacts. 13

The perceived value of, and demand for, social support has resulted in many health behaviour change websites having chat forums or bulletin boards, which facilitate support from other users. These provide empathy and encouragement, but may not be able to build on evidence of the importance of who provides the social support and the many mechanisms through which social support can facilitate and sustain behaviour change. None of the resources we explored focused on the combination of elements that we used in our intervention, most importantly social support from key individuals within that person’s social network. These are individuals whom they know well, and who are part of their everyday lives and available to support them when needed, in a sustainable ongoing way.

The summary above has identified goal-setting, self-monitoring and social support as promising behaviour change techniques. 69 Although they have featured in a number of apps, for which there is mixed evidence, we have not identified any existing intervention that specifically aims to mobilise support from existing social networks using an application- (app) and web-based intervention. This is the unique aspect of the HelpMeDoIt! intervention, and this is the first study to our knowledge to explore the feasibility of engaging social support from existing social networks, in combination with goal-setting and self-monitoring, in a digital health intervention for weight loss. If brief engagement with an app could catalyse input from existing social connections to support longer-term change, then this could offer a sustainable approach to behaviour change.

Policy

Improving health behaviours is a priority for government. However, current health behaviour change initiatives require improvements in their reach and effectiveness to have a significant impact on the population’s health. The House of Lords Science and Technology Select Committee report on behaviour change8 highlighted that no single approach is likely to be effective in tackling priority health behaviours and that complex interventions addressing multiple levels of behavioural determinants are likely to be needed to bring about sustained change. HelpMeDoIt! is a complex intervention addressing two of these levels: the intrapersonal and the interpersonal.

Economy

Lifestyle-related illness represents a significant cost to the NHS. One-to-one individualised lifestyle interventions are unlikely to yield substantial population-level improvements at a realistic cost to the public purse unless they are highly effective. In comparison, web- and app-based behaviour change interventions can reach substantial numbers at a lower cost. New technologies, such as smartphone apps, present opportunities to promote healthy lifestyles cost-effectively on a large scale. 70 Content, covering evidence-based behaviour change components with the engagement of community-based social support resources, can be delivered in an engaging and accessible way.

Evidence base

Although the intervention elements of goal-setting, monitoring and social support are well established, and although new technologies have shown promise, the evidence base is limited and theoretically underdeveloped. 50,52 Studies are often limited by small, short-term effects71 and high attrition. 12,72 There are significant gaps in understanding how these elements work together, for example how social support operates through personal networks mediated by new technologies, and what impact this has on mechanisms such as monitoring and goal-setting. There is a need to (1) explore the application and mechanisms of goal-setting, monitoring and social support via web and app-based interventions; (2) explore how they interact with each other: and (3) test this type of intervention in both a feasibility and a full-scale effectiveness trial. Feasibility trials of this nature are a necessary first step in developing public health improvement interventions, particularly where mechanisms, such as social support, are not well understood and where technological innovations present new possibilities.

Future impact

The proposed intervention has the potential to have both reach and effectiveness in all socioeconomic groups including those who are traditionally hard to reach. If the intervention were proven effective in a future powered trial, it could be applied to other behaviour change areas and would be universally available through a free-to-access website and/or promoted in specific NHS and community settings across the UK.

Approach 1: the 6 Steps in Quality Intervention Development (6SQuID) model

This model by Wight et al. 74 was developed to address gaps in current guidance for the development of interventions. For example, the Medical Research Council (MRC) guidance on developing and evaluating complex interventions77 is primarily devoted to evaluation and does not provide sufficient detail on actual intervention development. The 6SQuID method involved six steps:

(1) defining and understanding the problem and its causes; (2) identifying which causal or contextual factors are modifiable: which have the greatest scope for change and who would benefit most; (3) deciding on the mechanisms of change; (4) clarifying how these will be delivered; (5) testing and adapting the intervention; and (6) collecting sufficient evidence of effectiveness to proceed to a rigorous evaluation.

Wight et al. 74 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/

We adhered to this six-step process, and used it as the overarching approach throughout our intervention development from initial idea to testing in the feasibility trial.

Approach 2: the Person-Based Approach

The Person-Based Approach is a framework for developing digital interventions that are based on a comprehensive understanding of the social, psychological and environmental context of the target group. 75 This approach involves potential users in the development of the intervention and incorporates their perspectives. As well as focusing on the views of users about engagement, content and usability, crucially it also addresses the behaviour change techniques included in the intervention. We considered the Person-Based Approach complementary to the approaches we were already using to identify theory- and evidence-based approaches to changing weight-related behaviours. The Person-Based Approach provided a systematic approach to intervention development, relying on qualitative methods throughout the whole process to inform intervention design. This type of iterative, consultation approach, understanding user perspectives and incorporating key contextual influences, was key to developing an intervention that could be engaging and have a chance of being effective.

Approach 3: the Behaviour Intervention Technology model

Another digital health-related framework is the Behaviour Intervention Technology (BIT) model. 76 The BIT model was developed to fill a gap in the literature on the design of behavioural intervention technologies, such as lifestyle interventions using web-based technology and mobile phones. The BIT model helps answer the questions why, what, how and when. For example, ‘why’ was reflective of the intervention aims; ‘what’ includes the BIT elements such as app notifications and report logs; ‘how’ includes behaviour change strategies, such as goal-setting, and technical characteristics, such as personalisation of the app; and ‘when’ refers to the navigation through the app/website as determined by either the software or the user. This model was particularly helpful in guiding the development of software characteristics.

Approach 4: utilising theory and behaviour change techniques

Between 36% and 89% of interventions that seek to change health behaviours are not clearly theory based. 78 Interventions based on theories of behaviour change have been shown to be more effective, although reviews relating the impact of using theory to the success of interventions have shown mixed results, and theory is often poorly applied. 79 The use of behaviour change techniques and how they relate to theoretical concepts is also often inadequately reported. 80 We sought to address both of these issues by exploring multiple behaviour change theories and identifying the most appropriate candidate theories and associated behaviour change techniques that could be useful in the HelpMeDoIt! intervention. We used these theoretical underpinnings to develop a programme theory, identifying multiple causal mechanisms describing how the HelpMeDoIt! intervention could lead to positive outcomes for weight management in adults with obesity.

Figure 2 presents how these four approaches were combined. This involved the 6SQuID model being used as an overarching cradle-to-grave process, with the Person-Based Approach, the BIT model and behaviour theory approaches being incorporated to address more specific development issues. The other elements represented in Figure 2 will be described in detail throughout this chapter.

FIGURE 2.

The combined framework for the development of the HelpMeDoIt! intervention.

6SQuID step 1: ‘define and understand the problem and its cause’

The initial step of intervention development involved undertaking a literature review to identify up-to-date evidence on obesity. This involved gathering data on the prevalence, causes and associated risks. Obesity is a well-researched area, so there was extensive evidence providing insights into this public health problem. The causes of obesity are complex and multilayered, and the solutions are likely to involve intervening at multiple levels, including individual, interpersonal, organisational, community and policy. The HelpMeDoIt! intervention focused on the first two of these levels. An overview of our literature review findings was provided in Chapter 1. Further detail can be found in our study protocol (see Report Supplementary Material 1).

6SQuID step 2: ‘clarify which causal or contextual factors are malleable and have greatest scope for change’

As part of the above review, we explored causal and contextual factors that could be influenced as part of the intervention. The key influencing factors associated with weight loss are diet and physical activity. Harnessing positive social support was identified as an important factor, as were motivation and self-efficacy. These have already been discussed in Chapter 1.

6SQuID step 3: ‘identify how to bring about change: the change mechanism’

Identifying the potential mechanisms of change was a critical step in the development of the intervention. It was at this stage that we incorporated the first of our other intervention development approaches by identifying relevant behaviour change theories and techniques. We conducted a search to identify the intervention components most likely to contribute to successful weight loss, as well as components associated with successful and unsuccessful interventions. We also conducted a search for web-based and other technology-based interventions and those involving any kind of social support, whether from friends, family, colleagues or groups such as Weight Watchers. We also conducted an internet search for websites or apps aimed at changing health behaviours around diet and physical activity. We found many sites that utilised monetary incentives or prizes, competition with others, behavioural goals, and social support elements such as chat forums. However, we did not identify any that included family members and friends to encourage and promote weight-related behaviour change in the way we envisaged. Our review of the literature and 6SQuID steps 1 and 2 provided us with the rationale to develop an intervention for adults with obesity, using technology in the form of an app/website, to facilitate social support for weight loss. The resulting intervention is, thus, a combination of (1) individual behaviour change approaches, (2) social support and (3) the use of technology.

6SQuID step 4: ‘identify how to deliver the change mechanism’

Version 1.0 of the logic model was based on the research evidence, using 6SQuID steps 1–3. The logic model was used as a starting point for then engaging with stakeholders, with a view to further refining the mechanisms of change and intervention content. At this point we began developing the intervention alongside potential users and technical experts from a software company. This was a critical stage in our intervention development and involved the use of the two other approaches: (1) the Person-Based Approach75 and (2) the BIT model. 76 Each of these explored how to effectively deliver the mechanisms of change and specific content of the interventions.

6SQuID step 5: ‘test and refine on a small scale’

In addition to our development panel, we used a separate testing group to gather feedback at various points throughout the development process.

6SQuID step 6: ‘collect sufficient evidence of effectiveness to justify rigorous evaluation/implementation’

In step 6 of the 6SQuID process, the resulting version 2.0 logic model and intervention were ready for implementation and testing in a feasibility RCT with accompanying process and health economic evaluation (stage 2). The methods and findings for this stage are presented in Chapters 3–6.

Findings from the development panel

We recruited 10 individuals with a range of characteristics to our development panel. We had a good gender balance (female, n = 6; male, n = 4) and spread of ages (18–70 years). All participants were interested in losing weight. We also attempted to include individuals with characteristics associated with a higher likelihood of obesity, and thus included participants of non-white ethnicity (n = 2) and a greater number of individuals from areas of lower socioeconomic status (n = 7). We included two individuals with limited experience of using smartphone apps (the majority of people who responded to the study advert were experienced in this). When participants were unable to attend one of the focus groups, their feedback was collected via a one-to-one interview (n = 1) or by e-mail (n = 2).

Insights and feedback from the development panel were useful and informative at all data collection time points. Participants were engaged, motivated and creative in their approach to the software development and many of their ideas and suggestions were incorporated into the prototype of the app and website. Some suggestions were not included as they were beyond the scope of the app, such as taking photographs of a meal and the app providing an accurate nutritional analysis.

Thematic analyses of the focus groups identified three main themes and related subthemes. These are detailed in Table 5 along with examples of how the different themes were addressed within the app and website.

Findings from the testing group

Feasibility of the app and website was also evaluated by the 28 participants recruited to the testing group. We purposely aimed to recruit participants with a range of characteristics: 19 were women and 9 were men; age range was 18–64 years; nine participants were from the top two quintiles of socioeconomic deprivation; two participants were from minority ethnic backgrounds; and two individuals had limited experience of using apps (the majority of people who responded to the study advert were experienced in the use of smartphone apps). Participants were recruited via word of mouth (n = 15) (e.g. by members of the original developmental panel to people in their social network), study adverts on Gumtree (www.gumtree.com) or Facebook (Facebook, Inc., Menlo Park, CA, USA) (n = 8), and a university-wide staff e-mail announcement (n = 4). Although feedback was collected on an ongoing basis between August and December 2015, participants provided feedback on only one occasion as a ‘user’ (n = 11), a ‘helper’ (n = 11) or both (n = 6). Detailed feedback from the testing group is presented in Appendix 4. A brief overview is provided below. Overall, feedback from the testing group was constructively critical.

Early feedback at the first time point (August and September 2015) focused on gathering feedback on the initial ideas and designs for the app and website. Feedback was positive, with participants in agreement about the proposed key features of goal-setting, self-monitoring and helper interaction, as well as how these would be delivered. They provided feedback on the initial design, look and feel of the app, and the majority of responses were positive comments on the simple design and colour scheme. Helpful suggestions related to grammar were received to help make the app more engaging. They also provided insights to help the team refine the initial design, for example guidance to choose from more than one goal category.

Feedback collected at time point 2 (October and November 2015) focused on participant insights on our refined app and website ideas. Feedback highlighted that some guidance was required to help participants choose and set goals using the app. This phase of testing also identified useful content to support helpers in their supportive role. Ideas participants provided for the helper section of the website included (1) example conversations between the helper and their friend; (2) guidance on what a helper could do if their friend was struggling to meet their goals; (3) things not to say to their friend; and (4) an online quiz to engage and motivate helpers:

When mentoring at my work it’s about trying to be encouraging so maybe some examples of encouraging dialogue.

TG08, female, 33 years

At the final time point (December 2015), participants were presented with the prototype version of the app. This was accessed using a third-party testing platform. This enabled participants to access and use the app in its beta version and sync to the latest version as features were updated and errors were rectified.

Feedback was gathered via the ‘think-aloud’ approach. 87 This approach facilitated identification of software bugs and errors, which the software company worked on rectifying immediately (e.g. icons obscured by text, inconsistent display of progress graphs). Many of the software bugs were caused by the interaction of our software with a third-party testing platform. Some of these were addressed when the third-party testing platform released an update near the end of our testing phase. In addition to software issues, the feedback helped highlight navigation issues, things that were not intuitive in the design and areas of the app that could be strengthened further. One key suggestion was the need for tutorial guidance to help first-time users understand the process of choosing template versus custom goals:

I’m not sure what I’m meant to do here [on goal-setting screen]. Do I press this button? How do I go back and see the list of goals again?

TG18, female, 39 years

When time allowed, some participants in the testing group also provided feedback on the website (n = 8). Feedback for the website was very positive, with the majority of participants commenting on the simple layout, easy navigation, clear display of information and fun animations. Several suggestions were made to strengthen the website further, for example embedding hyperlinks within the text and simplifying some of the grammar.

Usage statistics for the app and website

There were 498 individual logins recorded for the app, showing that the development panel accessed the software regularly during the testing period. All key features of the app were used. The website was accessed by the development panel on 70 occasions, with 687 web page views in total. The average login for each member of the development panel was four sessions of around 10 minutes’ duration. During this initial testing phase, a number of software issues and bugs were identified and several participants had difficulty logging in. This is normal and expected when beta-testing software.

Findings from the heuristic evaluation

A heuristic evaluation was undertaken by two independent technical experts. The aim of the evaluation was to identify strengths and limitations of the software and to highlight areas for further refinement. This involved applying and assessing set heuristic criteria to both the app and the website. The heuristic criteria were scored on a scale of 1–5 (1, very poor; 5, excellent).

Overall, the findings of the heuristic evaluation were positive. The full heuristic report is presented in Appendix 3. The majority of criteria for the website (71%; n = 35 of 49 criteria) and the app (71%; n = 24 of 34 criteria) were assessed as either ‘good’ or ‘excellent’. Eleven issues were identified as needing improvement. Most of these were minor, for example slowing down the images on the website homepage, and clarifying the goal template headings in the app. Two issues with the app were rated as ‘poor’ and related to a lack of a clear method of returning to (1) the main navigation menu and (2) the main dashboard. All issues highlighted by the heuristic evaluation were addressed by the study team and the software company.

Findings from the Usefulness, Satisfaction, and Ease of Use questionnaire

During the final focus group, participants were asked to complete the Usefulness, Satisfaction, and Ease of Use (USE) questionnaire. 86 This explored the acceptability of the software and also allowed the study team to assess the feasibility of using this questionnaire as a process measure in the trial. Overall, the questionnaire was quick and simple to complete and it was identified as useful for stage 2. Participants responded to 30 individual statements on a scale of 1 (strongly disagree) to 7 (strongly agree). The results are presented in Appendix 5. Despite the app being a prototype, all of the mean responses (with the exception of question 29) had a score of ≥ 4 [i.e. all mean participant responses were greater than or equal to a score of 4 (neutral)]. However, it should be noted that in this initial test run some of the USE questions were difficult to answer as participants did not have access to the software over a longer period of time to be able to assess items such as ‘the software helps me be more effective’. In addition, some participants had access to the software for less than 1 week (owing to difficulties with the login process caused by third-party software), which affected their ability to answer all of the USE questions, as some were not applicable because they referred to longer-term use. The USE questionnaire would therefore benefit from a ‘not applicable’ response option.

The HelpMeDoIt! website

The website presented evidence-based information for participants on healthy eating, physical activity and weight loss, and guidance on how to select and/or be a good helper.

Participant information included:

• guidance on how to use the app to set SMART goals and monitor progress (tracking goals achieved and recording weight) and how to identify appropriate helpers using helpful hints to evaluate the support around them

• evidence-based information on healthy eating (e.g. daily portions of fruit and vegetables, reading food labels), physical activity (current UK guidelines) and behavioural strategies to support weight loss (e.g. regular sleep pattern)

• ‘top tips’ for weight loss (based on the key points of the evidence-based information, e.g. ‘add volume to your meal with liquid or fibre)

• ‘helpful links’ to other relevant pages [e.g. NHS Eat Well (www.nhs.uk/live-well/eat-well/; formerly NHS Food Mixer) for healthy recipe ideas].

Helper information aimed to provide helpers with the guidance needed to be an effective helper for the participant who was trying to lose weight. This information included:

• tips on how to be a good helper (e.g. ‘be a cheerleader not a coach’)

• methods of positive feedback and encouragement to the participant (e.g. the option to send animated smiles or send a well-done text message when their friend completed a goal)

• examples of non-food rewards to help motivate the participant [e.g. new music for their friend’s iPod (Apple Inc., Cupertino, CA, USA)]

• examples of dialogue and motivational language to support the participant (e.g. helpful phrases for helpers to use when their friend had experienced a difficult week)

• Helpers could interact with participants over the telephone, in person, by sending a text message or by sending ‘smiles’ via the app.

The HelpMeDoIt! app

The app focused on the three key elements of the intervention: social support, goal-setting and self-monitoring.

The participant version of the app included the following features.

• Goal-setting: participants were encouraged to set goals for healthy eating, physical activity and other lifestyle factors, such as sleep. There was no maximum goal limit; however, participants were advised that five or more goals could be difficult to manage. They could either choose predefined template goals or write their own custom goals. Custom goal-setting was a two-step process that encouraged participants to action plan and problem-solve as they set their goals. This involved setting a short title for each custom goal, followed by adding SMART goal detail; for example, ‘stand up more’ – watching evening television (TV) on Monday and Wednesday I will stand up each time there is an advert break’.

• Monitor progress: participants could update the app to either log their most recent weight or report when a goal had been achieved. Personal progress was presented over several graphs, including one that showed their change in weight over the past month and also their change in weight since they started using the app; a figure showing their most recent weight; and a list of all set goals, each with a ‘% completed’ bar.

• Nominate helpers: participants could enter the contact details of one or more individuals from their social network that they wanted to invite to be a helper. Nominated helpers received an invitation by e-mail. Participants could also use this feature of the app to remove a helper, for example if the helper was not supportive.

• Helper interaction: the app provided several ways to interact with helpers. These included sending animated smiles (e.g. ‘Great help this week’), sending text messages and making telephone calls.

The helper version of the app included the following features:

• display of the participant’s goals, where they were able to view new, completed and missed goals

• display of the participant’s progress for weight and goals completed

• methods of giving positive feedback and encouragement to the participant, for example sending animated smiles (e.g. ‘Keep up the good work’), sending text messages and making telephone calls.

An element of ‘gamification’ was used within the app to encourage frequent use and to support the ongoing engagement of both participants and helpers. This involved both participants and helpers receiving points for (1) regularly inputting progress data, (2) interacting with each other and (3) achieving goals. Once participants and/or helpers had accumulated a certain number of points, they were awarded virtual medals (i.e. bronze, silver or gold) or a trophy. Ongoing engagement was also supported through the use of (1) push and e-mail notifications (e.g. informative messages, progress summaries and notifications of new badges); (2) daily motivating messages (e.g. ‘Doing well? Think about how you can progress one of your goals this week’); and (3) weekly reminders for uncompleted goals. Helpers were sent (1) daily messages (e.g. ‘A kind word can do wonders for motivation’) and (2) regular prompts via push and e-mail notifications to remind them to provide encouragement, celebration or further support (e.g. ‘Your friend hasn’t set any goals recently. Get in touch to find out how you can help them’).

Overall, the guidance given to participants and their helpers regarding how to establish social support for weight loss was not prescriptive or structured. Participants and their helpers were encouraged to use strategies that worked best for them, and as part of this feasibility study we were interested to find out what the range of approaches were. Participants and their helpers were encouraged to interact with each other either via or outside the app and examples of how they could do this were given in both the app and the website.

Feasibility measures

Feasibility and acceptability of the HelpMeDoIt! intervention and study methods was the key focus of this trial. These relate to the overall progression criteria (see Table 8) and are summarised in Table 9. The majority of these measures will be discussed further in the qualitative and process evaluation methods sections below.

Exploratory primary outcomes

We explored the feasibility of assessing multiple outcome measures, including potential primary outcomes for a future trial. We collected data in relation to three outcomes that could be used as the primary outcome: BMI, physical activity and diet (see Table 9). We explored which of these was the most feasible by assessing acceptability and data completeness. BMI was expected to be the most appropriate measure because it is objective, cheap to measure and more accurately measured than diet and physical activity, and has higher response rates. As measuring diet110 and physical activity111 in community-based trials is challenging, we assessed two ways of measuring these outcomes. Each outcome was measured at baseline and 12 months. Twelve months was chosen as the primary outcome point for the feasibility trial, which would be the same for any future effectiveness trial. This longer-term follow-up is important as, although there may be some benefits of short-term weight loss, weight loss maintenance is key to longer-term health benefits and cost savings to the NHS and society. 112

Body mass index (kg/m²) was calculated from measures of height and weight. Height was measured using a Seca Leicester Height Measuring Stadiometer (Seca GmbH & Co., Hamburg, Germany), with the participant facing forward, wearing no shoes and with their head in the Frankfort plane (parallel to the floor). Height was recorded once, in centimetres, to one decimal point. Weight was measured, without shoes on, using Tanita HD 352 High Capacity Low Profile Electronic Weighing Scales (Tanita Europe BV, Amsterdam, the Netherlands). Scales were calibrated before first use. Weight was recorded once, in kilograms, to one decimal point.

Physical activity was measured using ActiGraph GT3X (ActiGraph, LLC, Pensacola, FL, USA) accelerometers, objective activity monitors that measure the duration, intensity and frequency of physical activity. Participants were asked to wear the accelerometer on their right hip for 7 days during waking hours (except when swimming or bathing). Participants received their accelerometer during a face-to-face visit; this enabled the researcher to demonstrate the correct placement of the device. Data were collected in 1-second epochs at a sample rate of 100 Hz, and converted to 15-second epochs for analysis using ActiLife 6 software (ActiGraph, LLC, Pensacola, FL, USA). 113 Non-wear time was identified by > 60 minutes of continuous 0 counts and removed before analysis. Data were included for analysis when the accelerometer had been worn for a minimum of 4 days, and with a minimum wear time of 10 hours per day. Freedson cut-off points98 were used to determine the amount of time spent sedentary and in moderate to vigorous physical activity. The 7-day Physical Activity Recall questionnaire114 was used to measure physical activity by self-report. Participants, guided by the researcher, reported their activity over the previous 7 days in relation to moderate, hard and very hard exercise. This measure has been validated for use in adult populations,99 and researchers adhered to the protocol published by Sallis et al. 114

Diet was measured with the Dietary Instrument for Nutrition Education (DINE) questionnaire,96 a validated seven-item questionnaire that explores the frequency of consumption of different food types, for example bread and rolls, cereals and meats. Field workers asked participants to report the frequency with which they ate specific foods. The frequencies were scored using DINE guidelines96 to produce an overall score for fat and fibre. Diet was also measured using repeat 24-hour dietary recall97 collected by a researcher over the telephone on 4 separate days within a 10-day period (including one weekend day). Participants self-reported their food intake, prompted by the researcher, for the previous 24 hours. Researchers were guided by photographic and textual examples of portion sizes, which they could use as prompts over the telephone. Participants’ food intake was entered into weighed intake software programme (WISP) dietary analysis software (version 4.0; Tinuviel Software, www.tinuvielsoftware.co.uk/wisp4.htm; accessed 1 May 2018)115 and analysed for energy intake, macronutrients and fibre. The 24-hour dietary recall measure was found to be a poor method of data collection at baseline and, therefore, it was not used at follow-up. Full details of this decision are provided in Chapter 6.

Secondary outcomes

Waist circumference was measured using a 2-metre flexible tape measure with a buckle around the mid-point between the iliac crest and the inferior margin of the lower rib. Hip circumference was measured around the widest point of the buttocks. Measurements were recorded twice in centimetres to one decimal point (e.g. 95.2 cm). A third measure was taken if the difference was > 0.5 cm.

Health-related quality of life was measured using the five-item EuroQol-5 Dimensions, three-level version (EQ-5D-3L), questionnaire and quality-of-life thermometer. 100 This measure is used frequently in health-related research to explore five dimensions: mobility, self-care, usual activities, pain and anxiety/depression. An additional measure of capability well-being was carried out using the ICEpop CAPability measure for Adults (ICECAP-A)101 scale. This is a new scale which, compared with the EQ-5D-3L, explores less clinically related changes in quality of life over four dimensions: feeling settled and secure, being independent, achievement and progress, and enjoyment and pleasure. This was considered an appropriate measure in our population owing to the potentially large range of participant characteristics. Mental health was measured using the General Health Questionnaire – 12 items (GHQ-12),102 a validated and frequently used 12-item self-report questionnaire. We gathered data on NHS resource use and participant-borne costs using a specially designed resource questionnaire. These data helped us establish key cost drivers of the intervention.

Questionnaires assessing potential mediators of change were used to explore their feasibility in a future trial. These included the Exercise and Eating Habits Social Support Scales (social support),103 the Weight104 and Exercise Efficacy Lifestyle Scales (self-efficacy)105,106 and the Treatment Self-Regulation Questionnaire (motivation). 107

At 12 months we administered the USE questionnaire86 to assess the usefulness, ease of use, ease of learning and satisfaction with the app and website. We also asked participants to complete the Heaviness of Smoking Index (HSI)108 and Alcohol Use Disorders Identification Test (AUDIT-C)109 questionnaires at 12 months, using this opportunity to assess the feasibility of additional questionnaires for data collection. These may be helpful in identifying other potential lifestyle changes made by participants in a future trial related to ‘spillover’ effects of the intervention.

Quantitative process data

Qualitative process data

Intervention costs

Intervention costs were identified and measured for the development of the intervention. The cost of the design of the mobile application and maintenance for the purposes of the feasibility study are reported.

Health, social care and personal resource use

All resource use data were collected from participants with a specially designed resource use questionnaire (see Report Supplementary Material 8 and 9). This collected information on (1) primary care services provided in the NHS (e.g. GP visits, practical nurse, dietitian); (2) secondary care services provided in the NHS [e.g. accident and emergency (A&E) attendances, hospital stay, visit by consultant]; (3) and personal costs (e.g. household income spent on different food and drinks items, including groceries, alcohol, takeaway food, meals out and lifestyle activities). Medication use at baseline and follow-up was also recorded. Mean group costs were calculated by attaching the unit costs to frequency of resource use per group participant. Medians and ranges for resource use quantities and costs are reported.

Reference costs

Reference costs for items of resource use were identified from readily available UK reference sources including the Personal Social Services Research Unit reference cost book. 117

Health economics outcomes

The economic analysis assessed the feasibility of using the EQ-5D-3L instrument100 and the ICECAP-A101 instrument as a means of capturing any short-term effects on health-related quality of life or capability well-being. This involved exploring data completeness and response rates. Participant responses were converted to a utility score using the value set elicited from UK general population. The EQ-5D-3L is the preferred measure of health-related quality of life in adults by NICE technology appraisal. 118 The ICECAP-A is a complementary measure of capability well-being.

The economic evaluation adhered to guidelines for good economic evaluation practice as outlined by Gold et al. 119 Specific guidance was also sought from the NICE economic evaluation public health reference case120 as it was anticipated that this intervention was likely to have an impact on costs and outcomes beyond the NHS and Personal Social Services and thus require a broader public sector evaluative perspective.

Baseline characteristics

These were summarised overall and by randomised group. Participant characteristics were summarised in relation to sociodemographic, lifestyle, occupational, health status and quality-of-life variables.

Feasibility measures

These were the primary focus of the analysis. Study-related data, including recruitment, randomisation and retention at 12-month follow-up, were reported overall and by randomised group, with 95% CIs. The association between baseline factors and follow-up was assessed using logistic regression, with follow-up (yes/no) as the response variable. Intervention-related data, including use of the intervention, was summarised for the intervention group, overall and in relation to selected baseline characteristics. The availability and utility of data relating to use of the app and website was explored, and a range of summary measures was presented in the final statistical outputs.

Exploratory outcomes

These were summarised overall and by randomised group, and compared using linear regression models, with randomised group, the baseline measurement of the outcome, age and gender as predictor variables. Regression models also adjusted for the minimisation factors. The residuals from each regression model were assessed for normality. When necessary, the outcome measure (at follow-up and at baseline) was transformed to improve model fit. All analyses were conducted under intention-to-treat principles and complete-case analysis used, unless > 20% of cases were lost as a result of missing data, in which case multiple imputation was also performed. These analyses were exploratory and underpowered, so no formal hypothesis testing was performed. P-values are presented for descriptive purposes as a guide to the interpretation of the magnitude of reported associations. Effect sizes were reported in line with CONSORT guidelines for reporting feasibility and pilot studies. 124

Process evaluation measures

These were descriptively analysed to summarise use of the app and website. A per-protocol analysis was conducted using simple proxies for adherence (e.g. website login) to explore the treatment effect associated with adherence.

Health economics cost data

These were analysed as follows. Resource use data were summarised and described using mean values and variation around these estimates. Key fixed and variable costs of developing the intervention were described and summarised. EQ-5D-3L and ICECAP-A outcome data were reported by within-attribute response rates, mean values and associated variance. Within-trial economic analyses were performed using Stata^® 12.0 (StataCorp, College Station, TX, USA).

Recruitment

One hundred and eighty-eight individuals were screened for inclusion in the study, of whom 156 were identified as eligible. A total of 109 participants (70% of those eligible) proceeded to be randomised between 22 April 2016 and 22 October 2016 (Figure 5). This was slightly short of the target of 120 participants randomised. Of the 109 participants, 73 participants were allocated to the intervention group and 36 participants were allocated to the control group. Figure 6 shows the cumulative numbers randomised over time during this period. Although initially a slow start, the recruitment rate continued in line with or quicker than target, which is positive sign for a future trial. No individual signed up as both a participant and a helper.

FIGURE 5.

The CONSORT flow diagram for the HelpMeDoIt! study.

FIGURE 6.

Recruitment rate in comparison with target rate (dashed line) over 6 months.

Baseline characteristics

Key baseline characteristics of the randomised population are reported in Tables 10–12 (see Appendix 8, Tables 49–65, for additional baseline summaries). Overall, more women (69.7%, n = 76) than men were recruited, and the average age was 47 years [standard deviation (SD) 10.7 years]. The average BMI was 37.6 kg/m² (SD 5.9 kg/m²) and just over one-quarter were currently attending a weight loss group. We recruited across the spectrum of levels of deprivation, with over one-third being from the highest quintile of socioeconomic deprivation. Most participants were employed and the majority were married. Over 60% had received higher education. Participants’ daily moderate to vigorous activity was a mean of 45 minutes per day (SD 32.9 minutes per day).

Randomised groups were generally well matched, although intervention group participants were more likely to have higher levels of education. They were also more likely to have higher levels of physical activity (measured objectively), as well as more variability between participants. They also had more sedentary time. Slightly more people in the control group were attending a weight loss group and those in the control group were more likely to weigh themselves regularly (see Appendix 8, Table 50).

Physical activity

Table 15 summarises physical activity measures derived from the 7-day physical activity recall, and from accelerometers, in the randomised groups, along with estimated between-group differences, with 95% CIs and p-values, derived from linear regression models of 12-month outcomes, with randomised group, baseline score, age, gender and BMI of ≥ 40 kg/m² as predictor variables.

For each estimated intervention effect estimate, the standardised effect size is reported with a 95% CI. The original plan had been to derive this effect size estimate by dividing the intervention effect estimate and confidence limits on the original scale by the standard deviation of the outcome measure at baseline in the total study population. However, it was recognised that for some measures, there was considerably more variability between individuals in baseline scores (or follow-up scores) than there was between individuals in the change from baseline. For an individual, a change in outcome equal to the standard deviation in the whole population might be an unachievable goal, but a change in outcome equal to the standard deviation of the distribution of within-person changes may be much more attainable, so it was felt that standardising between-group differences relative to the pooled standard deviation of changes from baseline would be more relevant.

As part of the modelling process, outcomes were initially analysed on their original scale. The residuals from each regression model were visually assessed for normality. For those measures where substantial non-normality was observed, data transformations (at follow-up and at baseline) were attempted to improve model fit; if no suitable transformation could be found, then changes from baseline to follow-up are compared between groups using the Wilcoxon–Mann–Whitney test. See Appendix 8, Tables 105 and 106, for additional summaries.

Only 39 participants (36% of those randomised, 46% of those followed up at 12 months) provided valid activity monitor data at both baseline and 12 months, in comparison with 81 (74% of those randomised, 96% of those followed up) who provided self-report physical activity data at both time points.

Both groups showed reductions in objectively measured physical activity (moderate to vigorous physical activity and step counts), but the intervention group had higher mean values, and showed greater variation, at each time point. Adjusted analyses suggest effect sizes of around 0.4, but unadjusted analyses give effect size estimates close to zero. Similar reductions in physical activity in both groups were seen in self-reported energy expenditure data, with intervention effect estimates close to zero. The intervention group had lower objectively measured sedentary times at 12 months, with 50–60 fewer minutes per day spent in sedentary activities relative to the control group. This represents an effect size estimate of around 0.7. We also examined whether self-report physical activity differed between those who did and those who did not provide valid accelerometry data (Table 16). There was no difference.

The results seem to offer evidence that self-report physical activity was no different between those who did and those who did not provide accelerometry data. This increases confidence in the differences found between the groups in the accelerometry data.

Diet

Table 17 summarises dietary outcome measures derived from the DINE questionnaire. These have been analysed in the same way as the physical activity data. Additional data summaries can be found in Appendix 8, Tables 107 and 108.

Dietary scores were available for 81 (74% of those randomised, 96% of those followed up) participants at baseline and 12 months. These generally improved or remained stable in both groups between baseline and 12 months, with the exception of the fibre score in the intervention group. Both groups achieved an increase in healthy eating score (difference between fibre score and fat score), although this was greater in the control group, with an effect size in favour of control. The intervention appeared most promising in terms of the unsaturated fat score, with an effect size of approximately 0.45. Fruit and vegetable scores were slightly higher in the intervention group, but fizzy drink and sugar scores were skewed towards zero and changed little in both groups during the study, and therefore show little promise as outcomes for a future study.

Body mass index

Table 18 summarises BMI data, which were analysed in the same way as the other primary outcomes (see also Appendix 8, Table 109). Data were available for 82 participants (75% of those randomised, 98% of those followed up). Both groups showed similar reductions in BMI during the study of approximately 1 kg/m².

Table 19 presents thresholds of weight loss for each group. A greater proportion of participants in the control group gained weight than in the intervention group (47% vs. 34%). A greater proportion of participants in the intervention group lost < 10 kg than in the control group (52% vs. 31%). However, weight loss of > 10 kg was more frequent in the control group than in the intervention group (22% vs. 14%).

Intervention costs

The intervention cost comprised the cost of developing a mobile app and website and the cost of maintaining and updating the app. Table 27 shows the cost of the intervention, including app development and maintenance, alongside the research costs. The cost per participant for the current study was high, at £740 (see Table 27: calculated by the fixed cost of the study plus variable cost of the study divided between 109 participants); however, these costs involved the upfront cost of intervention and software development. Now that the app and website are developed, a future trial would simply need some refinement based on the feasibility results (£10,000), ongoing costs for hosting and support (£10,000), and costs of the study (variable depending on sample size). This would mean a low cost per participant in a future trial, at approximately £370 per participant for a sample of 400 or £220 per participant for a sample of 1000. If rolled out ‘in the real world’, only hosting (£10,000) and maintenance costs (£10,000) would be required, resulting in a low cost, for example, of £2 per participant per 10,000 users.

Health and social resource use

The most frequently used health and social resources within the 3-month period prior to data collection were visits to GP, practice nurse and physiotherapy (see Appendix 9). The data identified individual participants with high attendances at GP, practice nurse and physiotherapy; these outliers giving rise to right-skewed data are a typical pattern occurring in resource use data. The mean cost of baseline health service resource use for the control and intervention groups at baseline was £285 (SD £571) and £80 (SD £119), respectively. The mean cost of follow-up health service resource use for the control and intervention groups was £553 (SD £1554) and £152 (SD £224), respectively. No formal statistical tests of the difference in costs were conducted owing to lack of power.

Data revealed that the HelpMeDoIt! resource use data collection instrument identified and measured resource use items across a range of health services in both arms including GP, nurse, physiotherapist, hospital consultants for various specialties, community psychiatric nurse, A&E and hospital inpatient stay. NHS weight management services were also used by both control and intervention arm participants at follow-up only.

Personal resource use

The total weekly cost of participant food and drink at baseline for the control and intervention groups was £98.93 and £104.75, respectively (see Appendix 9). Overall, grocery costs were the highest component of weekly costs at baseline and follow-up (64% and 58% at baseline for the control and intervention groups, respectively; 67% and 61% at follow-up for the control and intervention groups, respectively). Meals out constituted between 20–22% and 18–24% of weekly food expenditure at baseline and follow-up for control and intervention, respectively. Alcohol spend was the smallest component of weekly food and drink cost, at between 5% and 8% at baseline and follow-up, in both arms. Overall, these weekly food and drink data collection questions appeared feasible for gathering information on food and drink spend.

The mean spend on lifestyle activities (including slimming clubs, health clubs, gyms, swimming pools and exercise classes) in the 3 months prior to baseline and follow-up was £43.29 and £30.08 at baseline for the control and intervention groups, respectively. At follow-up this spend was £51.46 and £55.23 in the control and intervention groups, respectively. These lifestyle costs exhibited the same pattern of right skewness as health services data, including a small number of high values. This method of directly collecting lifestyle spend, pre-empted by a closed-ended yes/no question, proved acceptable to participants.

EuroQoL measure of quality of life

Responses to each level of all domains of the EQ-5D-3L are shown in Table 28. The majority of responses are in the level 1 (no problems) category across all five attributes, with the exception of the attributes ‘pain and discomfort’ and ‘anxiety’. These two are the only attributes that report higher responses in values in the level 2 and 3 categories (some/extreme problems). The EQ-5D-3L health index and visual analogue (VAS) values for each treatment group at baseline and follow-up are shown in Table 29. Table 29 shows that the EQ-5D-3L index values from baseline to follow-up are in the range 0.67–0.79 across control and intervention groups. These index and VAS values are slightly lower than the population norms reported for this age group. 126

ICECAP-A measure of capability well-being

The ICECAP-A instrument feasibly identified and measured responses across all levels of capability well-being across the control and intervention groups. The highest percentage of responses typically lie in level 3, with few responses in level 1 (see Appendix 9). These results reveal that the ICECAP-A instrument is able to distinguish between different levels of self-reported capability well-being in this population group. No data were collected for the domain ‘love, friendship and support’. This question had been omitted from the questionnaire in error.

Theme 1: contextual factors

Participants described contextual factors that influenced their behaviour with regard to losing weight, physical activity and diet. These including the importance of social influences and their own traits and preferences. Life context was seen as influencing both past and current experience with weight loss.

Theme 2: software and barriers to use

Theme 3: mechanisms of change

Theme 4: impact of the HelpMeDoIt! study

Theme 5: study- and intervention-related insights

Participants were also asked about their views of the study evaluation methods.

Theme 6: the helper experience

Theme 7: the HelpMeDoIt! software and barriers to use

Theme 8: helpers’ impact on participants

Theme 9: social support

Theme 10: the HelpMeDoIt! study and intervention

Contextual factors

Contextual factors influencing the effect of the intervention include (1) the context in which the intervention itself takes place and (2) contextual factors that have either a negative or a positive impact on various pathways of the intervention (see Figure 4).

The HelpMeDoIt! intervention was designed to be implemented with minimal contact with the study team or health professionals, so it was delivered as an app and accompanying information website. The HelpMeDoIt! app was designed for use on both the Apple (Apple, Inc., Cupertino, CA, USA) and the Android (Google, Google Inc., Mountain View, CA, USA) operating system. Slightly more than half of participants used Android software (n = 38, 52%) on numerous different versions of smartphones (e.g. Samsung, LG, Huawei), and the remainder used Apple operating systems on iPhone devices (Apple Inc., Cupertino, CA, USA) (n = 35, 48%). The HelpMeDoIt! website was accessible from smartphones, desktop computers or tablets. Of those who accessed the website (n = 55), the majority of participants used a desktop computer (n = 35, 64.1%), followed by a smartphone (n = 17, 30.6%) and a tablet device (n = 3, 5.4%).

Contextual factors influencing the intervention have already been described in Chapter 5. In brief, these included (1) participants’ previous experience of weight loss and relapse; (2) significant life change, such as new job or moving house; (3) the influence of personality traits and mood; (4) influence and/or motivation of health issues; (5) social and group norms; and (6) multiple barriers to lifestyle change, for example environmental, participant-related, helper-related and/or social context barriers.

Fidelity of delivery of the intervention

Although the app was tested for a 3-month period using a testing group, some technical issues arose when it was initially launched via the app stores. The testing phase used a third-party platform allowing the app to be test-run before it was submitted to the app stores for release. This helped identify numerous bugs, which the software company resolved. There was no built-in testing phase after the app was available for download from the app stores. This unfortunately resulted in many technical issues being detected after the app had ‘gone live’. The majority of technical problems were associated with (1) the app crashing or working slowly because database information was downloaded to phones on each login; (2) elements of the key features not displaying correctly, which had an impact on how effectively participants could operate the app; and (3) the frequent emergence of new software bugs as a result of ‘app updates’ intended to solve previous issues. The software company had difficulty resolving software bugs for Android phones, which, unlike iPhones, are designed by numerous companies, using multiple combinations of operation systems. After several months of trying to stabilise the app, the software company carried out a ‘technical rebuild’. This did not change the content of the app but resolved the key issues and improved the overall quality of the app.

These issues had an impact on initial fidelity of the intervention being delivered as some participants were not able to access the key intervention components of the intervention via the app (i.e. goal-setting, self-monitoring and social support) and some of the app features did not work as intended. However, qualitative findings demonstrated that participants received the social support elements outside the app (e.g. meeting helpers face to face), which was in keeping with our programme theory and was not always because of technical issues but sometimes was participant and/or helper preference. Qualitative findings also indicated that some helpers did not access the website, which contained guidance on being a good helper, so they may not have delivered the social support elements as we had intended, for example using language that supported their friend’s autonomy.

Facilitators that might promote future fidelity of the intervention delivery include (1) resolving software bugs using both a ‘pre-live’ and a ‘post-live’ testing phase; (2) further streamlining the app and website based on findings from this feasibility study; (3) installation of the app on participants’ phones and a run-through with field workers at the baseline face-to-face visit (relevant for the study but not for wider roll-out); and (4) ensuring better signposting to the website.

Exposure to the intervention

As part of this feasibility study, we were interested in exploring how the intervention was used by participants and their helpers. One way of measuring this is through data use statistics; these were gathered and analysed to explore which features of the app and website were used, and how often they were used. The analysis of app and website use provided us with some meaningful insights into how the intervention was used and what elements of the intervention were potentially effective. A summary of app/website use is presented for participants in Table 34 and for helpers in Table 35. Engagement with the intervention was also explored in the qualitative work (see Chapter 5).

Participant engagement with the HelpMeDoIt! app

Participant engagement with the app varied widely. Nineteen participants (26%) did not use the app at all. We assessed engagement with the app in comparison with 2017 use statistics for commercial apps, which reported that 21% of users will engage with an app only once. 94 In our study, of the 54 (74%) participants who downloaded the app, six (11%) used the app only once, 48 (89%) used it twice or more, 38 (70%) used it three times or more, and six (11%) people used it ≥ 30 times. Thus, there was greater engagement with the HelpMeDoIt! app than the current standard within the app industry.

Data use statistics explored how often participants engaged with key features of the app (see Table 34). After logging in to the app, the top five activities were (1) creating custom goals, (2) viewing the main dashboard, (3) completing goals, (4) updating goals and (5) viewing progress charts. Of the 957 goals created by participants, 94% were custom goals, compared with 6% template goals, and 61% were recorded as completed.

A key feature of the intervention was the nomination by participants of one or more helpers from their social network. In total, 19 participants (36%) invited at least one helper via the app. In total, 45 helpers were invited, ranging from one to eight helpers per participant. Of the 45 invited helpers, 25 (56%) accepted the invitation and downloaded the app (helper use is reported below). Table 34 also demonstrates that participants used the social support elements of the app: the ‘send a smile’ feature; and ‘contact helper’ feature. Qualitative findings demonstrated that participants and helpers also interacted outside the app, either face to face or with a telephone call, text message or e-mail (see Chapter 5).

Participant engagement with the HelpMeDoIt! website

Of the 73 participants in the intervention group, 18 (25%) participants did not visit the website (similar to the number of participants who did not use the app). Among those who did use the website (n = 55, 75%) the number of visits ranged from 1 to 12 individual sessions, each session lasting 18 seconds to 44 minutes, with a total of 1–45 individual web pages viewed (see Table 34). Qualitative findings suggested that some participants were unclear about the purpose of the website, forgot about the website or did not find it helpful. More effective signposting of the website and the help that it can provide is an important consideration for a future trial. Although website use was limited, participants and helpers may have gathered sufficient information to proceed with the intervention from one or two visits.

Helper engagement with the HelpMeDoIt! app

App use by helpers is presented in Table 35. Forty-five individuals were invited as helpers. Twenty-eight (62%) accepted the invitation, of whom 25 (56% of nominated helpers) used the app at least once. Overall engagement was assessed against similar criteria to those used for participants: eight helpers (29% of accepted helpers) used the app only once; 17 (61%) used it twice or more; 10 (36%) used it three times or more; and only two (7%) used it 10 times or more. In total, there were 122 logins by helpers (ranging from 1 to 48). The top three features used by helpers were sending smiles to participants, viewing participant goals and liking participant goals. Qualitative findings suggested that helpers were uncertain how to help the participant using the app, with many providing support outside the app with face-to-face chats, text messages or telephone calls (see Chapter 5). The level of helper engagement with the social support element was therefore higher than indicated by the app use data.

Helper engagement with the HelpMeDoIt! website

Of the 28 helpers who accepted a participant invitation, 19 (68%) used the website at least once, and nine (32%) used the website twice. No helper visited the website more than twice. The number of pages viewed by helpers ranged from one to eight, with a session duration of 1.5 minutes to 27 minutes (see Table 35). Similar to insights from participants, qualitative findings showed that helpers were unaware either that a website was available or of how it could help them. Signposting the website more effectively to helpers should be considered for a future trial. A consideration is that helpers may not need to revisit the website often after reading the ‘helper guidance’.

Characteristics of app and website use

Table 36 shows summaries of selected measures of app use in relation to participant gender. Complete summary tables are provided in Appendix 8. Women were higher users of the app than men, but app use by helpers did not vary between helpers of male and female participants. Apart from gender, there were no other associations between participant baseline characteristics and app use. Participant age, Scottish Index of Multiple Deprivation (SIMD), BMI, physical activity and diet showed no correlation with the number of logins by participants or their helpers, or the number of goals set by the participants.

Although participants were informed that this was an option, no participants enquired about the potential to be both a participant and a helper for someone else in their social network.

Potential mediating effects of intervention use

Within the intervention group, the potential mediating effects of measures of app/website use were assessed by testing the correlation between each measure of app/website use and each primary outcome measure (Table 37).

Those who made greater use of the app, as measured by the number of logins and goals created, showed greater reductions in BMI. Objectively measured physical activity (step counts and time spent in sedentary activities) was correlated with the number of logins, more than with the number of goals created. Healthy eating scores from the DINE questionnaire were correlated with increased app use, as were fruit and vegetable scores, particularly with the number of goals created. There was also a positive correlation between app use and DINE unsaturated fat scores. Helpers’ use of the app demonstrated less association with participants’ outcomes, with only fruit and vegetable scores showing some signs of a (counter-intuitive) negative association with the number of helper logins.

It is important to note that, although these identified associations could indicate mediating effects, these results could also be found as a result of reverse causality or be artefacts of another predictor of success; for example, people who are losing weight will maintain engagement with HelpMeDoIt! as it is going well, but they may have been successful anyway.

Dose–response relationship: illustrative case studies

Given the range of engagement with the app, and the possibility that for some individuals the intervention might be particularly helpful, we explored use characteristics and outcomes for the six (8%) intervention participants who used the app most frequently (range 30–408 logins). All of these participants lost weight (between 3% and 19% of their starting weight) and that weight loss typically increased as app use increased. In addition to having the largest number of logins, these six participants also had the highest engagement with the three key features of the intervention: goal-setting, self-monitoring and helper interaction. Interestingly, the participant who had greatest success with weight loss (a clinically significant weight loss of 19% body weight) was the individual who used the app most frequently, completed the greatest number of goals, had the greatest number of helpers using the app and had the greatest number of helper interactions. However, while it may be the case that using the app leads to more weight loss, it could also be that individuals may use the app more because they are being successful and losing weight.

Participant insights on Usability, Satisfaction and Ease of Use questionnaire

Overall, the response rate for the 30 individual USE questions ranged from 40% to 49% (n = 21–24 of the intervention group at follow-up). This response rate was lower than that for other measures in the questionnaire and was probably due to participants who had never, or rarely, used the app/website feeling that they could not provide informed responses. For example, it was difficult for participants to respond to ‘I can use it without written instructions’ if they had never used the app/website or had logged in only once. A ‘not applicable’ or ‘don’t know’ option should be included for a future study to improve data completeness. It is also possible that participants who did not enjoy using the app did not complete all of the USE questions, potentially skewing the data.

The USE questionnaire presented summed scores over four categories: (1) ease of use, (2) ease of learning, (3) usefulness and (4) satisfaction. Lower scores indicate more positive results. Among those participants who did respond, the mean ease of use score was 52.1 (range 14–77 from a potential range of 11–77), the mean ease of learning score was 15.2 (range 4–28 from a potential score range of 4–28), the mean usefulness score was 41.9 (range 8–56 from a potential range of 7–56) and the mean satisfaction score was 36.8 (range 7–49 from a potential range of 7–49). These findings suggest that, although the majority of participants found the app/website moderately easy to use and quite easy to learn, the majority did not find the app/website useful and were moderately unsatisfied. Qualitative findings suggest that the reasons for dissatisfaction and lack of usefulness were related to the technical issues experienced with the app. Importantly, better scores were demonstrated in participants who started using the app after the technical issues had been resolved. For example, the first five users of the app (in the presence of technical issues) reported a mean total USE score of 146 (range 113–210), compared with the final five users of the app (in the absence of technical issues), who had an improved mean total USE score of 136 (range 108–173). In addition, participants who used the app most frequently, and experienced no technical issues, provided the most positive scores. This demonstrates that when the app operated effectively it was perceived positively by participants.

Reach

Participant demographics were presented in Chapter 4. As is frequently seen in weight loss studies, most participants were female (n = 76, 69.7%). 91 In terms of ethnicity, the sample was representative of people living in Scotland, with 97 (89.8%) being white British and 11 (10.2%) being of a minority ethnic or another background. The overall sample was also representative of Scotland’s relationship between obesity and socioeconomic status, with the majority of participants (n = 57, 57.6%) living in an area of high deprivation (Scottish Index of Multiple Deprivation level 1 and 2). Participants from a wide age range (25–68 years) were recruited, giving us insight into the feasibility of using an app and website intervention for different age groups. However, we had to approach (via primary care) or reach (via advertising) many people in order to recruit our sample, so it is likely to be a somewhat narrow and motivated group, which is not unexpected in a study on weight loss.

Qualitative and quantitative data were collected to measure the extent to which the intervention reached individuals other than participants, as reflected in the programme theory. Findings from participant and helper interviews demonstrated that some helpers made positive changes to their lifestyle in response to their involvement in the HelpMeDoIt! study. This included changes to physical activity and diet (see Chapter 5). Questionnaire data at follow-up also supported this finding, with 12 participants (14%) reporting their helpers making healthier food choices, 14 participants (17%) reporting their helpers increasing their physical activity, and six participants (7%) reporting that their helpers had successfully lost weight. This is an important consideration for the potential impact of this intervention in the future because if the intervention has a spillover effect, thus reaching a broader group of people, the potential impact could be positive even if only some of the individuals use the intervention as intended.

Programme theory and logic model

Chapter 2 previously described how the HelpMeDoIt! intervention was guided by a programme theory and logic model. Findings from stage 1 helped inform the second iteration of the logic model from version 1.0 to version 2.0 (see Chapter 2). In stage 2 we further explored the programme theory and the version 2.0 logic model using use statistics and qualitative data from participant and helper interviews. In addition to feedback from participants and helpers, observations by the study team and the software company helped refine a comprehensive programme theory for the HelpMeDoIt! intervention. The aim of this section is to summarise new contextual factors for consideration, how elements from the key qualitative themes influenced each other, and how they mapped onto the proposed programme theory and logic model.

Contextual factors identified within the HelpMeDoIt! programme theory

Contextual factors influenced whether or not the key mechanisms of change were successfully implemented. Various categories were identified in which contextual factors could negatively or positively influence the impact of the intervention. Categories included issues related to (1) available technology (i.e. make and model of smartphone), (2) participants themselves, (3) their helpers, (4) the environment and (5) broader contextual issues (previously described in Chapter 5). Many factors had already been considered in the early stages of development, such as aiming to develop an app/website that was easy to use and engaging, and being aware that participants would have varied access to physical activity opportunities. As the study progressed, we gained greater insight into additional contextual issues. Many of these focused on the participant–helper relationship and require consideration for a future trial. These included:

• participants feeling uncomfortable asking helpers for support or feeling that their weight goals were a burden to their helpers

• some participants feeling socially isolated and not having access to many helpers

• participants feeling embarrassed to share their weight and eating habits with their helper or to start using the app again after a long period of non-use

• invited helpers’ motivation to provide support being poorly understood (ethical constraints meant the study team could not contact helpers directly for feedback; helpers who were interviewed were contacted via the participant)

• participants’ ability to effectively and confidently ask for support, which influenced the clarity, or lack of clarity, that helpers felt towards their role.

Mapping qualitative themes to the HelpMeDoIt! programme theory and logic model

Table 38 summarises the ‘mechanisms of change’ and ‘intermediate outcomes’ from the version 2.0 logic model (see Figure 4) and identifies which of them were strongly supported by qualitative findings (i.e. discussed by the majority of participants). Theme numbers in bold strongly supported the identified mediator as a mechanism of change (e.g. increased/improved social support is strongly supported by qualitative theme number 3.3, ‘social support’). Theme numbers in italics suggest that the identified mediator is a potential mechanism of change (i.e. it was discussed by some participants but less often). However, further exploration in a future trial is needed (e.g. increased problem-solving is identified as a potential mediator by qualitative themes number 3.1 ‘goal-setting and self-monitoring’ and number 4.1 ‘change in behaviour’).

Exploring the qualitative findings in this way demonstrated that two of the four intermediate outcomes proposed by the logic model were strongly supported by the qualitative findings: (1) improved social support and (2) healthy habit formation. Insufficient data were provided by participant/helper interviews to support the other two intermediate outcomes, namely improved self-efficacy and improved self-image or self-esteem, as integral processes in HelpMeDoIt!. Although self-esteem and self-efficacy are supported by the evidence base,22 they may not be intermediate outcomes in the HelpMeDoIt! intervention. They should, however, be included in the programme theory for further study in a future trial.

Eight out of the 11 proposed mediators of change above were found from the data to be key processes in the HelpMeDoIt! intervention. These were increased social support, increased engagement with helpers via the app, increased interaction with helpers not via the app, reflecting on and setting ongoing goals, increased action-planning, increased self-monitoring, increased skills and knowledge, and increased motivation. Four of these eight mediators emerged from the data as the strongest elements of the HelpMeDoIt! intervention: (1) increased motivation, (2) increased social support, (3) increased goal-setting and (4) self-monitoring.

The revised HelpMeDoIt! programme theory and logic model

The revised version 3.0 programme theory and logic model incorporating the above findings is presented in Figure 10. Version 3.0 more clearly identifies motivation, social support, goal-setting and self-monitoring as the key mediators of change within the HelpMeDoIt! programme theory. These have been grouped together and given hierarchical priority in the logic model (illustrated by the use of solid lines and deeper colour). Improved social support and healthy habit formation were identified as the key intermediate outcomes, and have also been given hierarchical priority in the logic model (using solid lines and deeper colour). Other mediators and intermediate outcomes, which were not strongly supported by the data and require further exploration (e.g. increased self-efficacy), are illustrated by the use of dashed lines and lighter colour. Additional considerations were added to appropriate categories within the pre-conditions and contextual factors section, for example complementary use of other lifestyle apps under ‘technology’. Any future evaluation of HelpMeDoIt! will use the version 3.0 programme theory and logic model.

FIGURE 10.

Version 3.0 of the HelpMeDoIt! programme theory and logic model.

Recruitment and retention

Contamination and the control group

As this was a feasibility study, it was important to explore how the HelpMeDoIt! intervention would be used in everyday life. We gathered data on any other lifestyle-related activities that participants were engaged in. Of the 109 participants, 29 (26.9%) reported using another service either at the time or within the 3 months prior to the study start date (23.6% intervention vs. 33.3% control). These included Scottish Slimmers™ (Aberdeen, UK), Weight Watchers^® (Maidenhead, UK) and the NHS Weight Management Service (NHS Greater Glasgow and Clyde). Some of these activities provide similar elements to the HelpMeDoIt! intervention, such as advice on how to set achievable goals for healthy eating, and tools for self-monitoring. However, the HelpMeDoIt! intervention was designed as a complementary intervention that could enhance the activities that individuals were already engaged in, primarily by harnessing greater social support. At both baseline and follow-up, three participants had taken weight loss medication in the previous 12 months compared with five in the intervention group, and a similar proportion of control and intervention participants paid for lifestyle services (e.g. gym memberships) over the duration of the study [45.0% control (n = 11); 48.8% intervention (n = 40)]. As the numbers of people accessing these services were similar between the intervention and control groups, the risk of significant contamination bias is low. Furthermore, no participants in the control reported that they had seen or had access to the intervention content. No participant in the control group acted as a helper for a participant in the intervention group, and no nominated helper was enrolled as a participant. At the end of the study, five (14%) control participants took up the offer to use the app/website.

General insights from the study team

The study team reflected on the research processes as part of the study aim to examine the feasibility of delivering a larger study. Insights were gathered from members of the study team, support staff and field workers. Several issues were identified that need consideration for any future trial. These include the following.

Recommendations to improve the intervention

Finally, based on the findings of the process evaluation, a number of changes should be made to the app and website in anticipation of a future trial. These are summarised below.

Objective 1: to develop an internet and web-based intervention that enables participants to set and monitor goals and facilitate effective social support

Phase 1 of the study involved designing the app and website with comprehensive consultation with potential users. In this phase we also completed initial testing of the website and app to assess their usability and acceptability. We combined four approaches to intervention development, which included an overarching intervention development framework (the 6SQuID model)74 and a review of the literature to identify relevant theories and behaviour change techniques that could be employed,16 as well as digital health-based approaches (Person-Based Approach and BIT model). 75,76 User input was at the centre of the whole development process.

Initial development of the intervention was completed in tandem with a user development panel, the study team and the software company. This was an iterative process; as different aspects were developed, they were discussed in the development panel and updated according to what was agreed. In addition, a testing group of users helped with the initial evaluation of early-stage versions of the app and website. Although users were asked to provide input and feedback from a ‘helper’ perspective, this overall phase lacked input from individuals who formally identified as a helper. This is a consideration for future work. Once the beta version of the app and website had been developed, it was tested fully by both groups, and feedback was given in focus groups or individual interviews. The majority of the feedback was positive and we updated both the app and the website in anticipation of the feasibility trial. An expert heuristic evaluation was also completed at this point, which was very positive. During this phase we also further developed the HelpMeDoIt! programme theory and logic model, which incorporated behaviour change techniques and relevant theory, and addressed multiple contextual factors.

Objectives 2 and 3: to investigate recruitment and retention; to explore the potential of the intervention to reach traditionally ‘hard to reach’ groups (e.g. lower socioeconomic groups)

The trial methods and design were acceptable and feasible to implement. Although the recruitment rate was slow at first, it then was in line with or slightly exceeded the target rate. The average number of participants recruited per month was 17 (minimum 6 and maximum 27) and we recruited and successfully randomised 109 participants in the specified time frame (progression criterion 1). This was slightly short of our original target of 120. However, the trial team and steering committee felt that this number was sufficient to answer the research questions, so recruitment was stopped at 109. Participants also found randomisation acceptable, and none refused (progression criterion 2). This study has shown that it is feasible to recruit participants and to follow them up. There was an imbalance in follow-up rates between the randomised groups, which could introduce bias in a larger study if it is not addressed. A number of key learning points related to recruitment will inform a future study, and these are discussed in Chapter 7, Limitations of the study.

The sample was broadly representative of the target population; however, as is often seen in studies of interventions for obesity, more women than men were recruited. 91 The majority of men were recruited via online Gumtree adverts. The participants in the sample were relatively well educated, with 42% having degree-level education or higher. The age range was good, although we were not successful in recruiting younger people (i.e. those aged < 25 years). Participants were relatively active at baseline, with those in the intervention arm achieving an average of 50 minutes and those in the control arm an average of 36 minutes of moderate to vigorous physical activity per day. Standard cut-off points were used for the accelerometer data. 98 Participants may have increased their physical activity in response to using the activity monitor. 130

Recruiting and engaging participants from lower socioeconomic groups131 in research remains a challenge, and many current interventions widen inequalities. 132 However, we were particularly successful in recruiting participants from across the socioeconomic spectrum, with over one-third from the highest quintile of deprivation in Scotland and a further 21% from the next most deprived quintile. There were, however, slightly higher dropout rates among those from the most deprived group.

Baseline characteristics were broadly similar between the arms, although participants in the intervention arm were more likely to have higher levels of education and physical activity. Retention rates were good, with 84 (77%) participants followed up at 12 months (progression criterion 3). This included 14 participants who provided a minimum data set over the telephone. The split between intervention (11 of 52 participants, 21%) and control (3 of 32 participants, 9%) broadly reflected the 2 : 1 recruitment ratio. The waiting list control design, and the technical issues affecting intervention participants, may have affected the engagement of some participants, leading them to opt for the minimum data set. Those in the intervention group were slightly less likely to be followed up at 12 months. This may reflect either (1) a tendency for people who had not engaged with the intervention, or had experienced technical difficulties, to be less inclined to continue participating in the trial; or (2) the motivation of control participants to remain in the trial to receive the intervention. The participants least likely to complete follow-up data were younger, male, self-employed and deprived and had a higher BMI. Those currently attending a weight loss group were more likely to be followed up, as were those who complied with the baseline physical activity assessment by wearing a physical activity monitor. Both of these factors may represent markers of increased motivation to take part in the trial.

Objectives 2 and 4: to investigate the feasibility and acceptability of the intervention; to explore the barriers to and facilitators of implementing the intervention

Although engagement with the app was moderate for participants and low for helpers, the qualitative data indicated that most of the participants and helpers interviewed engaged with the concept of the intervention around mobilising social support to help with goal achievement. Helpers engaged with participants outside the app, and some participants sought social support from informal helpers who were not signed up to the study. This is a positive finding, considering that social support was the unique aspect of the HelpMeDoIt! intervention. Apart from some technical issues with the app that are described in Chapter 6, participants found the app and website acceptable and in the main were very positive about the intervention (progression criterion 4). The USE questionnaire data were not as positive as we had hoped; however, it is likely that the technical difficulties we experienced with the app, which were ultimately resolved, will have negatively influenced these responses. Participants who used the app most frequently, and after the technical issues had been resolved, provided more positive scores for usefulness, ease of use and satisfaction. This demonstrates that when the app operated effectively, it was perceived positively by participants. Interview data were more positive, but this may reflect interviews being conducted with participants who were potentially more motivated and engaged with the intervention. The qualitative data have provided a number of suggested refinements to improve the intervention (progression criterion 6). These would be implemented before delivery in a future study and are discussed in Chapters 5 and 6.

Adherence to the intervention was superior to that seen in general app use. 94 However, commercial apps lack a recruitment phase, as in this study. Of the 54 (74%) participants who downloaded the app, 89% (n = 48) used it twice or more and 70% (n = 38) used it three times or more (progression criterion 5). In the study logic model, this level of participant engagement with the app and website was seen as necessary in order to receive a ‘dose’ of aspects of the intervention, in particular to receive key information about the importance of monitoring, setting realistic goals and social support to help with making lifestyle changes. The central aspect of the intervention was the support from the individual participant’s social network. It was not regarded as key whether this was via the app or another method, for example seeing someone face to face, sending them a text message or telephoning them. The interviews indicated that, in many cases, even though the helpers did not use the app, they gave the participants support outside the app. In the logic model, the app, and indeed the intervention as a whole, was seen as a prompt to help mobilise people’s social networks to help them in their behaviour change goals. The intervention helps start the conversation by getting the need for help ‘on the radar’. The process evaluation data indicated that this was achieved. There was also some evidence of changes in helper behaviour as a result of being nominated to help the participant. This ‘spillover’ effect could potentially be an important mechanism of impact of the intervention.

A recent expert international workshop considered the issue of engagement and concluded that engagement is complex, has many different aspects and cannot be evaluated purely on the basis of how often a person uses an app. Members of the workshop differentiated between the digital elements of the intervention and the behaviour change elements and concluded that continued engagement with the digital intervention (app/website) is not always needed for behaviour change, as delivery of the behaviour change elements via the digital intervention could kick start a process of behaviour change of developing new skills and new habits. 95 This is particularly relevant to HelpMeDoIt!, where a brief interaction with the website or app can lead to the engagement of significant and sustained social support from existing, stable and durable social resources, the effects of which on resultant behaviour are not dependent on further app use.

The key barrier to implementing the intervention was the technical issues, described in detail in Chapters 5 and 6. Participants identified a number of other barriers to changing their behaviour, including various contextual factors such as time and money issues. They also described personal barriers, including willpower, motivation and a lack of routine. Some participants also described barriers to nominating helpers. Participants made a number of suggestions to facilitate use of the app, including receiving more reminders, receiving additional guidance on use and being able to choose what information to share on diet and weight.

The data suggest that the app seems to have worked particularly well for a minority and that the engagement rates are above industry norms. An intervention such as this has the very real potential to be cost-effective even if a relatively low percentage of the initial users engage with it repeatedly.

Objective 5: to assess the feasibility and acceptability of different outcome measures for diet and physical activity in this population

One of the key objectives of the study was to inform the choice of primary outcome in a future evaluation. Three potential primary outcomes were tested: BMI, physical activity and diet. Physical activity and diet were measured in two different ways: physical activity via a self-report measure and accelerometery, and diet by two different self-report measures. One of the criteria for selection for a future study was feasibility of the measures including data completeness (progression criterion 7). The measures were generally feasible to use, and qualitative data indicated that they were acceptable to participants and not burdensome. Data completeness was good for all of the measures apart from the accelerometry data. BMI data were collected in 98% of those who were followed up. The 7-day physical activity recall (PAR) data were collected for 96% of those followed up; however, valid accelerometry data (provided at baseline and follow-up) were available for only 46% of the sample. However, when these data were obtained, they provided the most robust evidence for an intervention effect in this trial. This measure required a greater level of commitment from study participants to obtain valid data for analysis. A wrist-worn accelerometer may have had better compliance if it had been possible to collect physical activity data in a less invasive way, for example via participants’ smartphones, throughout the follow-up period; this might have reduced the burden on participants, and provided objective physical activity data for those who did not engage with follow-up data collection. In addition, in other similar studies, strategies such as giving vouchers for returning the accelerometer have led to improved adherence and return rates, so vouchers combined with other methods could significantly improve return rates. 128,133 In a future trial, accelerometry data could be collected for a subsample to validate the self-report physical activity measure, where self-report data are collected before sending out accelerometers. Alternatively, accelerometry data could be collected for the full sample but not analysed and instead used to increase the accuracy of the self-report.

For the diet measures, the collection of diet data using the 24-hour multiple pass recall was not considered feasible because, although we managed to obtain data at baseline from 74% of the participants, only 31% of the questionnaires were completed properly. In addition, this outcome also required a large amount of resources in terms of data collection and processing and, because of these issues, was not used at follow-up. The other dietary measure, the DINE questionnaire, was feasible and was completed by 96% of participants at follow-up.

Objective 6: to use outcome data (diet, physical activity, body mass index) to help decide on a primary outcome and to estimate the potential effect size of the intervention to facilitate the calculation of an appropriate sample size for a full trial

The study was not powered to assess effectiveness. Although these results should be treated with caution owing to the numbers contributing to these analyses, the objective physical activity data showed moderate to large effect size estimates across several measures, particularly the daily step count and sedentary time. These findings were amplified in per-protocol analyses, and appeared strongest in those with lower levels of physical activity at baseline. In addition, there was no evidence to suggest that self-report physical activity was different between those who did and those who did not provide valid accelerometry data, thereby increasing confidence in these results. There was also some suggestion that those who used the app more achieved better outcomes in terms of BMI, objectively measured physical activity and diet, which could potentially reflect an effect of the intervention, but would also be consistent with those who are most motivated to engage with the app or to provide follow-up data achieving better outcomes.

With regard to choice of the primary outcome for a full trial, BMI was feasible to collect and was objectively measured. Measuring diet and physical activity accurately is challenging. 134,135 Although accelerometry was objective, valid data were available for only just under half the participants. However, methods to address issues with return of the accelerometer and compliance with the data collection protocol have been used successfully in other trials and could be applied in this population. 128,133 Including interim data collection points could alleviate problems due to missing outcome data, and offering further incentives for completion of follow-up assessments could improve data collection rates. The effectiveness of these measures could be tested as part of an internal pilot within a future trial.

In terms of sample size for a future trial, there are three key parameters, namely (1) the minimum clinically important difference (MCID), in this case percentage weight loss (and its standard deviation); (2) proportion of recruited participants who engage with the intervention (among whom the MCID will apply, with zero weight loss assumed among those who do not); and (3) loss to follow-up.

If we take the MCID as 5% weight loss with a SD of 7.3% (as observed in this feasibility study; effect size 0.68), then a sample size of 47 per group would be required to achieve 90% power to detect a difference, with no loss to follow-up and 100% engagement. However, if only two-thirds of participants engaged with the intervention, as occurred in this feasibility study, then the population effect size is 0.46 (0.68 × 0.67) and 101 per group would be required. This assumes complete follow-up; allowing for 20% loss to follow-up, a study of approximately 127 per group would be required.

Recognising that it is highly risky to base sample size calculations on feasibility study data (where estimates of key parameters have wide CIs), a highly cautious set of assumptions may be to assume the SD of weight change to be 10%, 50% engagement with the intervention (giving a population effect size of 0.25) and 40% loss to follow-up, for which a sample size of 564 per group (1128 total) would be required to achieve 90% power.

Given that the intervention is very low cost and has the potential to reach large numbers, it would feasibly be cost-effective at a population level if the achieved MCID was 3% among those who engaged, and/or if engagement levels were < 50%. This would equate to population effect sizes of ≤ 0.15, and thus sample sizes at 90% power > 2000. Appendix 10 includes a range of potential sample sizes across different assumptions for the three key parameters.

Objective 7: to assess data collection tools and obtain estimates of key cost drivers to inform the design of a future cost-effectiveness analysis

The health economics feasibility component assessed procedures, validity and reliability of resource use data collection, quality of life and capability well-being tools in addition to prior estimation of parameters such as the variance of the outcome variable. Without conducting formal between-group statistical comparisons, it can be seen that health and social care resource use, food and drink and lifestyle activity spend patterns were broadly similar between the groups. The HelpMeDoIt! mean EQ-5D-3L values across groups were slightly lower than published population norms; however, values exhibited typical variance around mean values. The ICECAP-A instrument identified and measured responses across all levels of capability well-being across the control and intervention arms. The results reveal that the ICECAP-A was able to distinguish between different levels of self-reported capability well-being in this population group.

Overall, the results of the health economics feasibility study showed that the questionnaires designed for measuring health and social care resource use, food, drink and lifestyle activity spend, quality of life (EQ-5D-3L) and capability well-being (ICECAP-A) would be suitable for inclusion in a full study with some slight changes to the resource use questions. The costs of developing the app and website were mostly upfront, which meant that the cost per participant for the intervention was relatively high in this study. In a future trial, the delivery of the intervention would be a low per-participant cost, which would mainly cover the technical support and ongoing hosting. The low expected cost per participant would permit even modest reductions in BMI, equivalent to the low population effect sizes discussed in the previous section, to attain cost-effectiveness in a future trial.

Objectives 8, 9 and 10: to investigate how participants and helpers engage with goal-setting, monitoring and social support using new technologies and how these elements interact within a behaviour change intervention; to develop a conceptual model of how the key mechanisms of goal-setting, monitoring by self and others, social support and behaviour change are facilitated by the intervention; to test the logic model and theoretical basis of the intervention in stages 1 and 2

The above objectives broadly relate to the intervention theory and logic model. These objectives were addressed in both Chapters 5 and 6. Overall, participants and helpers engaged with the intervention concept but not necessarily via the app. Data from the interviews led to support for the hypothesised intervention logic model and programme theory and has led to further refinements (see Figure 10). Key changes relate to the importance of some of the hypothesised mediators and intermediate outcomes. Eight of the 11 proposed mediators of change were identified as being central to the HelpMeDoIt! intervention. These were increased social support; increased engagement with helpers via the app; increased interaction with helpers not via the app; reflecting and setting on-going goals; increased action planning; increased self-monitoring; increased skills and knowledge; and increased motivation. Increased motivation, increased social support and increased goal-setting and self-monitoring were identified as key mediators. Two out of four intermediate outcomes were also supported by the findings, namely improved social support and healthy habit formation. Although supported by the evidence base, improved self-efficacy136 and improved self-esteem137 do not appear to be integral processes. Indications in the qualitative data were that those with the strongest social support lost the most weight; they also reported more rigorous goal-setting and monitoring and had more successful workarounds for their identified barriers. This provides further support for the logic model. A number of key contextual factors were also identified that were incorporated into the revised logic model (see Figure 10).

Objective 11: to explore the characteristics of participants’ social networks and the influence social networks have on participant experiences and outcomes of the intervention

As noted at the beginning of this report, objective 11 was not part of the original funding application. Although some aspects of this objective were briefly mentioned within the qualitative findings, data for this additional objective will be analysed and published at a later date.

Objective 12: to assess whether an effectiveness trial is warranted

The results of the study indicate that an effectiveness trial is feasible and warranted. All of the progression criteria have been met, apart from obtaining intervention costs (progression criterion 8). We are in the process of negotiating with relevant bodies to cover the treatment costs associated with the intervention and to host the website and app in any future roll-out.

Implications for health care

This was a feasibility study. However, if effectiveness was demonstrated in a full trial there are several implications, including the following.

1. HelpMeDoIt! may have the potential to deliver a low-cost, potentially high-reach intervention for adults with obesity.

2. HelpMeDoIt! could be used as a complementary intervention used alongside other health-care or lifestyle services, including other apps.

3. HelpMeDoIt! may have potential to positively influence the lifestyle of individuals in a participant’s broader social network.

4. This approach to mobilising social support could be used in other areas of health behaviour change.

Recommendations for research

1. To further understand the motivation and engagement of helpers in relation to providing social support to participants.

2. To assess the effectiveness and cost-effectiveness of the HelpMeDoIt! intervention after further refinement of the intervention.

3. To further explore the key mechanisms of change identified by the HelpMeDoIt! feasibility findings.

Final analysis of HelpMeDoIt

External collaborator: Professor Sharon Anne Simpson.

Report description: Final.

Prepared by: Sarah Barry.

Last run on: Tuesday 20 March 2018 at 15:16:10 by Sarah Barry.

Created by program: \\Rcb-storage\filestore\Studies\HelpMeDoIt\statistics\programs\HelpMeDoIt_Analysis_v1_0.R.

Created using software: R version 3.4.1 (2017-06-30) for windows with additional packages mice interplot.

Source data file: \\Rcb-storage\filestore\Studies\HelpMeDoIt\statistics\data\\v1_0.

Protocol version: v5.0 (28 February 2017).

SAP version: v1.0 (17 January 2018).

Assumptions document: \\Rcb-storage\filestore\Studies\HelpMeDoIt\statistics\programs\Assumptions\HelpMeDoIt_Analysis_Report_v1_0_Assumptions.doc.