Text messaging and financial incentives to encourage weight loss in men with obesity: the Game of Stones feasibility RCT

Endowment incentive contracts

Endowment incentive contracts are financial incentives whereby participants receive a hypothetical endowment, for example £400, at the start of a trial. They can ‘keep’ the money at certain time points if they achieve a weight loss target [e.g. 10% weight loss from baseline, the top-level target suggested by the National Institute for Health and Care Excellence (NICE)6] but will ‘lose’ money if targets are unmet. This is in line with the evidence for deposit contracts5,30 and insights from behavioural economics, including loss aversion and endowment effects. 34,35 Deposit contracts are not used in this study as they are likely to have a negative impact on the engagement of men from disadvantaged backgrounds. 5 However, an endowment incentive contract can mimic the effects of a deposit contract in terms of loss aversion: people are more motivated to avoid losses than they are to achieve similarly sized gains. 34 A randomised controlled trial (RCT) examining framing financial incentives as losses compared with framing them as gains among overweight and obese adults found that incentives framed as losses were more effective for achieving physical activity goals. 36

The endowment design used in this study draws on insights that people ascribe more value to something because it belongs to them. 34 A DCE12 combining incentives and SMS attributes showed promise for smoking cessation in women. Owing to the different active ingredients and mechanisms of action, the effects of SMS and incentive components could be synergistic.

Phase 1: trial design

• To collaborate with a charity for men and men with obesity to minimise inequalities and maximise intervention appeal and reach.

• To refine the theoretical basis of the intervention by integrating systematic review findings and NICE evidence for reducing diets, physical activity, BCTs and theory to refine a logic model.

• To operationalise acceptable and effective BCTs to embed in a novel narrative SMS delivery form, which builds on existing NIHR-funded narrative SMS alcohol interventions.

• To identify acceptable ways to provide a menu of information resources on, for example, diet, physical activity, visual feedback of self-reported weight and waist circumference, and pedometer readings using our own or another available open-source website.

• To optimise an endowment incentive to motivate behaviour change. This was to be achieved by applying insights from behavioural economics and a survey/DCE with men with obesity to define the frequency, constant or varying values and contingency of incentives on targets met for (1) initial weight loss and (2) weight loss maintenance.

• To select acceptable and valid outcome and cost-effectiveness measures that have the potential for future long-term data linkage.

• To produce a library of SMS content, recruitment materials and a phase 2 protocol.

Phase 2: feasibility trial to refine approach, recruitment, randomisation, intervention delivery, engagement, retention and follow-up processes

• To assess the acceptability and willingness to be randomised to (1) SMS, (2) SMS and an endowment incentive or (3) a waiting list for SMS.

• To assess the acceptability and feasibility of recruiting from general practice obesity registers and community venues, as identified in phase 1.

• To determine the acceptability of intervention content and delivery and the attendance levels for objective weight measures at baseline and at 3, 6 and 12 months, and any unintended consequences.

• To assess the likely impact on percentage weight loss at 12 months and health inequalities using an assessment of differential uptake and potential effectiveness by socioeconomic group.

• To assess if alternative SMS based on mixed-methods data analysis are acceptable and more likely to improve trial retention at 3 months than the narrative SMS intervention. This objective was added to the protocol following an approved study extension request.

Phase 1 qualitative interviews to inform narrative SMS

It was originally planned to carry out 20–30 qualitative interviews with men with obesity during phase 1 (months 0–7) to inform the intervention development. During phase 1, 10 qualitative interviews were carried out to inform the design of the incentive DCE and survey (see Chapter 4) and one focus group with the target population was held to inform the recruitment strategies (see Chapter 5). In addition, four audio-recorded small-group discussions took place about the narrative SMS, incentive intervention and recruitment strategies at a stakeholder workshop with 27 attendees (see Chapter 2). PPI to inform the narrative SMS text library is described in Chapter 3. The project management team considered that it would be more appropriate to assess the acceptability of the narrative SMS after men had received SMS for 3 months, rather than doing this hypothetically in phase 1 or waiting until the final 12-month assessments. This allowed the possibility of amending the SMS content and frequency based on participant experience and feedback. Men were keen to share their experiences at the assessments. The overall aim of the interviews was to seek overall feedback from men on their experiences of participating in the study, the acceptability of recruitment and the intervention components.

Writing of alternative SMS

The waiting list control participants were scheduled to receive SMS for 3 months after the 12-month final assessment. It was originally planned to use the first 3 months of the current 12-month narrative SMS intervention for this purpose. No assessment at 15 months was planned. Although many men found the narrative SMS acceptable, with some showing engagement in their content at that time, this view was not universal. A few expressed strong negative reactions that led to them withdrawing from the study or stopping or blocking the SMS. The aim of examining the acceptability and feasibility of an alternative style of SMS was to explore the potential for reducing strong negative reactions and dropout as a result of the SMS. (For details on how the alternative SMS were developed, see Chapter 5.) The waiting list control arm presented a unique opportunity to test the alternative SMS. Participants were followed up after 3 months and a further 12 qualitative interviews were undertaken to assess the acceptability of the alternative SMS. (For details on the analysis that informed the writing of alternative SMS, see Chapters 5 and 6.)

Study naming competition

The study name was selected by PPI during phase 1. A £20 voucher was offered for suggesting the winning name in the competition. An initial longlist of 91 potential study names was collated from suggestions by target group men, men’s health charities, co-investigators and University of Stirling staff. PPI partners, the Men’s Health Forum in Ireland, contacted individual men and asked partner organisations to request feedback on the longlist of names from their members by e-mail. This yielded feedback from 75 men from a range of ages and backgrounds, including members of Men’s Sheds; a young men’s project; a lesbian, gay, bisexual and transgender support group; a rural farmers’ project; a separated fathers’ support group; sporting clubs; and users of the Men’s Health Forum in Ireland’s online resources. Key considerations for selecting the study name were suggested and the five most popular titles were nominated.

Based on the PPI feedback and in collaboration with the Men’s Health Forum in Ireland, the three most popular study name suggestions were anonymously voted on at the stakeholder workshop towards the end of phase 1.

Stakeholder workshop

A stakeholder workshop with 27 attendees was held to finalise study procedures, intervention components and the study name, and to make final decisions on the study protocol. Workshop attendees included a senior public health manager, co-investigators and study researchers (n = 10), dietitians (n = 3) and independent academics (n = 5) with a range of expertise in weight management, public health and financial incentives. PPI included Men’s Health Forum co-investigators (n = 2), men from the target group (n = 4) and individuals with expertise in the community engagement of men in areas of deprivation (n = 2).

Four structured small-group discussions were conducted about tabled documents that provided a sample of the narrative SMS, options for the incentive structure linked to different weight loss targets met and recruitment strategies. These discussions were audio-recorded to summarise the views expressed and to ensure accuracy. They were not transcribed verbatim, as this would introduce ethical implications for the PPI. 43 Stakeholder workshop attendees voted on the final three most acceptable and popular study names identified by PPI.

One-to-one patient and public involvement feedback on study materials

Researchers met men (n = 6) on a one-to-one basis to discuss the study materials (e.g. information leaflets and questionnaires) to ensure that these were appropriate and understandable. These individuals were Men’s Sheds group members (n = 2), former participants of the Football Fans in Training weight management programme (n = 2), a target group member and former weight management service participant (n = 1) and a community worker experienced in working with men from areas of disadvantage (n = 1). The researchers took notes during these meetings to capture the feedback gained, and, where appropriate, changes were made to the study materials.

Narrative SMS patient and public involvement

Responsive PPI consultations involved feedback from individual men from the target group (n = 20) who were identified through co-investigator connections and research networks. The methods and results of having responsive and continuous PPI in the development of the narrative SMS are described in Chapter 3.

Alternative SMS patient and public involvement

Responsive PPI consultations involved feedback from individual men (n = 10) recruited through three channels: the University of Stirling PPI group, PPI co-investigator connections and researcher networks.

The men involved agreed to provide feedback by e-mail (n = 9) or in person (n = 1) on the 12-week drafts of the alternative SMS. E-mail invitations were as follows:

These texts have been written to help men lose weight, based on men’s experience of weight loss and evidence of what helps people change their behaviour. Any comments on how these texts can be improved based on your experience would be very welcome!

These comments can be about anything: words used, individual texts, strategies suggested, overall themes, general style.

Thank you for your help.

Incentive strategy patient and public involvement

To inform the design of the novel endowment incentive strategy that aimed to invoke loss aversion theory, the PPI co-investigators were involved in the design and the interpretation of the findings from the DCE, which was completed by 1045 men with obesity, and think-aloud qualitative interviews. Men’s preferences for the incentive strategy were then presented at the stakeholder workshop to guide the final decisions about the feasibility trial protocol. Those attending the workshop were asked to discuss what should happen when men were found to fall short of their weight loss targets at appointments.

Study naming competition

The five most popular study names from the feedback gained from 75 men in collaboration with the Men’s Health Forum in Ireland were ‘Game of Stones’, ‘Guts 2 Lose’, ‘Lean Mean Texting Machine’, ‘Lose It or Lose Out (LILO)’ and ‘tXt Men’. These men also put forward key considerations for the selection of the study name:

• Avoid a name that uses the word ‘fat’ or similar (e.g. CHUBS) as this terminology carries too much baggage and reminders of personal circumstances.

• Avoid a name with the word ‘fairies’ in it. This term was disliked regardless of sexual orientation.

• Titles with ‘slim’ or ‘slimmer’ were seen as being aimed more at women.

• Be cautious about being too smart; for example, younger men liked the name suggestion ‘W8M8’, but this name was poorly understood by older men.

• Do not choose anything that seems to blame men for their situation, as many men feel that they are already blamed for all the ills of the world.

Based on the key considerations highlighted by PPI, of the five nominated names, the research team removed two (‘Lean Mean Texting Machine’ and ‘Lose It or Lose Out’) because of the words ‘lean’ and ‘lose’, which could be misinterpreted. The stakeholder workshop vote found that the most popular of the final three names, and therefore the one selected, was ‘Game of Stones’.

Stakeholder workshop

Appendix 2 provides a detailed summary of the discussions held between academics, clinicians and PPI members at the stakeholder workshop. The key issue unanimously raised by workshop attendees was recruitment. Two suggestions made by PPI members were to use workplaces and gyms as settings for community recruitment. Based on this feedback, researchers recruited men from two council work premises and the foyer of a gym. As recruitment had been raised as a key issue, a focus group was conducted at the end of phase 1 with men from the target population in a very disadvantaged urban area. This focus group is described in Chapter 5.

Summaries of feedback, suggestions and decisions made by PPI at the stakeholder workshop in relation to both recruitment and study procedures and the incentive strategy were documented (see Appendices 3 and 4).

One-to-one patient and public involvement feedback on study materials

A full summary of one-to-one PPI feedback and decisions or changes made to the information materials as a result can be found in Appendix 5. One example is a sentence in the initial draft of the post-randomisation leaflets that aimed to minimise contamination and disappointment bias between trial arms. The original sentence stated that we ‘ask that you are confidential about your participation in this study’. This was included as two newspaper articles about the study were published after the lay summary became publicly available on the NIHR website. The co-investigators wanted to avoid such media publicity, which could bias the study findings. However, PPI participants suggested that after losing weight they would want to share this success with friends and family, and that this confidentiality section should make clear that this would be OK. The section was then amended to state that ‘you can talk to close friends and family’ about the study.

Narrative SMS feedback

Chapter 3 provides details of the 20 PPI men who commented on the narrative SMS. All 20 men self-identified as belonging to the target group of those carrying excess weight.

Alternative SMS feedback

Appendix 6 provides details of the 10 PPI men who provided comments on the alternative SMS. The majority of men (n = 9) self-identified as belonging to the target group of those carrying excess weight. The group included the editorial and creative consultant at Men’s Health Forum GB, who has written award-winning health guides and online material for men on weight and other health issues. 46,48

Feedback received on the alternative SMS was both general (n = 7) and specific (n = 3). The volume of comments ranged from a few sentences (65 words) to a multipage reflective letter (2152 words). The face-to-face discussion on the alternative SMS lasted for 1.5 hours. Specific comments were provided as in-text comments in the circulated electronic document, and these ranged from 15 to 35 comments.

Substantial changes were made to the draft alternative SMS in response to feedback. Examples of specific PPI feedback and decisions about or changes made to the alternative SMS can be found in Appendix 7.

Incentive strategy feedback

Options for the incentive strategy based on the findings of the DCE were discussed at the stakeholder workshop. Key PPI issues that influenced decision-making are described in Chapter 4.

A discrete choice experiment to inform financial incentive design

A DCE was used to elicit men’s preferences for different configurations of the financial incentive scheme in terms of scheduling, frequency and magnitude to optimise uptake and engagement. A DCE is a survey method that presents participants with a series of choices between two configurations of services, in this case configurations of weight loss programmes. Using regression modelling, the relative importance of the different dimensions (or attributes) of the programme can be estimated. This information can be used to decide on the optimal configuration of the financial incentive scheme.

The relevant attributes of the DCE were largely predetermined. The Adams et al. 59 framework was used to identify the different domains (or attributes) of a financial incentive scheme. The attributes included direction, form, magnitude, certainty, target, frequency, immediacy, schedule and recipient. Choices for several dimensions were predetermined based on existing evidence, trial logistics, feasibility if rolled out, and perceived acceptability to the public and to service providers (Table 4). These included cash, based on evidence for the preference of cash;31 certainty, based on trial logistics; target weight as a proxy measure for behaviour (i.e. weight loss); delivery at 12 months, based on trial logistics, and individual recipients based on trial logistics. The detailed justification for the a priori £400 ceiling level of the financial incentive is provided in Appendix 11.

The choice of direction of the reward was based on insights from behavioural economics. The majority of previous studies using financial incentives at part of weight loss programmes used deposit contracts,62,63 which have been shown to be effective. 58 With deposit contracts, individuals deposit their own money into an account, and the money is returned if they follow through with a predetermined goal (e.g. achieving a certain amount of weight loss). However, individuals lose the money if they do not achieve the goal. Deposit contracts focus on losses rather than gains, which in theory improves effectiveness, as individuals tend to be loss averse, that is they value losses at a higher rate than equivalent gains. 64 The preferred direction of the current financial incentive scheme was, therefore, avoidance of losses.

This study targeted men with obesity living in areas of deprivation, where men may face financial constraints likely to have a negative impact on uptake and engagement. Therefore, it was decided to use a hypothetical endowment to try to invoke loss aversion. A hypothetical endowment is pledged at the start of the scheme. Participants can then secure set values of the money at certain time points if they achieve a weight loss, but will ‘lose’ money if targets are unmet. This scheme is similar to a deposit contract, the difference being that a hypothetical amount is pledged at the start rather than participants depositing their own money. Pledging the money at the start is likely to invoke an endowment effect, with individuals ascribing a (higher) value to the money because they believe that they ‘own’ it.

Framing the incentives in term of hypothetical losses rather than gains has been shown to be effective in other settings65 and, more recently, as part of a financial incentive scheme for increasing physical activity. 36

The dimensions that were uncertain and were, therefore, the relevant attributes in the DCE were frequency, schedule and magnitude. The DCE elicited preferences for the incentives at the three time points at which weight was to be verified (3, 6 and 12 months). The value of the incentive at the different time points was determined by an overall magnitude of around £400 and a sufficient amount at 12 months to encourage weight loss maintenance. The attribute for the value of the incentive at 12 months did not include £0 (i.e. participants would always receive some amount of money if they met their target at 12 months). The attributes and the levels of the DCE were:

• value of the incentive at 3 months: £0, £75, £100, £150

• value of the incentive at 6 months: £0, £75, £100, £150

• value of the incentive at 12 months: £100, £150, £200, £250.

This means that the overall magnitude of £400 falls within the range of possible overall values (this ranges from £100 to £550). The range of values means that the impact of magnitude on uptake can be examined.

To explain the incentive scheme, respondents were told that they would be pledged a hypothetical endowment at the start of a weight loss programme and they could lose or secure part of the endowment at different time points, depending on whether or not they achieved certain weight loss targets. A series of visuals were used to help explain the concept. Participants were also informed that they would receive all of the money they had secured after the 12-month study was complete, and not at individual time points.

There was uncertainty about the appropriate weight loss targets from baseline weight, and it was unclear whether or not uptake might vary across set weight loss targets. For example, if men perceived a 10% target as unrealistic, they may have been more likely to opt out. DCE participants were randomised to two different sets of targets for the DCE: 3% at 3 months, 5% at 6 months and 5% at 12 months; or 5% at 3 months, 10% at 6 months and 10% at 12 months. This is in line with guidelines suggesting that the minimum target weight loss for clinical benefit is 3% and the top target is around 10% based on existing evidence. 6 Participants were shown what the percentage weight loss targets meant for them personally in terms of stones, pounds or kilograms, based on their preference.

Participants were asked which programme, if any, they would choose. They were told that they would receive the following as part of the weight loss programme, in addition to the incentives: information on ways to lose weight, text messages and a pedometer. They were told that they could choose to receive the incentive as either cash or a donation to charity. An example of choice is presented in Figure 8.

FIGURE 8.

Screenshot of the DCE choice scenarios presented to participants.

The full factorial design consisted of 64 possible incentive configurations (three attributes with four levels each). Ngene software (ChoiceMetrics; www.choice-metrics.com) was used to produce a D-efficient design of 16 choices. The 16 choices were divided into two blocks of eight to reduce response burden. Choice 3 was repeated as choice 8 to test for consistency of responses. Respondents were therefore presented with nine choices in total.

Questions to inform overall intervention design

Several questions were included in the survey to inform the trial design. Some of these questions were also used to get respondents to think about the different components of the weight loss programme on offer in the DCE. Respondents were asked whether or not they would be interested in receiving information, a pedometer, text messages and financial incentives. The narrative text message intervention was described as follows: ‘The messages will come from fictional men who are trying to lose weight and the different ways they go about doing this, informed by research about what works’. In addition, respondents were asked whether or not competition against other men in the programme was important; where they would prefer to be weighed; how they would like to receive feedback; how often they would like to receive the texts; and the distribution of the weight loss target they preferred (i.e. lose weight gradually over 12 months; lose weight over 6 months and then maintain; lose weight over 3 months and then maintain).

Questions about the financial incentive design asked whether respondents preferred to receive a charitable donation or cash and their preferred format of the pledge (i.e. cheque, bank statement or facsimile notes). They were also asked to imagine that they would be offered up to £400 to help them achieve the 5% and 10% weight loss targets. They were then asked if they would prefer to receive all of the £400 for meeting the final weight loss target at 12 months or as three instalments at 3, 6 and 12 months. If they chose the latter option, they were asked to indicate how they would distribute the money across the three time points.

Sociodemographic characteristics

As the intervention aimed to have broad reach and include men with obesity from disadvantaged backgrounds, the study examined how preferences varied according to deprivation level. Information on a range of indicators for deprivation was elicited, namely household income, education, employment status, car ownership and home ownership. Information was also collected on age, number living in household, ethnicity, self-rated general health, smoking status and physical activity.

Further questions included views on weight and weight loss, past weight loss attempts, ideal weight and weight perception (responses being about right, too heavy, too light, don’t know). The importance to respondents of losing weight and their confidence in their ability to lose weight and maintain weight loss were measured on a 1 to 7 scale. Respondents were also asked when they had last tried to lose weight and how often they had tried to lose weight in the past.

The acceptability of financial incentives and responsibility for being overweight were elicited using a survey measure by Promberger et al. 66 Participants were asked to indicate the acceptability of NHS funding financial incentives as an add-on intervention on a scale from 1 to 7, ranging from definitely yes to definitely no. Participants were also asked how responsible overweight people are for being overweight on a scale from 1 to 7, ranging from entirely responsible to not responsible at all.

As the financial incentive design was informed by behavioural economics, the survey also collected information on behavioural parameters using standard measures. Time preferences were elicited using the multiple price list. 67 A dummy variable was created for very impatient individuals (always chose option A in the first multiple price list). The survey also measured perceived temptation. 68 Participants were asked to imagine that they had won 10 certificates for ‘dream restaurant nights’ that had to be used within the next 2 years. They were asked how many certificates they would ideally like to use in each year. They were then asked if they thought that they would be tempted to deviate from their ideal allocation. The original question then asked them to predict how they would behave (i.e. if they would give into this temptation).

Discrete choice experiment analysis

The analysis of the DCE responses is based on random utility theory. The utility V of weight loss programme j is a linear and additive function of the attributes and levels:

where V_j is the utility of option j and β are the parameters to be estimated indicating the relative importance of the different attributes. ‘Programme’ is the alternative specific constant that takes on the value of 1 if the option is either programme A or programme B and 0 if the option is the opt-out, and Amount3, Amount6 and Amount12 are the values of the incentive at 3, 6 and 12 months, respectively.

If individuals’ preferences are guided by their time preferences only, then the amount at 3 months will be valued highest, followed by the amounts at 6 and 12 months, respectively, although this effect may be reduced as the money is paid out at 12 months only. However, individuals may realise that maintaining weight loss is challenging and, therefore, may prefer longer-term incentives over short-term incentives. This would mean that the amount at 12 months may be valued higher than the amount at 3 months. In terms of overall amount, a rational decision-maker should prefer the programme with the highest overall amount. However, this may not be the case if the individual has ethical concerns about financial incentives and may find certain higher levels unacceptable.

A number of estimation techniques were explored, including conditional logit, random parameter logit and latent class modelling. As financial incentives can be controversial, the sample was expected to have diverging preferences for the weight loss programmes, with some participants strongly opposed to incentives and others strongly in favour. Examining this preference heterogeneity was of interest for the intervention design. Preference heterogeneity can be accounted for by using random parameter logit regression, which allows parameters to vary randomly over individuals. A disadvantage is that a distribution needs to be specified for the random parameters, which is usually either normal or log-normal. However, given the diverging views on the acceptability of financial incentives, a bimodal distribution was expected to be more appropriate. Latent class modelling was therefore used, which does not require assumptions about the distribution of the random parameters. Instead, it identifies different utility functions across unobserved subgroups or latent classes. A latent class model fits the best possible model with a predetermined number of classes. For each class, different coefficients (or discrete random effects) are estimated for the attributes. The latent class model is a better fit for the current data, confirming that the random parameters are unlikely to be normally distributed.

The optimal number of classes in the DCE was determined in an iterative process by comparing models with different numbers of classes on the basis of goodness of fit (Akaike information criterion), precision of estimates and size of class shares. As highlighted by Hole,70 there is often a trade-off between goodness of fit and the precision of the parameter estimates. A model with more than three latent classes resulted in very small class shares (< 5%) and large standard errors for some of the estimates. It was therefore decided that three latent classes were the preferred specification.

The association between observable characteristics and class membership was explored. The characteristics included socioeconomic characteristics, weight loss characteristics, acceptability of financial incentives and behavioural parameters.

The socioeconomic characteristics included age (dummy: aged 35–54 and aged 55–75 years); education (dummy: General Certificate of Secondary Education/Ordinary level/Certificate of Secondary Education/no formal qualifications), income (dummy: income < £15,000 and income £15,000–30,000), working status (dummy: unemployed/not in paid work for other reason/not in paid work because of long-term illness or disability, and retired), car ownership (dummy: no car), home ownership (dummy: owns home), single household (dummy: single household), ethnicity (dummy: non-white) and region (dummy: Scotland).

The weight loss characteristics included BMI, importance of losing weight [continuous variable on a scale from 1 (not important) to 7 (very important)]; confidence in ability to lose weight and confidence in ability to maintain weight loss [continuous variable on a scale from 1 (not confident) to 7 (very confident)], whether or not currently trying to lose weight (dummy: trying to lose weight); number of previous weight loss attempts (dummy: one to three previous weight loss attempts and four or more previous weight loss attempts); and percentage difference between current and ideal weight. Individuals who indicated that weight loss was important might have valued incentives more strongly or at more time periods. Incentives might also have been valued more strongly or at more time periods by those who indicated a low confidence in their ability to lose weight; receiving incentives at all time periods would assist with early engagement and maintenance of weight lost. Similarly, those who indicated a lack of confidence in maintaining weight loss might value more strongly long-run incentives to provide an incentive to keep off lost weight. A number of health (behaviour) characteristics including smoking status (dummy: current smoker) and physical activity (dummy: participates in vigorous physical activity) were also included.

Unacceptability of financial incentives [continuous on a scale from 1 (incentives should definitely be funded) to 7 (incentives should definitely not be funded)] and responsibility of overweight people for being overweight [continuous on a scale from 1 (people are not responsible) to 7 (people are entirely responsible)] were also included. It was hypothesised that those participants who found incentives unacceptable might opt out and not value the amounts at 3, 6 and 12 months.

The behavioural parameters include perceived temptation (dummy: tempted to consume more than their ideal allocation) and present orientation (dummy: always chose the immediate option in the first multiple price list). It was hypothesised that individuals who recognised that they might be tempted to consume more than their ideal allocation in a given year might prefer to spread the incentive payments across the year to counteract the temptation they anticipated they would feel. Finally, it was hypothesised that individuals who displayed a strong present bias would value more strongly the incentives associated with the earlier time points.

Preferences varying by weight loss target were also explored (dummy: equal to 1 if the respondent had been allocated to the hypothetical 5% target, and 0 if he had been allocated to the hypothetical 10% target).

Including a large number of characteristics in the membership part of the latent class model can lead to unstable models with large standard errors. To obtain a parsimonious model, backward stepwise regression was used and characteristics were eliminated until all remaining characteristics were statistically significant at a 5% level and there were no large standard errors.

To assess the relative ranking of different incentive designs for the latent classes, we predict uptake π of the intervention. The utility of the opt-out is set to zero. Uptake is estimated as follows, where V_j is the utility of weight loss programme j (see Equation 1):

Preferences for all intervention components

Regression analysis was carried out to determine which factors influenced preferences for different aspects of the weight loss programme: interest in receiving information/advice, a pedometer, the text messages and incentives, importance of competition in weight loss, preferred distribution of weight targets, preferred weigh-in venue, preferred frequency of weighing, preferred form of incentive (cash or donation) and timing of payment (over three time periods or all at the final weigh-in). Ordered probit regression was used to model ordinal outcomes and probit regression for binary outcomes. The same individual characteristics were included as those included in the membership part of the DCE.

Sample characteristics

The sample consisted of 1045 men with an average BMI of 34.9 kg/m² (range 30.0–125.6 kg/m²). Appendix 14 (see Table 32) reports weight and perceptions about weight loss. The percentage difference between current and ideal weight was 19.2%. Around half of the sample was currently trying to lose weight and around 95% perceived themselves as too heavy. Around 8.5% of men had never tried to lose weight. On average, men had around seven previous weight loss attempts. Losing weight was relatively important on average, but men were less confident in their ability to lose weight and to keep lost weight off.

As quotas were imposed for age and region, the sample is representative of the UK population in terms of these characteristics. Appendix 15 (see Table 33) shows the full sample characteristics. As the intervention is aimed at men living in disadvantaged areas, it is important to be able to perform subgroup analysis by socioeconomic status. About 36% (n = 665) of participants had a level of education below Advanced level; 44% (n = 461) had an income of ≤ £30,000 per annum, 17% (n = 180) had no car and 33% (n = 346) were not homeowners. These were sufficient proportions to allow subgroup analysis. Although a relatively high proportion of the sample was of white ethnicity (98%, n = 989), a sufficient number of people were of non-white ethnicity to allow subgroup analysis.

Discrete choice experiment results

The DCE responses for the two repeated choices show that 83% of participants passed the consistency check and provided the same answer in the two identical choices. This in line with findings from other studies. 71 The opt-out was chosen in only 7% of all choices. This is relatively low, but it should be noted that all weight loss programmes were associated with an incentive of at least £100.

Table 5 shows the latent class results. The average class probabilities in the sampled population were 6.0%, 29.6% and 64.4% for latent class 1, 2 and 3, respectively. For latent class 1, the coefficient for programme (alternative specific constant for programmes A and B) is negative and none of the amounts is significant, indicating that incentives are not being valued. For latent class 2, only the incentives at 6 and 12 months are significant, suggesting that this class prefers longer-term incentives. The incentive at 12 months is valued relatively more highly than the incentive at 6 months. For latent class 3, all three incentives are important, with the shorter-term incentives being relatively more important. The incentive at 6 months was valued most highly, followed by the incentives at 3 and 12 months.

Class membership was a function of age, weight loss importance, perceived temptation, unacceptability of incentives and responsibility for being overweight. Individuals who perceived the funding of financial incentives to be unacceptable had a higher probability of belonging to class 1 (‘no incentive’ class) and class 2 (‘longer-term incentives’ class). Older age and attaching less importance to weight loss were associated with a higher probability of belonging to class 1. Perceived temptation increased the probability of belonging to class 3 (‘incentives at all time points’ class) compared with class 2, which may suggest that individuals who know that they are likely to be tempted to deviate from their optimal future choices value incentives at all three time points to deal with their temptation. Participants who thought that individuals are responsible for being overweight had a lower probability of belonging to class 2 (‘longer-term incentives’ class) than to class 3.

Table 6 shows the predicted uptake for different configurations of the financial incentive scheme (the choices offered in DCE) for latent class 2 and 3 and for the whole sample. Note that uptake is not a function of the incentives for class 1 as none of the incentive amounts was statistically significant. The same scheme with the highest overall value (£550) was ranked highest in all cases. The configuration ranked second differs between classes 2 and 3. Note that the predicted uptake is very high for latent class 3 for most configurations.

To explore the impact of the overall incentive magnitude, the uptake for a range of overall values was predicted. Figure 9 shows the predicted uptake by overall magnitude for two different distributions across the three time points: 10%, 45% and 45%, and 45%, 45% and 10%. Uptake was relatively low for class 3 at low magnitudes, but once the incentive reached a certain threshold, uptake was high and marginal increases in uptake thereafter were low. For class 2, incentives continue to increase uptake along the whole magnitude range and marginal increases remain relatively large. Although a lower magnitude could be used to encourage uptake by latent class 3, class 2 requires a larger magnitude. The £400 incentive level predicts an uptake of > 90% for class 2.

FIGURE 9.

Uptake by overall magnitude when the incentives are distributed over three time points as (a) 10%, 45%, 45%; and (b) 45%, 45%, 10%.

Preferences for intervention components

Table 7 shows the descriptive statistics for the questions on the different intervention components. The majority of participants were interested (yes or maybe) in all intervention components: information, pedometer, text messages and incentives. Text messages were associated with the lowest level of interest. This may be because participants were less familiar with the use of text messages for weight loss support. The preferred venue at which to be weighed was the GP surgery, followed by the pharmacy and the community hall. The most frequently mentioned venue under ‘other’ was home, with additional suggestions including a commercial weight loss class or the gym. A small number of men highlighted the importance of the weigh-in taking place in private. The most preferred frequency for weigh-ins was at the start, 3, 6 and 12 months. It should be noted that the responses to this question are likely to have been framed by the timing of the weigh-ins in the DCE. There was preference heterogeneity in terms of preferred frequency and timing of text messages.

The preferred method of feedback (about how much money was secured based on achieving weight loss targets) was e-mail, followed by text message. The use of hypothetical cheques was perceived as the best way to make the pledge at the start of the intervention most realistic. The majority of men preferred cash, with 11.6% preferring charity donations.

Just over half of the sample preferred to spread the incentive over the time points. The highest amount was allocated to 12 months, followed by 6 months. In total, 60 out of 556 allocated < £400 across the time points. In respect of all participants who allocated £400 (also those who allocated the full amount to 12 months), the average amounts allocated across the time points were £51 at 3 months, £55 at 6 months and £294 at 12 months. This suggests a higher importance attached to 12 months than was seen in the DCE results. However, it should be noted that the results are not directly comparable, as the open-ended questions were framed in terms of reward incentives, whereas the DCE was framed in terms of endowment incentives. In addition, evidence suggested that open-ended questions and closed-ended methods such as DCEs can produce different results. 72

The preferred distribution of the weight loss target over the 12 months varied according to the weight loss target. More participants preferred a gradual weight loss for the 10% target than for the 5% target. This may reflect that losing 10% over 6 months is perceived to be much harder than losing 5% over 6 months. The difference is statistically significant (χ² = 7.08; p = 0.03).

Appendix 16 (see Table 34) shows how preferences for the different intervention components differ by individual characteristics. The regression techniques allow for the ordinal nature of the dependent variables. The coefficients do not allow inference of the magnitude of the impact, allowing only interpretation of the sign and significance of the estimated coefficients. For example, a positive estimated coefficient in the first model is interpreted as increasing the likelihood of being interested in information, while negative estimated coefficients are interpreted as decreasing the likelihood of being interested in information.

The results show that preferences did not seem to vary systematically by deprivation. The indicators were statistically significant in only a few models. Men with lower levels of education only (no qualifications or General Certificates of Secondary Education) were less likely than men with higher levels of education to be interested in receiving financial incentives. However, men who did not own a car were more likely to be interested in financial incentives than men who owned one or more cars. Men with household incomes of < £15,000 were less likely to be interested in receiving information than men with incomes of ≥ £30,000, and men with household incomes of between £15,000 and £30,000 were less likely to prefer a donation to charity than cash than men with incomes of ≥ £30,000.

Older men and men living in single households were less interested in text messages, pedometers and financial incentives, and men of non-white ethnicity and retired men were more interested in information. Importance of weight loss increased interest in receiving information, pedometers, text messages and financial incentives. Men with four or more previous weight loss attempts were more likely to be interested in information and text messages than men with fewer than four previous weight loss attempts. Men with a larger difference between their current and ideal weight were more likely to be interested in financial incentives and men who participated in vigorous physical activity were more likely to be interested in pedometers and text messages. Men with a higher BMI and men who viewed funding of incentives for weight loss as less acceptable were less likely to be interested in financial incentives, whereas men who had tried to lose weight four or more times and men who thought that overweight people are responsible for being overweight were more likely to be interested in financial incentives.

Older men and men in single households attached less importance to competition as a motivator to weight loss, whereas men who attached more importance to weight loss were more confident in their ability to lose weight. Smokers and men who were physically active attached more importance to competition as a motivator.

Men who were currently trying to lose weight were less likely to prefer to lose weight quickly (lose the target weight loss over the first 3 months), whereas men who had tried to lose weight four or more times were more likely to prefer to lose weight quickly.

Older men and men who were more confident in their ability to lose weight were more likely to prefer to receive all of the incentive at 12 months, whereas men who thought that they would be more tempted to use more of the restaurant vouchers in the first year than would be ideal were less likely to prefer to receive all of the incentive at 12 months. It could be argued that those who knew that they might be tempted preferred to spread the incentives to help them control their temptations.

Older men, men of non-white ethnicity, men with a household income of between £15,000 and £30,000, men with lower levels of health and men who were more present oriented were more likely to prefer the GP surgery as the location for being weighed. It may be that older men and men with lower levels of health visit their GP more often and so a GP surgery may be more convenient.

Focus group to inform recruitment

At the phase 1 stakeholder workshop (see Chapter 2), recruitment was raised as central to the success of the study. Therefore, prior to recruitment commencing, an audio-recorded focus group with men (n = 5) living in a disadvantaged urban area was conducted to explore the acceptability of the planned recruitment strategies and to inform the research team’s approach to recruitment. Focus group participants were, on average, 52 years old (range 32–58 years), had an average of 1.8 household members (range 1–3 members) and lived in an SIMD 1 area (n = 5), and four were classified as overweight or obese. The researcher used a topic guide and focus group plan (see Appendix 17) to guide the discussion and used recruitment materials (e.g. the draft GP participant invitation letter) to support the questions asked. A summary of the themes and findings of the focus group can be found in Appendix 18. Learning from this focus group helped to inform and finalise the recruitment strategies for the feasibility RCT.

Participant recruitment was undertaken through two channels: (1) community outreach recruitment and (2) general practice obesity register recruitment.

Community outreach

Study information stands, usually consisting of a table displaying information leaflets and two small table banners with the study logo, were hosted in community venues such as supermarkets, fitness centres, hospital foyers, health centres, council workplaces and community centres (Figures 10 and 11). Researchers were in attendance to provide information, answer questions and discuss the study with men who showed interest. Interested men were given a brief information leaflet and asked to provide their contact details (i.e. a telephone number or alternative contact information) to arrange a face-to-face appointment. Recruitment took place in some of the most disadvantaged areas of the sites and often in the evenings and weekends to ensure that working men had an opportunity to be recruited. To ensure researcher safety, a researcher assistant pool of 10 master’s or PhD students was established to ensure that recruiting researchers could always work in pairs.

FIGURE 10.

Community outreach recruitment activity in a supermarket.

FIGURE 11.

Community outreach activity in a community venue.

Men in the community were also recruited by word of mouth, usually as a result of researchers’ discussions with friends, family, voluntary-sector staff and sports workers who passed on study information to eligible men they knew. Brief information leaflets were distributed across localities in local shops, libraries, barbers and community centres. Those interested in participating contacted researchers by telephone, SMS or e-mail to arrange a face-to-face appointment. The eligibility of those men recruited from community settings was assessed either during an initial contact (e-mail, telephone, face to face) with the researcher or during the baseline appointment.

General practice obesity register letters

The NHS Research Scotland Primary Care Network (NRSPCN) provided a list of general practices with patient list size in areas of high deprivation in the two study sites. The research team selected practices in more deprived areas (i.e. SIMD 1 and 2) and NRSPCN invited these practices to participate in the study. Overall, 33 practices were identified as suitable from deprivation and logistical perspectives (n = 13 in site A and n = 20 in site B). Five practices (n = 4 in site A and n = 1 in site B) agreed to participate. Two further general practices (n = 2 in site B) expressed an interest in participating after the study was full.

The NRSPCN ran searches on practice databases to identify men who met the inclusion criterion, namely having a documented BMI of ≥ 30 kg/m², and these lists were then screened by clinical practice staff, who removed anyone considered to be unsuitable to take part. These lists, which included names, addresses and contact information, were transferred to The Health Informatics Centre (HIC) at the University of Dundee to be stored securely. The HIC then sent GP-headed study invitation letters and brief information leaflets to men from the five practices across both localities. Men who were interested in the study either contacted the research team directly or returned a freepost ‘opt-in’ reply card to the HIC. The contact information of those opting in was uploaded to the HIC online ‘participant tracker’ system for the researchers to access. When it was not possible to contact men to arrange a face-to-face appointment, a reminder letter was posted from the research team asking them to get in touch if they were still interested in participating.

Face-to-face follow-up appointments

Men attended face-to-face appointments with the two research assistants. At the appointment, the researcher provided men with a copy of the full study information leaflet, discussed the study in detail and answered any questions. If men wanted to proceed with the study, written informed consent was gained. Men who did not meet the BMI, waist circumference or other eligibility criteria were thanked for attending and advised that they would not be able to take part in the study. No further information was provided.

While the participant completed the baseline questionnaire, the researcher created a website login and requested randomisation (see below for details) to one of the three arms. At the end of the baseline assessment, the researcher informed the participant of his allocation, provided a trial-arm-specific post-randomisation information sheet, talked through the allocation and answered any questions. The researcher calculated weight loss targets for men randomised to the intervention arms. These targets were to lose 5% of baseline weight by 3 months, to lose 10% of baseline weight at 6 months and to keep 10% off at 12 months. The targets were calculated according to men’s measurement preferences (stones and pounds or kilograms) and were linked to the incentives for the SMS + I arm.

All participants received a pedometer, a weight loss fact sheet (British Dietetic Association Weight Loss Food Fact Sheet: www.bda.uk.com/uploads/assets/c5761f35-ec82-4449-9fed3bcfe1fd08a5/Weight-loss-food-fact-sheet.pdf; accessed 6 February 2020), a personalised participant card and access to the study website.

The participant card included reminders of the dates for future appointments, website login details and a space for appointment visit weight and waist circumference measurements to be recorded. The weight loss targets for those in the intervention arms were recorded on the card.

All participants were given login details for the study website (www.gameofstonesresearch.com; accessed 25 June 2019), described further below, the content of which varied depending on the arm allocation.

All participants received a £20 voucher as recompense for their time when attending the 12-month appointment.

Randomisation

The researcher requested independent randomisation by Tayside Clinical Trials Unit using a secure remote web-based system with telephone back-up, stratifying by recruitment method (GP and community) and recruitment site (site A and site B). For practical reasons, the two researchers who conducted the outcome assessments were not blinded to the arm allocations, with the exception of a brief trial period of blind outcome assessment. The feasibility of blinding researchers conducting intervention arm outcome assessments was tested for 11 of the 6-month appointments. This involved one of the researchers conducting 11 out of 44 6-month appointments while unaware of the arm allocation. The researcher noted if the participant disclosed their intervention arm. None of the 11 participants mentioned their allocation arm, illustrating that blinding could be feasible in a full trial. All other members of the study team were blinded to participant arm allocations.

Narrative SMS

For a full description of the development of the narrative SMS, including the underlying approach and narrative PPI, see Chapter 3. In summary, the narrative SMS component included a library of 604 SMS that were sent to participants over 12 months. All participants received SMS automatically according to a predetermined schedule. SMS were sent between 08.00 and 22.00 and ranged from 0 to 5 SMS per day. The SMS were delivered by the HIC using existing automated technology linked to Tayside Clinical Trials Unit. Participants could respond to the SMS but would not receive a reply. The same narrative SMS were sent to the SMS + I and SMS-only arms. Participants could request that the SMS be stopped at assessment time points, or by contacting the research assistants either by telephone or by replying to the SMS.

Endowment incentive

For a full description of the background to the incentive component, see Chapters 2 and 4. In summary, all SMS + I participants were ‘endowed’ with a £400 incentive at the start of the trial. The incentive was placed into a hypothetical personal account at the University of Stirling (no interim withdrawals were possible). For the endowment effect (perceiving the money as their own and ascribing value to it), men received a hypothetical cheque at randomisation. The full £400 could be secured by meeting weight loss targets, based on assessments at the face-to-face outcome assessment appointments: 5% of body weight lost since baseline at 3 months (£50 secured/lost), 10% at 6 months (£150 secured/lost) and 10% at 12 months (£200 secured/lost).

The incentive strategy is detailed in Table 9 and was available on the SMS + I web pages. At all appointments, if not achieving a weight loss of at least 5% of their starting weight, men lost all the money allocated to that time point. At 6 and 12 months, men lost some of the money for each percentage weight loss not attained between 5% and 10%. To secure money, participants’ weight at appointment visits had to be measured on the study scales within 2 weeks either side of their target date (exactly 3, 6 or 12 months after their baseline appointment). This information was entered onto their personalised credit-card-sized card. Men received the money after the 12-month assessment based on the targets they met. Weight at 12 months had to be less than at baseline to receive any money, regardless of whether or not interim weight loss targets had been met. If men did not attend the 12-month assessment, they received no money.

Feedback to men on meeting incentive targets was sent by automated SMS 1 hour after the researcher completed the weight assessment data entry and was available on the participant’s SMS + I web page. Secure options for delivering the incentive to men were offered that minimised any risk to researcher safety: Bacs (Bacs Payment Schemes Limited, London, UK) direct transfer or cheque by post.

Web pages

Each participant was provided with a unique login ID for web pages after randomisation according to his trial arm allocation.

The front page was accessible to all participants and included essential information about the trial and links to existing online resources offering a choice of evidence-based diets, physical activity approaches, hints and tips. The web pages included:

• the NHS choices website: www.nhs.uk/LiveWell/Loseweight/Pages/Loseweighthome.aspx (accessed 25 June 2019)

• ManvFat website: https://manvfat.com (accessed 25 June 2019)

• the British Dietetic Association website: www.bda.uk.com/uploads/assets/c5761f35-ec82-4449-9fed3bcfe1fd08a5/Weight-loss-food-fact-sheet.pdf (accessed 25 June 2019)

• Men’s Health Forum GB website: www.menshealthforum.org.uk (accessed 25 June 2019).

Both the SMS + I and the SMS-only web pages had a brief biography and images of the characters featured in the narrative SMS, as well as information specific to the trial arm. There was also a link for self-monitoring weight, pedometer steps, waist circumference and belt notches that produced visual progress charts.

The SMS + I arm had access to additional pages describing the financial incentives and a visual progress chart of money secured/lost in relation to weight loss targets. The progress chart was automatically populated by the HIC database using an algorithm after weight assessment appointments.

Pedometer

The study pedometer was the 3DFitBud, A420S, manufactured by 3Dactive (Goldhat Ltd, London, UK; https://3dactive.com).

Primary feasibility outcome for this feasibility randomised controlled trial

The primary outcome was a decision about whether or not to proceed to a full effectiveness and cost-effectiveness RCT for the SMS intervention, with or without an incentive, compared with a waiting list control, based on the prespecified progression criteria. The progression criteria are detailed in Chapter 1 and assessment of whether or not they are met is based on both quantitative and qualitative data analysis and the views of the Study Steering Committee.

Secondary feasibility outcomes

• Recruitment rate.

• Overall withdrawal rate:

  • withdrawal because of SMS

  • withdrawal because of incentives

  • withdrawal for other reasons.

• Number of appointments attended/missed.

• Number of SMS responses made by participants.

• Number requesting that SMS be stopped.

• Uptake of interactive web page usage by entering self-monitoring information.

• Process data (differential recruitment strategy, i.e. general practice or community, differential retention by socioeconomic status).

• Feasibility of collecting primary outcomes for a potential full trial

  • weight change from baseline (percentage) at 12 months

  • weight change from baseline (absolute) at 12 months, measured in kg

  • waist circumference at 12 months, measured in cm.

• Measures of quality of life [EuroQol-5 Dimensions (EQ-5D)] and mental well-being (Warwick–Edinburgh Mental Well-being Scale).

• Theory-based mediators including confidence in weight loss and weight loss maintenance, importance of weight loss, self-regulation, self-monitoring (weight), maintenance processes (satisfaction with outcomes, satisfaction with experience, goal conflict, social support, environmental barriers), weight loss techniques, weight loss mindset (loss vs. maintenance mindset), quality of motivation (intrinsic and extrinsic), automaticity (eating fruit and vegetables, physical activity).

• Weight loss strategy use: owning step counter, step counter use, self-weighing, weight management strategy use.

• Health behaviours: moderate physical activity, vigorous physical activity, sedentary behaviour, alcohol consumption, smoking, fruit and vegetables consumption.

• Intervention experience: satisfaction with intervention, satisfaction with materials, helpfulness of materials.

• Weight change from baseline and waist circumference at 3 and 6 months (in intervention arms only).

Recruitment activity

The NRSPCN and study researchers systematically recorded all GP and community recruitment activities, respectively, such as venue details, time spent at the venue, number of information sheets distributed and number of contact details obtained.

Anthropometry measures

Weight and waist circumference were measured by researchers at all face-to-face assessment appointments following standard operating procedures and device instruction manuals. At baseline, height was measured using a portable standing stadiometer (Seca 217, Seca, Birmingham, UK) to the nearest 0.1 cm. Prior to weight measurements, participants were asked to take off their shoes and remove any bulky clothing or items from their pockets. At all assessments, weight was recorded using portable electronic scales (Marsden M-420, Marsden Weighing Group, Rotherham, UK) to the nearest 0.01 kg. The scales were new and calibrated prior to the first outcome assessments. Waist circumference was recorded using a tape measure (Seca 203, Seca, Birmingham, UK) to the nearest 0.1 cm.

Questionnaire measures

Self-report questionnaires measured the following variables:

• sociodemographics, comorbidities, disability, ethnicity

• perceptions of intervention acceptability

• lifestyle behaviours (physical activity, sedentary behaviour, smoking, alcohol consumption)

• well-being and quality of life

• theory-based mediators of behaviour change

• health-related quality of life [EuroQol-5 Dimensions, five-level version (EQ-5D-5L)] and resource use.

SMS responses

Responses received to the SMS were recorded, including the content, timing, frequency and original SMS that was responded to.

Website engagement data

Engagement with self-monitoring tools (i.e. weight, steps, waist circumference, belt notches) on the website was captured.

Incentive payments

The money secured by and the payments made to participants were recorded.

For the alternative SMS arm, the same variables were collected after 3 months, as was done in the SMS-only and SMS + I arms after 3 months.

Table 10 provides an overview of the feasibility and acceptability targets, their descriptors and information on where these are presented in this report.

The integration of the qualitative with the quantitative data occurred at three key stages in the mixed-methods analysis. First, owing to early concerns about men withdrawing from the study or stopping the SMS, the analysis of the 3-month qualitative interview data was conducted iteratively primarily in relation to the acceptability of the interventions and study procedures. This was to inform any early changes that might be required to the trial processes and the intervention delivery that could improve participant intervention acceptability or trial feasibility. This took place while the 3- and 6-month assessments were in progress and before the questionnaire and weight outcome data were available. Second, all study qualitative data were analysed in parallel with the analysis of the full outcome data set once the 12-month trial was complete. The research team met regularly to triangulate findings and to search for confirming or disconfirming data relating to research questions. Finally, the alternative SMS interview, baseline and outcome data were integrated into the analysis. Relationships between participant deprivation scores, 12-month intervention acceptability and weight loss outcomes were explored by developing matrix coding in QSR NVivo12 (QSR International, Warrington, UK), thus applying another technique of integrating the quantitative and qualitative analysis. 76

General practice recruitment

General practice recruitment was undertaken in five practices (site A, n = 4; site B, n = 1). Overall, 45 men were recruited and randomised from general practices. Fewer men were recruited this way than from community outreach because of delays in practices confirming their interest in taking part and returning the screened practice lists. A total of 879 invitation letters were sent out, with 90 men (10%) opting in to receive further information about the study (see Figure 12). Owing to delays in obtaining access to screen the practice obesity registers, some men opted in after the study was full (n = 37). Of those opting in before the study was full, 45 out of 53 (85%) were randomised.

The details of individual general practice characteristics, as well as recruitment and retention rates, are given in Table 11. Practices varied in the number of patients who were eligible to participate. Practices in site A were classified as in ‘large urban areas’ and the practice in site B was classified as in ‘other urban area’. 87 All practices were in areas of deprivation (i.e. SIMD category 1 or 2) and an average of 37% of the patients in these practices were classified as living in the 15% most deprived data zones. Opt-in rates varied between practices from 17 out of 347 (5%) to 33 out of 228 (15%).

Qualitative data suggested that some men felt that study recruitment from health settings or via health professionals was more appropriate than from non-health venues. Participants who received a study invitation letter from their GP found the letter easy to understand and felt that the invitation coming from their GP validated or legitimised the study:

Knowing that you have the support of the doctors’ practices and the health service, it gives it that legitimacy rather than it’s just one of these programmes that you pick up on the television, there’s always somebody who’s tried this programme, try that weight loss, or . . . that doesn’t do anything because the market’s full of these programmes.

110003, text only, 3 months

However, this may have been an artefact of recruitment source and wanting to please the researcher, as other participants recruited from the community felt that they would dismiss any letter from their GP as ‘junk mail’. Some general practices at site A engaged frequently in research. Some participants were already familiar with receiving research study invitations sent by a GP, and were happy to participate in research. By contrast, the single practice in site B had had no recent research activity and provided the required number of participants.

Community outreach recruitment

Overall, 60 men were recruited and randomised through community outreach. Table 12 outlines details of recruitment, retention and venue characteristics for community outreach activities. The majority of participants were recruited by researchers through active community outreach at information stands in community venues (42/60, 70%). Researchers attended 14 venues, on 23 occasions, for a total of 97.5 hours across both study sites. On average, for every man randomised, a team of two researchers spent around 2 hours running an information stand. Recruitment rates were unpredictable, varying between sites, days and times; for example, over a 2-hour period at a supermarket, five men were recruited and subsequently randomised, whereas no men were recruited or randomised the following day at the same venue and time.

Some men (18/60, 30%) were recruited from the community by word of mouth, which included leaflets, friends, family, voluntary sector organisation staff and other researcher contacts. One health worker picked up study leaflets for their patients from researchers at a recruitment site (n = 3 recruited). Others picked up leaflets from local venues (e.g. barbers, shops and libraries) where these had been distributed (n = 3 recruited). Community recruitment by word of mouth differed somewhat between site A (7/34, 21%) and site B (11/26, 42%). Appendix 22 provides a breakdown of community recruitment by word of mouth.

Qualitative views on community recruitment were positive. The Game of Stones study name was popular, and some participants commented that the ‘catchy’ name was one of the reasons that they had decided to participate. Some felt that it had particular appeal to men, while others had felt encouraged to approach the researchers by the factual language used at the information stands:

Yeah, that tagline, it works because I’m a man and you want to lose weight, that was just it, factual.

120017, incentive, 3 months

Retention

Differential dropout rates between the study arms were observed, with fewer SMS + I participants completing the study (64%) than SMS-only (79%) or control arm participants (83%). A total of 12 men withdrew from the study and gave the following reasons: dislike of the narrative SMS intervention component (n = 4), health (n = 3), family (n = 1), unknown (n = 1), dissatisfaction with trial arm allocation (n = 1), appointment logistics (n = 1) and multiple reasons (n = 1). Retention differed slightly by recruitment channel; of the 45 participants recruited from general practices, 36 (80%) completed the 12-month follow-up assessment, 6 (13%) withdrew from the study and 3 (7%) were lost to follow-up. Of the 60 participants recruited from the community, 43 participants (71%) completed the 12-month follow-up assessment, 6 (10%) withdrew and 11 (18%) were lost to follow up.

Retention rates of participants from more deprived areas in the SMS + I (14/19, 74% vs. 8/16, 50%) and control (10/14, 95% vs. 20/22, 71%) arms were higher than those of participants from less deprived areas, but no difference was observed in SMS-only participants (16/21, 84% vs. 10/12, 83%). The small number of men who dropped out from or disengaged with the intervention because they disliked the SMS was similar between more and less disadvantaged areas (SIMD 1 and 2, 9/15; SIMD 3, 4 and 5, 6/15).

The qualitative analysis suggested that men had a range of motivations for attending study appointments. They reported being particularly encouraged to attend if they had managed to lose weight, as this meant that their efforts would be noted by the researchers rather than by friends and family. For many participants, the interaction with the researcher was central to their participation and continued involvement in Game of Stones, with one participant saying how highly he valued having an opportunity to discuss his weight and receive support:

To discuss my progress is what encouraged me to come along. [. . .] I’ve spoke to other people about the programme, like, friends and relatives, but there’s no one that I can really discuss my issues with, if you know what I mean, not even my wife, she’ll say ‘yeah, yeah, yeah you’re doing well, keep on going’ but coming here and discussing it with you I felt gave me the motivation to continue.

210010, SMS only, 12 months

Others felt a sense of social responsibility and altruism that was bound up with their motivations to remain in the study, perceiving the wider importance of the study to society and understanding the need to attend appointments to provide researchers with complete data. For some, this was also a matter of pride, and one man reported that he was going to come back because he was ‘a committed person’. The notion of reciprocal benefit and social contract played a role for some who remained in the study:

Yeah, absolutely because there’s a contract when you sign up for something like this, you know, how binding that contract is varies from the individual but I think because you’re providing resources to help me to achieve something I want to achieve anyway, then it seems only fair to follow through with everything that you commit to at the start.

220040, SMS + I, 12 months

Often, participants speculated that those not attending may have felt embarrassed about not achieving weight loss targets and, therefore, had decided not to attend. One participant who attended all of the study appointments and did not lose weight commented that he must have a ‘high embarrassment toleration’ in comparison with others.

Participants were positive about the strategies used in the study to maximise retention. These included appointments being offered after working hours and at convenient locations; the £20 voucher; and the appointment reminder SMS or telephone calls. Suggestions to improve retention included sending out more appointment letters and ensuring that during the baseline appointments researchers emphasised the importance of subsequent attendance, even if participants had not lost weight:

So maybe reiterating then that we need you to come back even if it doesn’t work out [. . .], which I don’t remember being told.

120030, SMS only, 12 months

Overall acceptability ratings and contamination

Table 16 displays intervention satisfaction indicators over the course of the study for participants attending assessments. Overall mean satisfaction with the intervention was 81%, 77% and 87% for the SMS + I, SMS-only and control arms, respectively. Satisfaction over time was comparable between the two intervention arms, and highest in the control arm at 12 months. Participants mostly scored 4 out of 5 on average for the programme attributes ‘understandable’, ‘useful’, ‘helpful’, ‘interesting’ and ‘relevant’.

Helpfulness ratings of the narrative SMS, web page and pedometer showed that the pedometer received the highest scores over time. At 12 months, helpfulness ratings for the pedometer were 3.9 out of 5, 3.3 out of 5 and 3.7 out of 5 for the SMS + I, SMS-only and control arms, respectively. The narrative SMS (SMS + I, 3.4/5; SMS only, 3.3/5) and the web page (SMS + I, 3.6/5; SMS only, 3.4/5; control, 3.5/5) were perceived as somewhat helpful on average at 12 months. Helpfulness ratings were relatively stable throughout the measurement time points for all three intervention components.

Acceptability ratings in the two intervention arms at 12 months were comparable across deprivation status (see Appendix 23, Table 39). Comparing SIMD category 1 and 2 participants (n = 30) with those in SIMD category 3–5 (n = 17) suggested similar mean ratings for overall satisfaction (SIMD 1 and 2, 4.1/5; SIMD 3–5, 4.1/5), helpfulness of the programme (SIMD 1 and 2, 78%; SIMD 3–5, 81%) and helpfulness of the narrative SMS (SIMD 1 and 2, 3.2/5; SIMD 3–5, 3.6/5).

The potential for contamination was assessed. One participant at 6 and 12 months (SMS + I arm) and another participant at 12 months (control arm) reported meeting other men in the study. Two men who reported meeting someone else in the study out of 78 who were asked this question at 12 months gives a contamination rate of 3%. It is unknown whether these two men met each other or met men who were lost to follow up. Documentation suggests that this meeting was likely to have occurred between 3 and 6 months.

Acceptability of narrative SMS

Acceptability was assessed through questionnaire ratings (see Table 16), the qualitative interview data and engagement in narrative SMS. Engagement was measured by examining the number of replies in response to the SMS, as well as the number of participants withdrawing or disengaging from the study because they disliked narrative SMS. Table 17 details the replies to the narrative SMS, as well as the number of participants who withdrew or disengaged from the narrative SMS component.

The majority of intervention participants remaining in the study (54/69, 78%) did not request for the narrative SMS to stop. However, some participants reported strong negative reactions. Eleven participants (16%) requested that the narrative SMS stop but chose to remain in the trial at the time. Of the eleven participants who requested to stop receiving the narrative SMS, one later withdrew from the study and one was lost to follow-up at 12 months. Requests to stop the narrative SMS were similar in both arms (SMS + I, 5/36, 14%; SMS only, 6/33, 18%), and occurred throughout the study, with at least one participant per arm requesting the narrative SMS to stop between 0 and 3 months, 3 and 6 months, and 6 and 12 months, respectively. Four participants (6%) asked to withdraw from the study completely because of the narrative SMS. Complete study withdrawal because of the narrative SMS was comparable between the arms (SMS + I, 2/36, 6%; SMS only, 2/33, 6%). All withdrawals because of the narrative SMS occurred between 0 and 3 months.

Some participants replied spontaneously to one or more of the narrative SMS received (0–3 months, 36%; 3–6 months, 12%; 6–12 months, 19%). Most replies were received between 0 and 3 months (n = 370 replies), with fewer replies sent at 3–6 months (n = 16 replies) and 6–12 months (n = 39 replies). The number of replies differed between the two arms, with those in the SMS + I arm sending more replies during the study (289 vs. 81 at 0–3 months, 10 vs. 6 at 3–6 months, and 28 vs. 11 at 6–12 months for SMS + I and SMS only, respectively).

Based on the qualitative interviews at 3 and 12 months, polarised views of the narrative SMS emerged. Participants’ views ranged from extremely positive to indifferent or extremely negative. Those with extremely negative views tended to subsequently request to stop receiving the narrative SMS. The qualitative analysis supports the quantitative findings reported in Table 16 that satisfaction with the narrative SMS did not appear to change over time.

A matrix coding query and analysis of the quotations generated from the NVivo qualitative and demographic data suggested no obvious relationships between participant SIMD category and the qualitative accounts of acceptability of the narrative SMS (see Appendix 24). This is in line with quantitative data showing similar acceptability ratings for overall programme satisfaction and helpfulness ratings of the narrative SMS across SIMD categories. Those who expressed positive views described the narrative SMS as well written and entertaining and as having some resonance with their own lives. The storyline was engaging, and some participants felt a certain camaraderie with the main character:

. . . it’s like having a friend who’s in the same boat, and you’re both trying to achieve the same thing. And that’s always helpful.

110024, SMS + I, 3 months

It appeared that the more participants reported identifying with the characters and empathising with their situations, the more likely they were to enjoy the stories and look forward to the next ‘episode’:

You get involved in the content and you start following the script, they’re very funny and me being a [from city name] you can see the funny side and you can see the wit in that . . . and that’s what keeps you reading them.

210010, SMS only, 12 months

Participants who were happy to receive the narrative SMS found the indirect nature of the prompts appealing, providing a ‘nudge’ towards healthy eating or more exercise through the actions of the characters. Being part of a storyline encouraged engagement and adoption of the suggested behaviours:

I’m more likely to read them and notice them because they are a story, rather than just an alert on the phone saying, eat properly or walk today or something. So that combination of a story with an objective and a character, a developing character I think is useful in just implanting it in the mind somewhere.

110021, SMS only, 3 months

The balance of entertaining narrative and weight loss messages was considered to be about right by those who liked and felt engaged by the narrative SMS.

Some participants welcomed the main character’s gradual realisation that he needed to lose weight. When he began to take heed of advice from his health-conscious mentor, participants perceived this as more of an incentive to follow suit and apply some of the hints and tips to their own lives. It was suggested that this should have happened from the beginning in order to maximise the benefits.

However, even those who enjoyed the narrative SMS were less certain about the role of the SMS in helping to support weight loss:

I found that I was doing that myself without . . . they were amusing, don’t get me wrong, the scripts were quite funny, but they didn’t really do anything for me in terms of motivating me.

210010, SMS only, 3 months

A matrix coding query suggested no obvious relationships between participant weight change and acceptability of the narrative SMS. For instance, exploring the views of those who lost the most weight (≥ 10% at 12 months) showed a range of views about the narrative SMS from very positive to negative.

Many participants expressed an indifference towards the narrative SMS or were unsure what the point of them was but continued to receive them as they regarded them as part of the programme. Others described how their interest in the narrative waned over time or that they would engage with them intermittently. Some would have appreciated interactive feedback and an opportunity to engage in a dialogue with the characters or storyline:

I’m thoughtful that others might not [respond to replies] and then just get switched off and not just by the story but from the whole thing of being interactive with their weight loss because this character’s not interacting with me.

220045, SMS + I, 3 months

However, even participants who were indifferent to the narrative SMS saw the value of receiving a regular reminder that they were taking part in a weight loss programme:

I often wonder, do they relate to myself here. And the only commonality is both trying to lose weight. So in that respect it’s just a reminder all the time, you should be reviewing your weight and watching what you’re eating, so . . . from that it’s a positive, yeah.

220017, SMS + I, 3 months

Those participants who did not enjoy the SMS and either ignored them or requested that they be stopped reported that the storyline did not resonate with their experience or that they did not empathise with the main characters. The characters were perceived to be stereotypical ‘fat guys’ who had gained weight by eating badly, drinking too much and being couch potatoes, whereas many of the participants had gained weight despite perceiving themselves to be active and have a good diet. Very few felt that they drank to a risky extent or drank excessively. They knew that they ate too much, but it was not necessarily unhealthy food:

I’m not a lover of chocolate, biscuits, cake, ice cream, I hardly ever touch them, hardly ever so hearing about somebody eating pizza or sweets means nothing to me because I don’t eat them anyway.

120002, SMS + I, 12 months

I paid attention to them for the first probably month, month and a half, but I found I had no relation to the guy, he just was a prat and he annoyed me.

110011, SMS only, 3 months

Other participants said that the frequency of SMS became a source of irritation that added to their perception that the SMS were not aiding their weight loss:

Three months I’ve had it, and I’m getting more than four a day, roughly four a day, so that’s going to be about 1500 texts in a year. And if they’re all like this, why? It’s not helping to lose weight, it’s not encouraging me one little bit.

220014, SMS + I, 3 months

Although some ‘wouldn’t change anything’ about the narrative SMS, others would have preferred the SMS to be based on more factual information and be more ‘encouraging’. A few participants felt that it took too long for the main character to start making health behaviour changes and they wanted to have more information about his approach to weight loss. Another suggestion was to draw on the stories of participants’ own weight loss experiences and write these into the SMS:

A person who is overweight or has been overweight for quite some time, it would be good for them to get the message over and say, yes, I was like that and I did this and this is how I feel, this is what I’m doing to keep the weight off, which is another important message.

110003, SMS, 3 months

The story-based style of the narrative SMS carried the risk of being too similar to participants’ real experiences and potentially raising distressing emotions. One example was the storyline about the impact of the main character’s possible divorce, a situation that had led to a participant’s relative demonstrating suicidal behaviour. The research fellows followed the study’s ethics protocol to ensure participant well-being.

Acceptability of endowment incentive

Two health boards were prepared to fund the financial incentives offered in the study, with agreement that the £400 endowment would be made by the University of Stirling, with money subsequently reimbursed.

At 3 months, 31 out of 36 SMS + I participants attended the appointment, with eight men losing ≥ 5% of their baseline weight and securing the £50. One of these participants who met the weight loss target at 3 months withdrew from the study for an unknown reason prior to the 6-month appointment and, consequently, did not receive any financial incentive. Attendance at 6 months was lower (n = 23/36), with five participants achieving the weight loss target of > 10% weight loss securing £150 each, and a further three participants losing between ≥ 5% and 10% of weight securing between £75 and £150. At the 12-month appointment, 23 out of 36 participants attended, with three achieving >10% weight loss securing £200, and a further seven losing between ≥5% and 10% of weight securing between £100 and £200.

Overall, participants received £2955 as a result of meeting or partially meeting weight loss targets. The full amount of £400 was secured by three participants who met weight loss targets at 3, 6 and 12 months.

Researchers did not noticed any instances of gaming in the SMS + I arm when the participants were asked to empty their pockets during the baseline assessments. Participants did not display any strong emotional reactions to not meeting weight loss targets at study appointments.

Qualitative interview evidence suggests that there were no negative views about the endowment incentive; however, only one person randomised to the SMS + I arm said that he was motivated by the financial incentive. Instead, participants appeared to stress their indifference to receiving financial gain regardless of their SIMD category, and implicit cultural and moral norms and values appeared to be associated with the question of incentives.

Participants often highlighted that the weight loss was reward enough and they were more likely to emphasise how important the health benefits associated with losing weight were to them rather than any financial gain. Motivations included reducing diabetes risk, reducing joint pain and improving mobility. Thus, men reported focusing more on increasing their physical capabilities rather than achieving weight loss targets:

The money’s not that much of an issue. I think the incentive for me is the health that, at the end of the day, your health improves. That’s the most important incentive. The money, it’s fine, it’s there, but it’s not the incentive.

120022, SMS + I, 3 months

Some participants made oblique references to the positive aspect of the incentives, often saying that this was a good idea and that having weight loss targets could be motivating. Contradictory views were expressed in interviews, when participants began by talking about their lack of interest in the financial incentive, but later expressed positive views about such incentives in an abstract sense. They appeared to demonstrate a desire to establish moral high ground by distancing themselves from the financial incentive. They also demonstrated a willingness to comment favourably on the intervention, potentially in an effort to please the researcher. Accountability at the appointment visits was raised as an incentive linked to weight loss targets:

The incentive was the result rather than money ‘cause I know that I always work better towards something if I’ve got an account at the end, you know, so I’ve got to either come here and be weighed and measured and either have successfully reached the target or not, whereas if it’s nothing, there’s no accountability then that for me is the bigger variant rather than the financial side.

220040, SMS + I, 12 months

A sense of moral value or self-worth appeared to be associated with the issue of financial incentives for weight loss. For instance, many men appeared to keep the incentive aspect of the study a secret from their friends and family. Reasons for this appeared to be mixed, including that others might believe that one is losing weight only for the money. This may have been an indication that the money was more important than participants admitted openly and that they wished to avoid potential family pressure to bring home the money. However, there may also have been some confusion in relation to the participant information leaflet, which asked men not to talk publicly about the incentives to minimise unwanted publicity and avoid contamination between arms. The ambivalent views on incentives were possibly linked to guilt around weight for some and a feeling that one does not deserve a reward for what is perceived as a self-inflicted problem. As one participant put it:

[T]here’s somebody at the background putting the money into it because they recognise there’s a problem with overweight men, particularly in my age group. So somebody’s trying to do something somewhere and that’s the main thing; there’s somebody somewhere saying ‘by the way our NHS can’t afford this overweight people coming in here all the time’.

210016, SMS + I, 3 months

Varied levels of understanding about the loss aversion aspect of the incentives emerged, whereby some fully understood the concept and others simply thought that a final reward was linked to their weight loss targets. Participants who fully understood the concept commented favourably on it, using the analogy of the difference between finding money and losing it. As one participant put it:

[W]e all know how much more distress it causes you to lose money than the pleasure of finding a fiver down the back of the couch, you know, so I think if you lose something it almost has a bigger impact on you than if you gain.

220040, SMS + I, 3 months

However, some men compared loss aversion with working towards payday or spoke of the money as being a reward. In another nod to the issue of morality and self-worth, one participant said ‘it’s not as if it’s money that I’ve earned’ (210016, SMS + I, 12 months).

Some suggestions were made for improving understanding about and impact of the loss aversion approach in future studies, such as using an animation to illustrate a dwindling sum of money linked to weigh-ins. Some suggested improving the link between the SMS and the endowment incentive:

If you get a message or two every week that was like ‘hey man £400 in this amount of time, put the crisps down . . .’, you know, the information’s there, but . . . there’s no reminder that this is what you can lose and there’s no reminder that this is what you can gain.

120017, SMS + I, 12 months

Participants who achieved or partially achieved their weight loss targets tended to have plans for the money. Some men had charitable intentions; for example, one participant planned to give some of the money to a friend who was ‘not doing financially well’, whereas another pledged money to a charity as he felt that he had already ‘had a benefit out it [the study]’ and that ‘somebody else can get the benefit’ from the incentive. Others either had already ‘spent it’ or planned to spend the money on new clothes, as their current clothes had become ill-fitting after their weight loss. Another participant was determined not to waste the money and had decided to ‘treat himself’ to reward his successful efforts to lose weight:

I’m certainly not going to just put it in the bank and fritter it away, I’m actually going to buy something that actually is a celebration of the study over the last 12 months. So yeah it might be a kilt, it might just be a laptop [laugh].

220045, SMS + I, 12 months

Acceptability of study web page

Table 17 displays engagement with the web page in terms of the number of participants who used the different self-monitoring features and the number of self-monitoring features that were used over time. Some participants used the self-monitoring features (0–3 months, 27%; 3–6 months, 16%; 6–12 months, 13%). Among those engaging in self-monitoring, recording of weight and of step count were the most popular web page features, with frequency of recording decreasing over time. Recording waist circumference and belt notches was uncommon.

Qualitative evidence from participants who used the web pages was that they enjoyed being able to visually track their weight on the website and found that this motivated them to keep working towards their weight loss targets:

. . . it’s like having targets, you see it starting to change ‘oh yes I’ve got to keep going’. So yes I found the website more beneficial than the text messages.

210010, SMS only, 3 months

However, others found the appeal short-lived, particularly when their weight loss progress was not reflected in the self-monitoring graph:

What I didn’t like about logging my weight on the Game of Stones website was the increments. I mean, my line looks like that and, although I’ve lost all this weight, when you look at my line on the site it looks like they’re seeing it as more of incline. [. . .] When you’ve got a visual thing and you’re putting in a loss of, say, two kilos a week or something like that, or a kilo a week, and you’ve logging it every day, you like to see the line coming down, you know.

110019, SMS only, 3 months

Others did not access the website because they had low computer literacy, did not have time to use it or found it hard to navigate. Another participant suggested that the web page would be more appealing if it had more content, such as nutrition and exercise plans. On balance, the web page is something that works well for those with computer literacy and can be a motivator. However, men clearly expressed the view that more work could be done to improve the access (e.g. speed and number of clicks required to self-monitor) and content of the website.

Acceptability of pedometer

The majority (77%) of participants did not own a pedometer at baseline and all were given a pedometer as part of the study. At 12 months, many intervention arm participants reported using a pedometer at least a few times per week (SMS + I, 59%; SMS only, 58%). Around half (47%) of control arm participants reported using a pedometer weekly.

Qualitative views suggested that many participants were very positive about the pedometer, describing it as ‘helpful’, ‘interesting’, ‘ideal’ or ‘fantastic’. This is in line with the quantitative helpfulness ratings, as the pedometer received the highest average ratings of all of the intervention components. Some participants described how they used the pedometer every day. Noticing that their step count was lower than normal gave some men ‘a wee kick in the rear’ or started to get ‘embarrassing’, prompting them to make a ‘conscious effort to go on a walk’ or to ‘try and up it the next day’:

The step counter, I carried that absolutely everywhere with me and I’ve found that a great benefit, something small and simple like that being such a great benefit.

210010, SMS only, 3 months

Conversely, some had not used the step counter or had decided that they were ‘not great with pedometers’ after trying it initially. One participant reported that the pedometer had not been taken out of its packaging (210004, researcher field notes). In some cases, participants reported that they had not used the pedometer because a family member had appropriated it for their own use. Others used a step counter on their mobile phone or an app other than the one provided. Some men noted that the pedometer was ‘not 100% accurate’ and that there was ‘quite a disparity’ between the pedometer provided and other step counters. However, regular users of the study pedometer felt that it was ‘cheap and cheerful’ and provided a ‘good indication’ of steps:

I’m pretty damn sure it’s not accurate but it doesn’t matter, it’s consistent and it’s more accurate than colleagues who swank around with their iPhones [Apple Inc., Cupertino, CA, USA].

110021, SMS only, 3 months

Health behaviours

Participants provided outcomes on physical activity, sedentary behaviour, fruit and vegetable consumption, alcohol consumption and smoking status (see Appendix 26, Table 42). Small changes in the intended direction over time emerged for physical activity, fruit and vegetable consumption and alcohol consumption, with little change in smoking status and sedentary behaviours.

Weight management strategies

Participants reported using a variety of weight management strategies (see Appendix 27, Table 43). Few participants reported attending a weight reduction clinic (no attendance at 12 months: SMS + I, 96%; SMS only, 92%; control, 90%) or a commercial weight loss programme (no attendance at 12 months: SMS + I, 96%; SMS only, 92%; control, 93%) in addition to the Game of Stones programme. Modest and inconsistent increases in strategy use emerged in line with the non-prescriptive approach of the intervention.

Theory-based mediators

Theory-based mediators were measured, with acceptable rates of completion (see Appendices 28–30, Tables 33–35). Small positive changes in the expected direction were noted for several variables. Autonomous regulation (i.e. performing behaviours for internal reasons) remained relatively stable throughout the study across intervention arms over time and remained comparable at 12 months (SMS + I 6.1, SD 0.9; SMS only 5.7, SD 1.1; control 5.8, SD 1.3). Controlled regulation (i.e. performing behaviours for external reasons) in the SMS + I arm did not increase substantially over time (baseline 3.6, SD 1.6; 3 months 3.9, SD 1.4; 6 months 3.6, SD 1.5; 12 months 3.7, SD 1.5). At 12 months, the three arms remained comparable on levels of controlled regulation (SMS + I 3.7, SD 1.5; SMS only 3.8, SD 1.5; control 4.0, SD 1.6).

Completeness of measures for economic evaluation

Of the 105 participants randomised, 57 (54%) completed resource use questions at all the time points required to estimate the costs over the trial period. Note that the required time points varied by trial arm. Complete data were required at 3, 6 and 12 months for the intervention arms, but only at 12 months for the control arm. The majority of missing data resulted from participants not completing the questionnaires at all at the required time points (n = 44, 42%). Among the 71 participants who completed the questionnaires at all required time points, one or more resource use items were missing for 14 participants. For 4 of those 14 participants, resource use was missing at all of the required time points. There were no invalid answers to the resource use questions. For further details, see Appendix 31, Table 47.

Of the 105 randomised participants, 66 (63%) completed the EQ-5D-5L at all the time points required to estimate QALYs over the trial period. Note that the required time points varied by trial arm. Complete data were required at baseline and at 6 and 12 months for the intervention arms, but only at baseline and 12 months for the control arm.

The majority of missing data resulted from participants not completing the questionnaires at the required time points (n = 34, 32%). Of the 71 participants who completed the questionnaires, five were missing one or more EQ-5D-5L responses. At 6 and 12 months, all domains were completed at the same rate. There is some variation across domains at baseline, but no clear pattern. For further details, see Appendix 32, Table 48.

Completers and non-completers of resource use and EQ-5D-5L questions did not vary systematically in terms of age, SIMD category, recruitment method, ethnicity, baseline BMI, age, comorbidity, intended weight loss, importance of weight loss and previous weight loss attempts.

Costs and quality-adjusted life-years

The intervention delivery costs are reported in Table 19 and are split into fixed and variable costs. The fixed cost includes items such as hosting the network, the study website and equipment. The total fixed cost is £17,526, which equates to £254 per participant randomised to the interventions.

The costs associated with weighing the participants at baseline and at 3, 6 and 12 months are included as these are considered integral components of the interventions rather than part of the study procedures. The baseline visit is estimated to be longer to allow time to explain the interventions and the final weigh-in at 12 months also includes a debrief.

Participants in the SMS + I arm could secure up to £400. In total, 11 participants secured money and completed the study. The average amount of money received by these 11 participants was £268.84. The average cost per participant randomised to SMS + I was £81.94 (95% CI £34.59 to £129.30). In terms of narrative SMS, participants received up to 604 messages; however, some participants opted out and the average number of SMS per participant randomised to the intervention arms was 519. Among observed cases, the average total interventions costs were £333.05 for SMS and £465.87 for SMS + I.

NHS resource use

Table 20 shows the resource use by trial arm at the different time points for the participants who completed all resource use questions to allow an estimation of the NHS costs over the trial period (N = 57; control, n = 26; SMS, n = 17; SMS + I, n = 14). This illustrates the feasibility of collecting resource use data. The numbers are small but are reported for completeness and in case other researchers want to use these estimates in future. Most notable are the high-use outliers in the control arm. One participant in this arm reported 20 days of inpatient stay and another participant reported 24 days (as well as 20 outpatient appointments). Outliers in terms of high health-care users are common in trials and unlikely to be directly related to the interventions. Outliers do not pose a problem as long as they are evenly distributed across trial arms, which is more likely to be the case in large trials.

Resource use varied considerably during the trial period across all arms of the trial (see Appendix 33, Table 49). The mean total cost was highest for the control arm, which is likely to have been caused by the outliers. It should be noted that the impact of the outliers is magnified as the resource use during months 9–12 is multiplied by four to obtain an estimate of resource use during the trial period for the control arm. The mean cost in the SMS + I arm is higher than the mean cost in the SMS-only arm. This is mainly because of differences in inpatient stays and outpatient appointments.

The narrative SMS and alternative SMS compared

The timing and frequency of both the narrative SMS and the alternative SMS was a key factor and it was noted that just getting a short ‘have a nice weekend’ message was enough to jog the memory and retain the focus on the weight management aim. There was some ambivalence around whether the actual content of the SMS (whether narrative or alternative style) effected a change in behaviour or whether the messages simply served as a reminder of the intention to lose weight.

Both styles showed promise. The narrative SMS appeared to generate stronger, more polarised views because the storyline either resonated with men’s lives or did not. Men who were unable to empathise with the characters found the messages irritating and either ignored, deleted or blocked them. This suggests that storylines with wider social relevance may have wider appeal. However, no clear link between deprivation levels and preference for styles of SMS was found among the small number of men interviewed. Although it is possible to compare the two styles only on an interpretive level, it is clear that, whereas the narrative SMS attracted positive and negative reactions, the alternative SMS attracted more muted responses, with no very strong negative or positive reactions.

Strengths

This study was underpinned by the ROMEO systematic reviews and qualitative evidence synthesis of weight loss interventions for men with obesity, led by co-investigator Alison Avenell,49,88 which commenced a continuing PPI partnership with the Men’s Health Forum GB and Ireland charities. Pat Hoddinott led the qualitative evidence synthesis for ROMEO. The interventions incorporated key findings from these reviews, in particular the need for men in the target population to be involved in programme design, and gender-specific features to facilitate engagement such as the use of humour. The breadth and diversity of PPI, including Study Steering Committee guidance from the directors of ManvFat and The Alliance, helped to make the study a success.

Mixed-methods research combined process, quantitative and qualitative data, which, together with co-production and stakeholder involvement, provided a multilens perspective on feasibility and acceptability outcomes important to men. This resulted in intervention components that were novel, appealed to men including those living in disadvantaged areas and engaged men with a broad range of sociodemographic and health characteristics. Novel narrative SMS were developed based on the principles and practices of a professional commercial scriptwriter with research qualifications and experience. Endowment incentives were systematically developed based on behavioural economics and informed by a DCE with a survey of 1045 men with obesity representative of the UK population.

A workshop with men in the target group, clinicians, policy-makers, topic experts and the research team finalised the trial design. Recruitment of men focused on areas with high levels of disadvantage. The higher retention of men from those living in disadvantaged areas suggests that intervention components have the potential to positively affect health inequalities. The majority of men reported one or more obesity-related health conditions, suggesting that the intervention is acceptable for both primary and secondary prevention of obesity-related morbidity.

Two changes to the original protocol demonstrate the strength of the iterative approach undertaken to optimise intervention components in this feasibility study. Instead of hypothetical qualitative interviews in phase 1 to ascertain the acceptability of the narrative SMS, actual experiences from trial participants were sought in semistructured interviews at 3 months. Following some early negative feedback about the narrative SMS, the majority of men from both intervention arms, rather than a smaller sample, were interviewed so that a diversity of perspectives could be heard. The early finding that a narrative style of SMS did not suit all men, and men’s suggestions for changes, meant that new alternative SMS were written that maximised learning through analysis of data. Being open about limitations, analysing data and then refining interventions is important in intervention development studies to maximise the likelihood of broad reach and implementation in the future. 89

There is potential for centralised remote delivery of SMS linked to online information sources. One of the research assistants assessing men had worked as an NHS band 3 weight loss programme assistant prior to this study. Efficient incentive design linked to weight loss targets, with weight confirmed by the equivalent of NHS band 3 staff in health centre or community venues, shows promise for scale-up and future implementation, if shown to be effective in a full trial.

Weaknesses

Researchers were unable to interview men who withdrew from the study or were lost to follow-up at the 12-month appointment, but did find out brief withdrawal reasons for most participants through communication by SMS or over the telephone. Those providing a withdrawal reason may have provided socially desirable responses; for example, no participants from the SMS + I arm mentioned that they withdrew because they had failed to meet weight loss targets and missed out on a financial incentive.

SMS were personalised by including the participant’s name and weight change, but no further tailoring was possible owing to the technical limitations of the delivery system. The narrative SMS combined many information sources underpinned by differing principles and approaches and demanded artistic and creative freedom. Although leading to novel and imaginative intervention content, the development approach and the detailed social storyline have the potential to dilute exposure to evidence-based BCTs. The narrative SMS were developed iteratively, which was partially informed by the SMS responses from some participants. Replying to the SMS was optional, not all men replied and the views of non-replying participants were not considered. The lack of interactivity in response to participant replies to the SMS was a further limitation, as pointed out in qualitative interviews with study participants.

A comparison between alternative SMS and narrative SMS-only participants is limited because of differences in participant time since enrolment, intervention length and randomisation status. The content of SMS replies were not examined in depth by the qualitative research team.

The progression criterion of 12-month outcomes on at least 72% of men randomised per arm could be seen as overly sharp. The somewhat ambitious nature of the target was taken into consideration when the Study Steering Committee made recommendations for study progression to a full trial. Future studies should consider using progression criteria based on a prespecified range, such as proceed when > 75% retention, review procedures when between 60% and 75% retention, and stop the study when < 60% retention.

Owing to technical limitations, the proposed online recording of questionnaire responses and automatic upload of weight outcomes via Bluetooth-enabled (Bluetooth SIG, Inc., Kirkland, WA, USA) scales could not be implemented. The completion of online questionnaires was tested at baseline, but owing to unreliable internet connections at appointment venues the risk of missing data was high. Instead, participants completed paper-based questionnaires during the appointment and initialled the weight recorded by the research assistant to confirm its accuracy. Additional researcher time was then required for them to manually enter the data.

The study received some unintended attention in the national media around financial incentives (two newspaper articles) triggered by information in the lay summary and protocol published on the NIHR website. This may have had an impact on recruitment and attitudes towards the incentives. Every effort should be taken to avoid this happening in a full trial as it has the potential to bias the results.

Recruitment

In Game of Stones, recruitment of 105 men with obesity within 4 months was feasible through both community outreach and general practice. The MACRO study14 examining a narrative SMS intervention recruited 69 men with obesity who drank > 21 units of alcohol per week in 3 months through community outreach and general practice list invitations. Another Scottish feasibility study13 examining a SMS intervention used community and general practice recruitment strategies and recruited 67 men who had had two or more binge drinking episodes in the preceding month. This recruitment strategy was, subsequently, employed in a multicentre RCT15 that consisted of four Scottish sites recruiting 825 men, demonstrating the generalisability across various contexts. Game of Stones adds to the evidence base demonstrating the feasibility of recruiting men for research on sensitive subjects such as obesity and alcohol consumption through a combination of community outreach and general practice lists.

Sample composition

Game of Stones included 59% of participants from more disadvantaged areas of Scotland, defined as areas in the SIMD 1 or 2 quintile. 69 In comparison, two Scottish men-only weight management studies recruited lower proportions of participants from SIMD category 1 or 2 areas. The FFIT study7 used community recruitment activities, and 35% of the participants randomised were from disadvantaged backgrounds. Irvine et al. 21 recruited 44% of participants from SIMD category 1 or 2 areas to a SMS intervention, including a face-to-face component, using similar recruitment strategies to those in Game of Stones. Two large English weight management studies delivered in primary care96,97 included a minority of men and recruited participants who were more affluent, on average, than the English population. Aveyard et al. 96 randomised 1882 participants (43% men) to an advice or a support group, with average Index of Multiple Deprivation scores of 15.7 and 16.4, respectively. Little et al. 97 randomised 818 participants (36% men) to a web-based intervention with face-to-face or remote nurse support or a control group, with average Index of Multiple Deprivation scores of 13.7, 13.3 and 14.3, respectively.

The BMI of around 35 kg/m² and age of around 50–55 years of the Game of Stones participants is similar to those of participants in UK weight management trials recruiting in the community,7 primary care,96 or a combination of community and primary care. 14 Three international SMS-based weight management studies examining outcomes after 12 months recruited younger participants with lower BMI. 90–92 However, three mixed-sex weight management studies with financial incentives (two including SMS components) reported broadly similar participant demographics to those in Game of Stones. 93–95

The current sample reported average weight loss expectations at baseline that could be considered unrealistic. NICE recommends setting realistic targets for weight loss; for adults, the targets are usually a maximum weekly weight loss of 0.5–1 kg, with the overall aim to lose 5–10% of original weight. 6 The Game of Stones weight loss targets of losing 5% by 3 months and losing 10% by 6 months are in line with NICE recommendations. The relationship between initial weight loss expectations and weight loss outcomes is complex. 98 A previous study found that weight loss was higher in those with high weight loss expectations who also engaged in the BCTs by formulating specific plans. 99 Future studies need to balance participant ambition with the potential weight loss that can be achieved realistically using a self-management intervention.

Sample composition by recruitment channel

Differences in sample composition emerged between the two recruitment channels in Game of Stones. GP recruits were more likely to live in disadvantaged areas, were older, were more likely to report having a comorbidity and had a lower BMI than community recruits. Community recruitment may have more potential than GP recruitment for the primary prevention of obesity-related morbidity. The employment status of the two cohorts was broadly comparable, potentially reflecting the recruitment activity that took place at evenings and weekends to ensure that men in full-time employment were represented in the study.

Differences between community and GP recruits are comparable with those in other studies where these recruitment channels have been used. In the MACRO study,14 men with obesity recruited in the community also had a higher average BMI, and more individuals from disadvantaged backgrounds were recruited through GP obesity registers than were recruited from the community. GP recruitment in MACRO resulted in 19 out of 32 participants (59%) from SIMD category 1 or 2 areas being randomised to the study, compared with 8 out of 30 (27%) from community recruitment. Game of Stones found the same difference, albeit less pronounced (SIMD 1 or 2: community 56% vs. GP 64%). Thus, evidence suggests that community outreach activities attract participants across the socioeconomic spectrum, whereas GP recruitment is most suitable for the focused recruitment of participants living in areas of high disadvantage.

Retention

Retention levels in Game of Stones were acceptable, with 74% of the 105 men completing the 12 months’ follow-up. Differential dropout rates between the study arms were recorded, with fewer SMS + I participants completing the study (64%) than SMS-only (79%) or control arm participants (83%). Retention in the control arm was highest. Several factors contributed: a continuing desire to lose weight after the 12-month appointment, the wish to receive the 3 months of SMS, the follow-up visit with the researcher and the £20 voucher for attending follow-up. Most of the control arm participants who attended the 12-month appointment took up the offer of the alternative SMS (29/30). There was minimal information about the incentives (and no mention of the amount of money linked to weight loss targets) in the pre-randomisation information materials, and this aspect of the study was not emphasised during recruitment. The finding that retention in the SMS-only and control arms was higher than in the SMS + I arm suggests that participants did not volunteer for the study to receive the financial incentive and stayed in the trial when randomised to a non-incentive arm; therefore, disappointment bias was largely avoided, as intended.

The participant retention rate of 74% in Game of Stones is similar to systematic review evidence of men-only weight loss interventions, which found an average retention rate of 78%, ranging from 44% to 100%. 88 The men-only FFIT study7 that recruited Premier League Club football fans achieved a higher retention rate, with 89% of participants in the intervention group and 95% in the control group remaining in the study at 12 months. Similarly, the MACRO study,14 consisting of a 3-month SMS intervention for men with obesity, achieved 98% retention overall after 5 months. Two large UK studies recruiting participants of either sex with obesity to weight management in primary care reported 75%96 and 81%97 retention at 12 months.

Retention levels in Game of Stones are similar to those in international SMS-delivered intervention studies for weight management in men and women. Three RCTs reported retention rates after 12 months following a SMS weight management intervention. A Finnish study90 reported 68% retention of 125 participants (intervention, 73%; control, 64%), and a US study91 retained 76% of 170 participants (intervention, 71%; control, 82%). One Latvian study92 showed superior retention, with 95% of 123 participants completing the 12-month study.

Retention by recruitment channel

Retention differed slightly by recruitment channel, with more participants recruited from general practices completing the study (general practice, 80%; community, 71%). Interview data suggest that recruitment by GPs and assessments at general practices could increase participant retention because of the perceived endorsement of the study and the familiarity of the venue. The study data and the PPI and stakeholder feedback suggest that the difference in site retention might be due to multiple factors, such as population composition, number of local research activities or familiarity with those seen to be endorsing the study by providing venues for assessments.

Crombie et al. ’s TRAM study15 found that 93% of participants recruited through general practice registers were followed up at 12 months, compared with 77% of community recruits. Taken together with Game of Stones, this suggests that between 11% and 16% more participants complete a 12-month behaviour change intervention delivered by SMS when recruited from general practice registers as opposed to when recruited from the community.

Retention by baseline characteristics

Differences between participants completing the 12-month assessment and those withdrawing or not returning for assessment emerged, although caution is required as numbers were small. Non-completers were slightly heavier and intended to lose more weight through the programme. The modest levels of weight loss typically achieved through self-management interventions such as this programme8,18 may have left some men with greater weight loss expectations, or with higher levels of obesity, to feel disappointed at a perceived lack of progress. One of the participants who withdrew from the study, who had a BMI of > 50 kg/m², reported multiple withdrawal reasons including a perceived lack of weight loss success. Social desirability may have prevented others reporting lack of weight loss as a withdrawal reason.

More participants from disadvantaged backgrounds remained in the study, suggesting potential for the intervention to make a positive impact on health inequalities. This could be random and there were no demographic patterns in relation to retention or intervention acceptability observed in the qualitative data analysis. Recruiting from disadvantaged areas can increase the likelihood of drop-out due to additional challenges involved in maintaining contact with participants such as changing contact information or the research competing with more pressing life priorities. 100

Game of Stones and a recent study by Crombie et al. 15 demonstrate that high retention of men from disadvantaged backgrounds participating in SMS-delivered intervention studies is possible. Crombie et al. 15 reported a retention rate at 12 months of 86% (intervention, 85%; control, 87%) for a 3-month narrative SMS intervention to reduce alcohol consumption delivered to a sample including 77% of participants from SIMD category 1 or 2 areas. Overall, the observed retention rates for men living in disadvantaged areas demonstrate promise for SMS interventions to engage men from across the socioeconomic spectrum.

Retention and relationship with the researcher

From a participant perspective, appointments with researchers for measurements, questionnaires or qualitative interviews constitute ‘an intervention’ as they would not happen in usual life outside the research context. 101,102 Nine out of 11 participants that requested to stop the narrative SMS remained in the study and attended a 12-month appointment. This indicates that these men valued other components and being part of research aside from the SMS. Qualitative findings from Game of Stones suggest that there is uncertainty about how the relationship with the researcher at the 3- and 6-month assessments affected trial retention and outcomes at 12 months. The direction of any impact cannot be assumed. The decision to undertake qualitative interviews at 3 months in the intervention arms to assess trial participants’ actual experience of receiving the narrative SMS affected the relationship with the researcher. The open questioning and active listening in qualitative interviewing contributed to supportive accountability,103 as researcher field notes documented that some participants were disappointed that the 6-month assessments were so brief. Some men valued the opportunity to share their experiences. This has implications for a full trial, where the 3- and 6-month assessments for incentive intervention linked to weight loss targets will be standardised and no qualitative interviews will take place.

Acceptability of narrative SMS

Narrative SMS were acceptable to most participants who were positive or indifferent to them, with a minority of men showing negative reactions. Two previous narrative SMS intervention studies in men targeting alcohol reduction reported high acceptability levels. The MACRO pilot RCT14 and TRAM RCT15 report no dropout or requests to stop SMS. Moreover, in both studies a high level of responses to narrative SMS were reported, with 92% of participants responding to at least one SMS at 3 months respectively, compared with 32% in Game of Stones. Both narrative SMS interventions differ from Game of Stones as they were 12 weeks in duration, contained fewer overall SMS (at 12 weeks, MACRO, n = 95; TRAM, n = 112; Game of Stones, n = 182), focused on alcohol consumption as the target behaviour and were developed closely in line with a single behaviour change theory. Moreover, both the SMS of the MACRO pilot14 and the TRAM full RCT15 interspersed narrative elements with factual information and questions that were outside the narrative story. One possibility is that narrative SMS may be more universally acceptable when used as an adjunct to, rather than the main vehicle for, intervention content delivery.

There were no obvious relationships between participant weight change and acceptability of the narrative SMS when analysing the mixed-methods data. Social comparison theory104 proposes that people can be motivated to change their behaviours through upwards and downwards comparisons with their peers. The qualitative findings from Game of Stones suggest that both of these comparisons were taking place for men during the narrative SMS.

Acceptability of financial incentives

The use of a financial incentive strategy for weight loss in men with obesity living in disadvantaged areas was feasible and acceptable. None of the men withdrew from the study because they disliked the incentives. The findings on financial incentives are in line with the PPI work that the research team undertook in preparation for the study, where the incentive component was seen as ‘probably not for everyone, but very important for some’.

As in Game of Stones, two previous studies have used SMS to inform participants about their achievement of weight loss targets, and found it acceptable. 93,95 These studies used daily SMS on weight loss and incentive achievements, compared with three SMS sent over 12 months following face-to-face appointments in Game of Stones. More frequent SMS reminders related to incentives might have increased the visibility of this intervention component. However, in Game of Stones, the decision was taken to keep the narrative SMS intervention component completely separate from the incentive intervention component, with no cross-referencing between the two. As incentive values remained the same between appointments and were available on the study website, and because participants already received frequent and relatively intensive narrative SMS, this was not necessary.

Game of Stones demonstrated that financial incentives could be used alongside an SMS intervention component containing no incentive-related elements. This extends the evidence base on the use of financial incentives as a complementary behaviour change intervention to the other components. 16,29

The financial incentive strategy was designed to have a low administrative burden, with one payout after 12 months and weight loss goals for participants that were set for a minimum of 3-monthly intervals. To objectively verify weight loss during the 12-month period, face-to-face appointments were scheduled for 3, 6 and 12 months. The selection of these time points were informed by the DCE/survey. Yancy et al. ,95 for example, provided the incentives that participants accrued through programme engagement and weight loss at 3 and 6 months for their 6-month incentive intervention, similar to the current study. John et al. 93 used more frequent follow-up schedules, with participants returning monthly to be weighed, for which they received US$20 per visit. Few RCTs have examined financial incentive strategies that are delivered for ≥ 12 months. 16 Two older studies96,105 providing financial incentives for ≥ 12 months integrated the provision of the financial incentives into weekly106 or monthly105 face-to-face meeting schedules, thus providing a somewhat more intensive intervention than Game of Stones.

Overall, 12 men secured money; 11 men received the money when they attended the 12-month appointment (study completers) and, of those, three secured the full £400. Two out of three men failed to reach the 5% weight loss target at any time point and, therefore, did not secure any money. A balance needs to be struck between weight loss targets that are ambitious enough to incentivise meaningful weight loss and targets that are achievable to avoid disappointment and withdrawal from the study. Previous studies typically have provided participants with weekly weight loss goals, such as 1 lb (0.45 kg) per week, on which financial incentives are contingent. 16 Although these goals are similar to the weight loss targets that participants were given in Game of Stones, there were no intermittent or weekly targets. A future option might be to reward lower levels of weight loss, particularly at 3 months, when, in Game of Stones, money was secured only if participants had met the overall 5% target. Rewarding any weight loss at 3 months might reinforce early weight loss achievements. Although retention was lowest in the SMS + I arm (64%), seven of the eight men who met their 3-month weight loss target went on to complete the study. A balance needs to be struck between encouraging engagement, rewarding progress and achieving meaningful and clinically significant weight loss targets.

The financial incentive strategy was designed to invoke loss aversion in a similar way to deposit contracts, whereby participants deposit their own money, which is refunded contingent on attaining weight loss goals. Deposit contracts are one of the most common strategies in financial incentive weight management interventions. 16,58 Game of Stones did not require participants to commit their own money; this was to avoid barriers to study participation, particularly from men who may experience financial constraints. A hypothetical cheque was provided to make the money seem ‘real’ for participants – a precondition for loss aversion effects to influence weight loss motivation and efforts. There was mixed evidence as to whether or not this was achieved. The qualitative evidence from this study suggests that whereas some men understood the endowment incentive, to others this was not clear. The use of visuals and more frequent reminders may improve this aspect of the financial incentive scheme.

Loss aversion framing has been used by a previous study examining the addition of financial benefits to a web-based behavioural programme. Leahey et al. 94 included a ‘bank’ web page that displayed financial incentive earnings in addition to loss aversion framed reminders. Patel et al. 36 applied a similar framing, which they called a loss incentive. University employees were allocated a monthly incentive of US$42 upfront, and US$1.40 was taken away each time their daily goal of 7000 steps was not met. The scheme was provided through the employer, making it likely that participants believed that the money was allocated.

Based on the DCE results, both the participant information and the researcher discussions at study enrolment mentioned that incentives could either be kept by the participant or donated to a charity of their choice. However, direct discussion with research assistants raising the option of a charity donation had the potential to place men in a moral dilemma, creating discomfort and potentially straining relations with the study and the researcher. To avoid this, researchers did not mention the option of charity donations in later appointments. Care should be taken in a future trial to make clear that participants can do anything they want with any financial incentive secured.

Previous evidence has suggested that the provision of financial incentives may undermine motivational quality. 107 This was not found in the current study. The SMS + I arm showed no increase in average levels of controlled regulation, and levels of autonomous regulation remained high and stable. Qualitative evidence suggested that participants frequently reported their internal motivation for losing weight. The endowment incentive did not appear to undermine the quality and strength of participants’ motivation to lose weight. This is in line with a systematic review that examined whether or not the provision of extrinsic incentives eroded intrinsic motivation (183 studies; 212,468 participants), which concluded that ‘incentives and intrinsic motivation are not necessarily antagonistic and are best considered simultaneously’. 108

Acceptability of other intervention components

The low-cost intervention website was acceptable and used by some to self-monitor behaviour and weight, but most did not engage with this component. Compared with web-based weight management studies,97 the Game of Stones interventions offered the web page as an adjunct to, rather than the main vehicle for, delivering intervention content, but more effective engagement109 with this study element could be promoted in the future. This could be achieved by engaging website designers to improve the usability and appeal, as some participants disengaged because they found navigating the web pages and information time-consuming.

The study pedometer was the most popular intervention component based on helpfulness ratings. The majority of participants (76%) reported that they had not owned a pedometer prior to the study, underlining the relevance of including this when recruiting men living in disadvantaged areas. Previous research has also found pedometers acceptable as a behaviour change tool,110,111 including for weight loss interventions for men. 88

Acceptability of waiting list control

The acceptability of the waiting list control and the high retention rates in this arm suggest that no change is required in a future full trial. In the light of the qualitative findings relating to social accountability arising from the participant–researcher relationship, the decision to only assess outcomes at baseline and 12 months is endorsed and provides as close to ‘life as usual’ over 12 months as is possible in a RCT.

Feasibility of study procedures

Feasibility of intervention delivery was demonstrated by high intervention fidelity. No major technical errors occurred. The majority of the sent SMS (95%) were returned with a successful delivery status, similar to previous SMS-based intervention studies. 13–15 Numbers of missing data were acceptable once the initial online questionnaire completion had been abandoned. The selected cost and outcomes measures for the cost-effectiveness analysis were also feasible, with missing data mainly resulting from questions being overlooked by participants when completing self-report questionnaires.

Of the 105 participants randomised, 57 (54%) completed resource use at all of the time points required to estimate the costs over the trial period. There are two limitations to the resource use measures used in this study. First, no data were collected on medications, which may differ across intervention arms; for example, Hunt et al. 7 found lower medication costs (antidepressants, painkillers, asthma, pain gels/creams, anti-inflammatories and sleeping tablets) among participants in the FFIT intervention than among those in the control group. Excluding medication costs may, therefore, result in more conservative estimates of the cost-effectiveness of the interventions. A full trial should consider using routine data to collect information on medication use and other resource use in the NHS for each participant. Men found routine data linkage acceptable in this study once a video explaining the rationale for anonymisation had been provided. Second, resource use in the control group was only measured for months 9–12 during the trial follow-up. Assumptions therefore had to be made to estimate the cost for the full trial period. Outliers that occur during this period are magnified and can bias results.

Two men reported that they had met other men in the programme, suggesting that the recruitment strategies used are unlikely to lead to high levels of contamination.

No adverse events were encountered in this study, similar to a previous narrative SMS study targeting obesity in disadvantaged men. 14

Indicative effects

The Game of Stones study was designed to assess the feasibility of outcome data collection and is not powered to detect effects on weight loss, so study and weight outcomes should be interpreted with caution. Favourable indicative effects of weight loss in the Game of Stones intervention arms support a fully powered RCT. The MACRO14 pilot RCT that aimed to reduce obesity by targeting alcohol consumption in men reported an increase of 0.4 kg in the intervention group, which received narrative SMS that were considered highly acceptable.

In full-scale trials, weight loss in the men-only FFIT trial delivered face-to-face at professional football grounds was 5·56 kg (95% CI 4.70 to 6.43 kg) at 12 months. 7 Another recent UK trial delivered to mixed-sex participants, examining a brief intervention delivered in primary care, reported –2.43 kg (SD 5.16 kg) in the intervention group. 96 A UK trial of a 6-month web-based weight management intervention with either face-to-face or remote nurse support found that participants receiving web support alone lost almost 3 kg at 12 months, with those receiving face-to-face nurse or remote nurse support showing an additional weight reduction of 1.5 kg (95% CI 0.6 to 2.4 kg) and 1.3 kg (0.34 to 2.2 kg). 97 International SMS intervention studies reporting weight loss outcomes following 12 months of SMS have found weight loss of –4.5,90 –1.6591 and –2.40 kg. 92

The evidence for the effectiveness of financial incentives for weight loss and weight loss maintenance when added to behaviour change interventions remains mixed. 16,29,58 A recent trial95 examining the effects of lottery or direct financial incentives found no additional benefit to daily self-weighing and SMS feedback interventions on weight loss maintenance at 12 months [lottery −1.8 (SD 10.5) kg; direct −0.7 (SD 10.7) kg; and control −0.3 (SD 9.4) kg]. Another study94 examined the addition of small financial incentives to a web-based behavioural intervention, reporting higher weight loss at 12 months [web + incentives: –3.1% (95% CI 1.8% to 4.4%); web only: –1.2% (95% CI 0.1% to 2.6%)].

SMS content and delivery

Tailoring SMS content and delivery might increase intervention engagement and improve study retention. Many men reported in phases 1 and 2 that they would have preferred fewer SMS or to receive the SMS at specific times only. A greater focus on weight management topics, including the use of factual information and links to sources of information, was suggested by several men. A future fully powered trial will require SMS that are optimally acceptable, are delivered over 12 months, and balance tailoring options important to men with the delivery of key active ingredients specified in the underlying study logic model and supported by evidence. In addition to tailoring, interactivity was an element desired by several participants, but this was not possible in the context of the underlying SMS system. Future studies should explore including interactivity and feedback elements in SMS.

Based on study findings and current evidence on SMS for weight management, SMS interventions can engage men in weight management. The mechanisms of action for the features of SMS and their relationship to acceptability and effectiveness warrant further research.

Endowment incentive strategy

In the SMS + I arm, retention was lowest and indicative effects on weight loss were highest. Findings suggest several possible reasons, although the numbers were small and the differential dropout might have been random. Disappointment at not achieving weight loss targets and losing the incentive could have been amplified for some by the loss framing of the intervention or the lack of perceived helpfulness of the narrative SMS. Social desirability, disappointment or social accountability to the researcher might be influences. Incentives were not prominent in recruitment materials, so some participants might have been indifferent to the intervention, particularly those who did not report that the incentives motivated them. PPI suggests that future iterations of the endowment incentive strategy could introduce incremental incentives for each 1% of weight loss between 3% and 5% at 3 months to improve motivation and reduce disappointment for some men. Given the controversy around financial incentives as an intervention, members of the public, service providers and policy-makers could find the provision of incentives for weight loss below the clinical significance level of 3% unacceptable. During the design of the incentive schedule for weight loss in phase 1, careful attention was paid to a variety of elements, such as the incentive size, spacing, presentation and weight loss targets, to balance the motivational potential of incentives with avoiding undue disappointment. There was no strong evidence to suggest that any of the elements that were carefully selected in the current study induced unintended consequences.

In a future trial, careful attention will be required to explore whether or not endowment incentives differentially influence weight loss and retention at 12 months according to participant level of disadvantage. Men in the SMS + I arm from more disadvantaged areas were more likely to stay in the study for 12 months than, but did not lose as much weight on average as, 12-month appointment attendees from less disadvantaged areas; however, the numbers are small. Effectiveness is known to be the key determinant of public acceptability overall,66 but this might be compromised if incentive interventions favour more advantaged individuals.

There is a case for a longer assessment window for all weight loss assessments as, particularly during the school holidays, it was problematic for researchers to schedule appointments within a 2-week window. Some flexibility and trust is required when participants miss appointments because of transport problems, admissions to hospital or other incidents96 outside their control, when financial loss is at stake.

Assessment schedules

Assessment schedules for intervention participants at 3, 6 and 12 months were feasible and acceptable, and these should be retained in a future trial. Attendance at the 6-month appointment was lowest. An option would be to provide reimbursement for engagement with study procedures at 6 and 12 months; however, this would be unlikely to be rolled out after the research and could potentially interact with the interventions, particularly with a waiting list control arm who do not receive the interim assessments. No evidence emerged to suggest that a 9-month appointment would have added value or increased retention. The assessment schedule of baseline and 12 months was feasible and acceptable to the control arm and should be retained.

Qualitative evidence suggests that intervention participants valued personal contact with researchers during the otherwise remotely delivered intervention. Given that the incentives are contingent on verified weight loss, some level of sustainable and scalable personal contact will need to be retained for the delivery of this intervention component. Data suggest that verifying weight and providing brief motivational support would be feasible for a NHS band 3 health promotion or dietetic assistant (the previous employment role of one of our research assistants) and cost £49 per participant attending all four scheduled meetings (see Table 19). The verified weight and brief motivational support intervention component has the potential to be more cost-effective than a brief intervention delivered by GPs and to release valuable GP time. 96

Refine participant information

Study information materials and researcher discussion at baseline and follow-up appointments could further emphasise that it is important for men to attend appointments regardless of what has happened to their weight. Qualitative data analysis suggests that researchers could emphasise to participants that they are there to make measurements and to listen to experiences, and will not be disappointed if weight loss has not occurred or, indeed, if weight gain has occurred. Including this in researcher recruitment and follow-up protocols would acknowledge that supportive accountability is possible, particularly when assessments are undertaken in the participant’s general practice.

Outcome measures

Two changes to assessments of key variables could be considered. First, data linkage to routinely collected data was found to be acceptable in this study and could complement self-reported health service use measures. Longer recall periods (i.e. 12 months) for health service use questions would allow comparisons between intervention and control arms. In the current study, health service use was reported for the last 3 months only, which made comparison between intervention and control participants possible for only the final 3 months of the study. Second, the current study focused primarily on weight and waist circumference as these would be the primary outcomes of a future fully powered trial. Other potentially important outcomes that matter to men could be added as relevant outcomes. This is currently being investigated by the Game of Stones PhD student.

Study website

The study website was low cost and has room for improvement; engaging with experienced website designers could improve its functionality, design and content. Some participants used the self-monitoring features offered. A future study might signpost participants to existing self-monitoring options, including links in SMS, rather than offering this on the current website, which could act as an information point rather than as an intervention tool.

Recruitment channels

Both the community and the GP recruitment channels were successful and each has advantages and disadvantages. Additional back-up strategies to improve recruitment rates, which had ethics approval, were not required. There was no evidence of gaming (e.g. deliberately increasing baseline weight by not disclosing objects in a pocket) among the financial incentive participants recruited from either channel.

There are strong arguments for keeping both recruitment strategies in a full trial, as both are feasible and acceptable, and some participant characteristics differed by recruitment channel. Beyond the full trial, we propose that men could be signposted to the intervention by GPs and other NHS and social care providers and through media promotion, as both the SMS and the incentive payment systems could be centralised and delivered remotely. Signposting through general practice, NHS and community routes would offer both primary and secondary prevention for men with obesity. General practice recruitment requires more set-up time for recruiting practices and for the clinical staff to screen lists of potentially eligible patients. Then, men opting in to the study could often be seen concurrently at general practices, making it logistically easy and efficient for researchers doing fieldwork. However, general practices are currently experiencing significant resource and time restraints that can impede recruitment to research. These issues are not unique to this study and require consideration by policy-makers and senior decision-makers.

Community recruitment of around 2 hours per participant randomised is comparable with that of many trials and returns participants from across the socioeconomic spectrum, including those without obesity-related comorbidities. It is a reliable and a steady source of participants, especially once research assistants develop community networks and gain experience. Broader community recruitment activities are an additional option, such as online social media and local media recruitment via adverts. This would be potentially less resource and time intensive and could increase participants recruited through word of mouth, but would decrease the traceability of participants and could increase the potential for contamination.

Previous review findings

A systematic review3 of incentives for weight loss reviewed RCTs that reported follow-up of at least 1 year. Nine studies were included in the original review. Most of the participants were women recruited through media advertisements. No significant effect of using financial incentives on weight loss or maintenance at 12 and 18 months was found.

Update of review

An update of the original search was conducted between the end of the original search and June 2015. Out of 1816 unique records, two additional studies5,6 met the inclusion criteria.

John et al. 93 conducted a three-group RCT consisting of a 24-week weight loss phase followed by an 8-week maintenance phase, which was followed up 36 weeks after the intervention finished. Participants were 66 obese US veterans (83% male, with a BMI of ≥ 34 kg/m²) randomised to one of three conditions: (1) a weight monitoring programme (n = 22) involving an oral consultation with a dietitian, discussion of weight loss strategies, weight loss goal, provision of scale, and US$20 for attending weigh-ins; (2) a weight monitoring programme plus deposit contract (n = 22), which involved the same weight monitoring programme as in 1, as well as a handout depicting daily weight goals and describing incentive plan, self-weighing, oral reporting of weight, text message feedback on weight loss goal and incentives earned, and financial incentive for weight loss; and (3) a weight monitoring programme plus deposit contract plus maintenance framing (n = 22), which involved the same weight monitoring programme as 1 and the same deposit contract intervention as 2, as well as a distinction between weeks 1–24, framed as the ‘weight loss period’, and weeks 25–32, framed as the ‘maintenance period’.

The deposit contract had the following elements. At the beginning of each month, participants could contribute between US$0.00 and US$3.00 per day of their own funds. During the month, participants accumulated rewards for self-reporting a weight at or below the weight loss goal that was equal to their daily deposit, plus a one-to-one match from the study. Awards were paid at the end of the month if participants weighed at or below their weight loss goal at an in-person weigh-in. Participants could earn US$84 net (US$168 gross) per month.

The SMS component had the following elements. Participants were instructed to weigh themselves each morning and to report their weight by noon. Every day, participants received same-day feedback via SMS on their progress and incentives. Non-adherent participants received feedback about what they would have earned had they met their target weight.

The results indicated that participants in the financial incentive conditions showed significant weight loss over an 8-month intervention, with no difference between the two conditions (weight monitoring programme –0.54 kg; deposit contract –3.44 kg; deposit contract and maintenance –4.35 kg). However, participants regained weight post intervention, and there was no difference in weight loss after 17 months (monitoring programme –0.13 kg, deposit contract combined –0.54 kg). Retention of study participants at 17 months was 65%.

Leahey et al. 94 conducted a three-group RCT consisting of a 3-month web-based community wellness initiative followed by a no-treatment follow-up period until 12 months. Participants (n = 236) were predominantly female (82.5%), of white ethnicity (88.7%) and obese on average (BMI of 33.6 kg/m²) and were randomised to one of three conditions: (1) a team-competition-based weight loss programme with access to a behavioural weight loss programme (n = 91); (2) a team-competition-based weight loss programme with access to a behavioural weight loss programme plus financial incentive for weight loss and website use (n = 89); or (3) a team-competition-based weight loss programme plus optional group sessions with private weigh-in and group discussion (n = 88).

The financial incentive component included small financial rewards for website use. Participants submitted 5 days of weight, calorie and activity information online to earn between US$1 and US$10 per week. Larger incentives were delivered at the start (week 1, US$8; week 2, US$10) and incentive size varied thereafter (e.g. US$1, US$2, US$7). Participants could earn a maximum of US$45 overall. The website included a ‘bank’ that displayed the previous week’s and the total earnings. Weekly reminders to submit information were framed using regret aversion language (‘Don’t miss out on your money, be sure to submit your information by Sunday at midnight’). A clinically significant weight loss was incentivised, with those who lost 5–10% of their initial body weight entered into a US$50 raffle. Those who lost ≥ 10% were entered into a US$100 raffle. Participants received all payouts after their 3-month assessment.

The results indicated that the team-competition-based weight loss programme with financial incentives resulted in significantly greater weight loss at the end of the 3 months than did the programme without financial incentives (incentive 6.4% vs. control 4.2% weight loss). There was no significant difference between the financial incentive and optional group session conditions (incentive 6.4% vs. group 5.8% weight loss). At the 12-month no-treatment follow-up visit, both the financial incentives and the group-based condition showed significantly greater weight loss than the team-competition-based weight loss programme alone (incentive 3.1% vs. group 4.5% vs. control 1.2%). The financial condition was the most cost-effective approach at both 3 (incentive US$34/kg; group US$87/kg; control US$34/kg) and 12 months (incentive US$64/kg; group US$113/kg; control US$140/kg). Retention of study participants at 12 months was 91.4%.

Relevance to current study

Both studies have elements relevant to the current study.

John et al. 93 recruited predominantly male obese participants, used a deposit contract approach, sent participants text messages, used a weight loss maintenance frame and conducted in-person weigh-ins. Although the study did not find any advantage of financial incentives at 17 months, the study sample was small (n = 22 per group). The general feasibility of using these elements was encouraging. Several points of difference from the current study include that deposit contracts were not advisable for hard-to-reach men who might lack personal resources; monthly weigh-ins and customised daily SMS were too resource intensive and might affect intervention scalability; the SMS focused on financial incentive only; and the weight loss maintenance frame had already been considered.

Leahey et al. 94 used loss aversion language to communicate financial incentive outcomes, rewarded the achievement of clinically significant weight loss goals and examined cost effectiveness of the different intervention groups. The finding that adding financial incentives to existing behaviour change programmes leads to increased long-term effects on weight loss was encouraging. Several points of difference from the current study include that incentives were provided for study engagement rather than for weight loss and were based on self-report, whereas this study focused on rewarding objective weight loss; the active intervention was of 3 months’ duration, whereas this study comprised a 12-month intervention.

In summary, the two identified studies have several elements of overlap with the current study in terms of the general approach (e.g. loss framing), forms of delivery (e.g. text messages, web page) and intervention goals (e.g. incentivising clinically significant weight loss targets). Owing to the differences in context, timescale and resources, specific elements of these studies were not directly relevant.

Previous review findings

A systematic review of the evidence for weight maintenance2 reviewed RCTs that reported follow-up of at least 1 year. Behavioural interventions that deal with both diet and physical activity were found to have small but significant benefits on weight loss maintenance. Moreover, orlistat in combination with behavioural interventions led to significant increases in weight loss.

Update of review

An update of the original search was conducted between the time period of the end of the original search and July 2016. Out of 2802 unique records, five additional studies met the inclusion criteria. Weight loss maintenance interventions included a mix of behaviour/lifestyle interventions for diet and physical activity, behavioural/lifestyle interventions for diet including meal replacements and one pharmacotherapy and behaviour therapy with diet and physical activity.

Tsai et al. 112 examined the effects of an intensified maintenance intervention compared with a control group. The overall mean difference in weight change was 3.4 kg (95% CI 3.08 to 3.72 kg). Those in the intervention group regained 1.6 kg, compared with those in the control group who regained 5.0 kg.

Coughlin et al. 113 examined the long-term effects of a continued personal contact intervention following a 30-month maintenance intervention of personal contact. Participants who received a personal contact intervention for 30 months were re-randomised to continued contact or to no contact for a further 30 months. The difference between the personal contact group and the control group was 0.6 kg (95% CI –1.4 to 2.7 kg). Mean weight change for the original personal contact group was –3.2 kg compared with –1.6 kg in the original control condition (95% CI –3.0 to –0.1 kg).

Aller et al. 114 compared diets with high or low protein/glycaemic index, and found an average weight regain at 12 months of 3.9 kg (95% CI 3.0 to 4.8 kg). High-protein diets led to less regain than low-protein diets; the difference in weight regain was 2.0 kg (95% CI 0.4 to 3.6 kg). No consistent effects were found of glycaemic index diets on weight regain.

Lowe et al. 115 compared the absence or presence of meal replacements and/or a reduced energy density eating programme. The results suggest that at both the 24- and the 36-month follow-up the energy density group showed significantly less weight regain than the control group.

Wadden et al. 116 examined the effects of liraglutide combined with lifestyle change compared with placebo and lifestyle change. The overall mean difference was –5.9 kg (95% CI –7.3 to –4.4 kg).

Relevance to current study

The findings of the updated review underline the conclusions of the original review. Lifestyle behaviour change interventions targeting both diet and physical activity and pharmacotherapy have the potential to increase weight loss maintenance. Effects of individual dietary approaches for weight loss maintenance remain inconsistent. The current study, although not a weight loss maintenance study per se, will thus focus on targeting diet and physical activity, focusing on dietary approaches that the participants can adhere to, without being prescriptive. Pharmacotherapy, although associated with increased weight loss maintenance, is not an option in the current lifestyle behaviour change study.

Previous review findings

A comprehensive systematic review examined the clinical effectiveness of weight loss and weight maintenance interventions for men. 5 The findings of this review directly underpinned the funding call for the current study.

Update of review

An update of the original search was conducted in the summer of 2015. As the review had been conducted recently, no new relevant studies were found.

Study name

• Participants vote on the study name: tXt Men, Guts 2 lose, Game of Stones. Result was revealed at the end of the workshop.

Narrative texts

General points

• Concern over whether this story would resonate with the participants or be relevant for them – or would they say ‘that’s not me’ and be turned off from the story.

• Concern from one PPI that from the first week the texts should highlight what Slim’s behaviours should be, e.g. shopping habits.

• Concern over how long he should go on struggling/not knowing what to do before he starts to implement change.

• Key messages – ‘vigorous’ activity to come out.

• PPI – Slim should be a weight that people can identify with – not way heavier. Perhaps around 17 stones/250 Ib. Or could Slim’s weight be the same as what each of the participants is and inserted in accordingly?

• Perhaps put weight in stones?

• PPI – exercise is an important starting point and can be a catalyst to help men in managing their weight. When people are exercising they aren’t eating and it’s important in itself for keeping weight off and health. It can break a cycle and increase motivation to improve diet. Eating is necessary, but it is not a past-time.

• PPI – the concept of the 2-4-1 deals, etc., in the narrative is good and should be explored further. First-hand experience of buying things that are not needed just because it looks like a good deal. This way you also end up spending more than you would have in the first place. As it goes further there should be more information ‘debunking’ what the supermarkets want you to believe that you are saving money. The poorest people are targeted in this way, and it’s the high-fat/high-sugar foods that are on these offers.

• PPI – the intervention may be more relevant for younger people, rather than older people. The storyline may be appropriate for older people, but the delivery method, through text, may not be the best way.

• Slim could be given a pedometer in the storyline to reflect what participants are given.

• PPI – unsure that the texts will work for everyone.

• PPI – gradual changes are important.

• PPI – for the key messages, although exercise is important, it should be clear that dietary changes are the most important factor for weight loss. Suggest an 80–20 split in emphasis towards diet.

• It’s important that these texts emphasise that it is a fault of the obesogenic environment – not because they are lazy or not strong-willed. This could help to empower people. It could provide pointers to improve this environment like walking to the shop, not going to the shop when hungry, etc. Every week a little hint could be provided.

• PPI – not sure that texting is the right medium for communication.

• Important to have a question asking about other services used, e.g. ‘Have you utilised any other services?’.

Incentive strategy

‘Pre-mortem’

Study name

• ‘Game of Stones’ was revealed as the winner of the study naming vote.