Pilot study of a randomised trial of a guided e-learning health promotion intervention for managers based on management standards for the improvement of employee wellbeing and reduction of sickness absence: The GEM Study (Guided E-learning for Managers)

Article history

The research reported in this issue of the journal was funded by the PHR programme as project number 10/3007/06. The contractual start date was in April 2013. The final report began editorial review in December 2014 and was accepted for publication in April 2015. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The PHR editors and production house have tried to ensure the accuracy of the authors’ report and would like to thank the reviewers for their constructive comments on the final report document. However, they do not accept liability for damages or losses arising from material published in this report.

Declared competing interests of authors

We confirm that there are no conflicting or competing interests in respect of DeltaNet International Ltd as the Anderson Peak Performance (APP) gateway and the companies responsible for website and logo design.

Copyright statement

© Queen’s Printer and Controller of HMSO 2015. This work was produced by Stansfeld et al. under the terms of a commissioning contract issued by the Secretary of State for Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK.

Background and rationale

Rationale for the pilot study

An efficient and potentially cost-effective way of improving the psychosocial work environment is training managers to provide more effective supportive management for employees; this support should make employees feel valued and help managers recognise stressful and unfair conditions in the workplace. When applied to managers at all levels, such interventions can be transmitted through work relationships to change the organisational culture. We decided to focus the intervention on line managers for employees, as these managers would have the most effect on employees and might be under most pressure from both above and below. We planned to test the acceptability, feasibility, risks and likely effectiveness of a primary preventative intervention that provides knowledge and skills about management standards and their implementation in terms of managing stress at work and promoting well-being. This is a pilot study and we did not expect to obtain definitive results on the effectiveness of the intervention as we were not powered to do so. Rather, we wished to assess the likely magnitude of the effect of the intervention as a guide for the future trial.

The intervention was delivered in the form of a guided e-learning education program provided as part of a continuing professional development (CPD) process. The advantage of targeting our intervention at managers who are line managers for a number of employees is that this is potentially a cost-effective way of influencing employees’ well-being. The study randomised the intervention to ‘clusters’ consisting of groups of managers and those employees whom they supervise. It has an advantage over approaching employees directly as managers have more power to change working conditions and these changes will apply to a number of employees in specific work groups. A cluster design allowed us to select managers and the employees working with them in service configurations in which they were carrying out the same types of tasks with similar groups of participants. Potentially, the cluster design also allowed us to recruit groups of managers who work in close proximity on similar tasks and who could share experiences in the facilitated meetings. We attempted to match up employees with their managers involved in the study.

The e-learning intervention allows managers to access the intervention at the most convenient time for them and to be supportive and perhaps facilitate change in working conditions. It can also be returned to again and again and was delivered in weekly or fortnightly instalments to make it more manageable to engage with within a busy working life. The hosting system for the e-learning package also enabled take-up (access, duration and frequency) to be measured so that the influence of intensity of package use could be assessed. The package is interactive (inviting reflection, requiring participants to make decisions during case studies, integrating real-life practical activities) and therefore is likely to engage the interest of the managers involved. The e-learning package could be applied to the whole cluster simultaneously. Thus, middle managers, who are particularly vulnerable to stress, benefit from the intervention directly through their own e-learning.

Several commercial organisations have developed e-learning programs based on management standards but there is no clear evidence of the effectiveness of such programs. In this pilot study we tested the likely effectiveness of an established e-learning program. We chose to use the Anderson Peak Performance (APP) e-learning package Managing Employee Pressure at Work [see www.andersonpeakperformance.co.uk (accessed 25 May 2015)], which is based on HSE management standards and addresses many of the aspects of workplace stress identified in observational research. It has been used in a number of public and private organisations by its originator Rosemary Anderson but its effectiveness has not been formally tested. A comparison of face-to-face stress intervention workshops led by an instructor with an e-learning format suggests that both formats can lead to significant reductions in reported stress, although attrition is significantly higher in the e-learning format. 39 Although there are a number of e-learning tools on stress at work, very few focus on what managers can do to reduce stress among their employees, which is a key feature of the APP e-learning package.

Objectives

The overall aim of the main randomised trial was to evaluate whether or not an e-learning health promotion intervention using management standards applied by managers improved employees’ well-being and reduced sickness absence in clusters selected from an organisation compared with similar clusters in the same organisation where it was not applied.

In this pilot study we tested within separate clusters of the same organisation:

• the acceptability of the trial

• the feasibility of recruitment

• the components of the intervention

• adherence

• the likely effectiveness of the intervention.

With regard to adherence, in the protocol we stated that ‘The acceptability of the intervention to managers will be assessed by managers’ engagement with the intervention and their attitudes to the intervention using qualitative methods’. Adherence was not explicitly defined.

As the aim of a pilot study is to prepare for and improve on the design for a full RCT, we formulated a set of progression criteria to assess the likely success of such a future trial and identify measures for improvement. We hoped to see the following results for progression to the main trial to occur:

• An increase in well-being scores of at least 3% among those employees whose managers completed the intervention compared with employees from the control cluster whose managers did not complete the intervention.

• Recruitment of 80% of the eligible sample and 80% follow-up of the recruited sample of employees (up to 20% withdrawal from the study), with at least 60% of managers invited actively engaging with the intervention.

• The acceptability of the intervention and the trial would be judged by the responses of managers to the intervention and the responses of employees to the trial, using data from the qualitative study.

• The feasibility of the outcome measures and their collection would be assessed by the response rate to the online and paper questionnaires for employees, at least 60% coverage of recruited employees and the ease of availability of sickness absence data.

The decision to progress would also take into account whether or not simple procedures had been identified that are likely to improve rates of recruitment, retention and adherence, taking all measures together rather than in isolation. The overall costs and benefits of the pilot study intervention were also assessed to judge whether or not these would support a full trial.

In the protocol (version 3) our criteria for progression were as described in the following section.

Patient and public involvement

Public involvement in the research was carried out through the steering group, which included a union member and a member of a mental health charity. Both contributed to fine-tuning the design of the study and to the management of the research. In particular, they were helpful in making suggestions about the content of the employee questionnaire and about ways to engage research staff and managers.

In addition, we held meetings with trust managers a month before the start of the study. As there were no ‘patients’ involved in this study and the relevant ‘public’ involved in this study were employees, we felt that managers and employees were the relevant groups to discuss the study with. This was very helpful in engaging the target audience and the discussion with the managers helped to refine the study methodology. Feedback was used to identify the best channels to communicate with managers during the intervention, for instance managers tended to favour telephone and face-to-face contact with the facilitator rather than web-based interaction. We also received helpful pointers about possible concerns regarding the understanding of cluster selection and randomisation and the confidentiality of the employee questionnaires, which helped us in the set-up of communications with the targeted participants.

Introduction to the mixed-methods approach

The pilot study was designed as a ‘mixed-methods’ study, incorporating a qualitative component alongside a RCT. The benefits of a mixed-methods design are increasingly acknowledged40–42 in terms of helping to illuminate the processes of a trial, the acceptability of a trial and intervention and possible reasons why an intervention does or does not work and to draw out the learning from a study.

Trial methods

Intervention

Name of the intervention

The intervention involved a guided application of the APP e-learning program Managing Employee Pressure at Work for managers. A summary of the intervention timeline is given in Figure 1.

FIGURE 1.

Intervention timeline.

Outcomes

The primary outcomes of this pilot study related to acceptability, feasibility and participation.

Safety considerations

The pilot study allowed us to study any risks of the intervention. Given the nature of the intervention and the absence of any routine clinical assessments or reporting procedures as part of this study, there was no systematic documentation of adverse events or adverse effects.

Potentially, we thought that managers could misinterpret the program and adopt maladaptive management strategies. However, the intervention was laid out in such a way that it was easy to comprehend, was organisationally supported and was sanctioned as being part of the HR and professional development strategies. Potentially, some managers could have become distressed by recognising that their previous management practices had not been ideal or that new techniques may be more time-consuming, requiring them to relinquish some control or be involved in more teamwork. We organised an e-mail or telephone-based discussion option for managers to discuss with the study facilitator any issues that arose with the program. The facilitator was asked to report any harms identified to the study team. Negative experiences by managers during the guided e-learning intervention, and by employees as a consequence of their manager’s participation, could be documented qualitatively in a sample of in-depth interviews.

The questionnaires for well-being (WEMWBS) and psychological distress (GHQ-12) are too non-specific to identify clinically significant distress reliably within individual distress. In case psychological distress was reported directly to the research team, protocol guidelines were in place to guide participants to relevant sources of support. Any such direct reports would be reported anonymously for the purposes of the safety evaluation of the study.

Sample size

The study was designed to recruit 120 employees from four clusters, expecting that 100 employees per cluster would consent. The response rate was estimated to within 3.8 percentage points, for example 76.1% to 83.9%. Measurements of intervention acceptability could be estimated from those managers who consented to the intervention, expected to be 40 individuals. If the take-up was 80% the 95% CI would be 64% to 91%. We envisaged recruiting 30–40 managers, each responsible for 5–20 employees. The pilot study was designed to provide evidence for the basic assumptions in our sample size calculations, to strengthen our calculations for the main study. From earlier literature on well-being measures52 the ICC was expected to have a value of about 0.07. Eldridge and Kerry53 showed that ICC estimates from small studies are dependent on the number of individuals and therefore four clusters provide a reasonable estimate. In this pilot study we estimated the likely effect size with its 95% CIs.

Randomisation

Three clusters, rather than individual participants, were randomly allocated to the intervention group and one cluster was randomised to the control group. This was to fit the aims of the trial, which were to test the recruitment process and obtain information about the implementation of the intervention. All managers within a specific intervention cluster were provided with the same intervention. One simple randomisation step was undertaken by an independent statistician. Randomisation of clusters was scheduled to take place after employees had been recruited and before recruitment of the corresponding managers, as only managers in clusters randomised to the intervention would be actively involved in the study.

Blinding

Employees were blinded to whether or not their managers had been randomised to the intervention or the control group. Managers in both groups were asked not to reveal their allocation to their teams.

As managers in the three intervention clusters only were recruited into the study, and as allocation to the intervention or the control group was self-evident, participating managers could not be blinded to their allocation. Likewise, the study facilitator and qualitative researcher were aware of the allocations.

As this was a cluster randomised study and no medical intervention was applied that would require emergency treatment, unblinding procedures were not applicable. Participants were informed of their allocation at the end of the study.

Members of the research team were kept blinded with regard to identification of the intervention and control group clusters and were unblinded only on a need-to-know basis. Members of the local team responsible for the recruitment of employees were not told of the allocation until the end of recruitment. Researchers performing analyses on the quantitative and economic data were blinded to the allocation until the main analyses had been performed.

On recruitment, each participant was allocated with a three-digit unique study identification number. These numbers were predefined and allocated in groups to one of the four clusters. This allowed for participants to be allocated to the correct clusters without identifying the cluster.

Statistical methods

The analyses were carried out using Stata version 12 (StataCorp LP, College Station, TX, USA). Pilot study analyses were descriptive and no formal statistical analyses were conducted to determine the effectiveness of the intervention,54 although CIs are presented. Participation rates are presented for each cluster and overall and 95% CIs for rates have been presented without adjustment for clustering. Effectiveness comparing the intervention and control clusters was estimated using a random-effects model with restricted maximum likelihood estimation. As the random-effects model assumes a large sample for the number of clusters, the CIs were calculated using the standard error from the model and the t-distribution with 2 degrees of freedom instead of the normal distribution. Post-hoc analyses were carried out to assess changes in well-being scores for employees of managers who did or did not engage with the intervention, as well as for employees of managers who changed position during the course of the study. This analysis used a random-effects model but did not adjust for the small number of clusters as the comparison was within rather than between clusters.

The same method was used to assess sickness absence but only employees who did not have an episode of sickness absence > 21 days were included.

Qualitative methods

Qualitative analysis

Data analysis took place concurrently with data collection, enabling progressive focusing on emerging themes. Two qualitative researchers were engaged in close readings of the transcripts of interviews and meetings, observational field notes and associated documentation. We individually and collectively identified themes emerging from the data, both within subsets of the data (i.e. themes emerging from key informant interviews, from manager interviews and from employee interviews) and across the data set. We discussed our preliminary findings with members of the study team, individually and at team meetings, and with the steering committee and drew on these discussions to interrogate our data further and develop our in-depth analysis.

In keeping with the narrative approach to interviewing, we drew on a narrative approach to analysis of the interview data. This meant that we paid particular attention to the story as a whole that respondents told rather than simply focusing on segments of text. The aim was that this approach would generate a rich picture of the subjective and situated experiences of workplace stress and the guided e-learning intervention.

Dissemination of the results

As part of this pilot study it was planned to disseminate our findings specifically among the study participants, as promised to them, to receive additional feedback. We designed a printed newsletter that was supplied to all participants and conducted two local feedback meetings for study participants at which we presented and discussed our findings. We have also presented the results at the following conferences: a Public Health England-sponsored conference on Work, Health and Wellbeing in Manchester (10 November 2014); the UK National Work-Stress Network annual conference in Birmingham (22–23 November 2014); and the European Association of Work and Organisational Psychology Conference held in Oslo, Norway (20–23 May 2015). Some parts of the GEM study were also presented at an ISER seminar at the University of Essex on 19 January 2015. Our website has been updated and will refer to published GEM study results in the future [see www.gemstudy.net (accessed 3 June 2015)]. Three papers are also in preparation: a mixed-methods paper reporting the results of the pilot study led by Stephen Stansfeld, which has been submitted for publication; a review paper of e-learning approaches in work stress management led by Tarani Chandola, targeted at an occupational medicine audience; and a mixed-methods paper with a qualitative focus co-led by Jill Russell and Lee Berney, aimed at a social science audience.

Participant flow

Inclusion of clusters

We recruited the clusters and participants from a mental health NHS trust in the north of England (participating organisation). The flow of clusters and participants is represented in Figure 2.

FIGURE 2.

Participant flow diagram.

Six clusters from within this organisation were considered for inclusion in the trial. In one cluster the workforce composition and organisational structure was considered too different for comparison with other clusters. Another cluster had insufficient availability of employment data. These two clusters are marked as not included for ‘other reasons’ in Figure 2.

The four selected clusters that took part in the study consisted of three adult mental health services working in separate geographical areas of the participating trust and the teams caring for adults with learning disabilities within the same trust. There were between 199 and 330 employees reported to be working in each cluster who were targeted for participation in the study (Table 2). It should be noted that the total number of targeted employees (n = 1116) is based on estimates given at the local sites on enquiry by the recruiting staff and is higher than the number of staff expected based on figures indicated by local stakeholders when we selected the organisation (n = 650). These employees were managed by between 11 and 19 managers per cluster (Table 3 and see Figure 2).

TABLE 2 - Participation by cluster for employees who were invited to take part in the trial

Respondents using both media (entry started online and then completed on paper).

Details of employees	Cluster 1	Cluster 2	Cluster 3	All intervention clusters	Cluster 4	Total
No. of employees invited (no. of employees approached personally by recruiting staff)	323 (142)	264 (174)	330 (204)	917 (520)	199 (129)	1116 (649)
Ineligible	0	1	5	6	1	7
Refused	2	7	1	10	1	11
Consented (% of those approached)	100 (70)	126 (72)	115 (56)	341 (66)	83 (64)	424 (65)
Baseline questionnaire (% of those consented)	89 (89)	101 (80)	93 (81)	283 (83)	67 (81)	350 (83)
Paper based	21 (19 + 2a)	14 (13 + 1a)	28 (27 + 1a)	63 (59 + 4a)	10	73 (69 + 4a)
Online	68	87	65	220	57	277
Follow-up questionnaire (% of those consented)	77 (77)	86 (68)	68 (59)	231 (68)	60 (72)	291 (69)
Paper based	20	27	22	69	10	79
Online	57	59	46	162	50	212
Primary outcome analysis (% of those consented)	74 (74)	85 (67)	66 (57)	225 (66)	59 (71)	284 (67)
Sickness absence data (% of those consented)	93 (93)	120 (95)	107 (93)	320 (94)	73 (88)	393 (93)

TABLE 3 - Participation in the intervention by managers

Three employees in cluster 3 were managed by a manager in cluster 2 who also managed two employees in cluster 2.

Details of managers	Cluster 1	Cluster 2	Cluster 3a	All intervention clusters
Managers invited	19	17	13	49
Ineligible	1	0	0	1
Refused	1	0	0	1
Consented (% of those invited)	12 (63)	17 (100)	12 (92)	41 (84)
Consented but no employees in the study	0	2	0	2
Adherence to the intervention
Completed fewer than three modules	6	9	5	20
Completed three or more modules	6	8	7	21
Primary outcome analysis: no. of employees whose managers
Did not consent	16	0	3	19
Completed fewer than three modules	20	38	28	86
Completed three or more modules	38	47	35	120
Primary analysis: managers changing role during study
No. of managers	2	5	2	9
No. of employees of these managers	12	28	12	52

Recruitment and follow-up

After receipt of all necessary ethics permissions, the participating site was activated for recruitment on 1 June 2013 and recruitment of participants was initiated. The first participant gave written informed consent on 6 June 2013.

Baseline questionnaire data collection took place in parallel with employee recruitment. After registration employees were sent a link to complete an online questionnaire or were offered a paper questionnaire in case of non-response. The first questionnaire was completed on 28 June 2013. Baseline sickness absence data from the organisational HR systems were collected for the period from 1 May to 31 July 2013.

It was planned that recruitment and baseline data collection should proceed in the following phases:

• June/July 2013 – recruitment of employees and completion of baseline questionnaires

• end of July 2013 – randomisation of clusters

• August 2013 – recruitment of managers from the three clusters randomised to the intervention arm.

By the end of July 2013, just over half of the targeted 400 employees had been recruited; however, randomisation of clusters and recruitment of managers could not be delayed for administrative reasons, as the intervention had to be scheduled within a specific time and deployed to all participating clusters at the same time. Recruitment of employees and completion of baseline questionnaires therefore continued after randomisation and the start of manager recruitment, but care was taken to keep employees and dedicated recruitment personnel blinded to the treatment allocation of the four clusters.

Randomisation of the clusters took place on 23 July 2013. Cluster 4 (the learning disabilities service) was randomly assigned to the control and clusters 1–3 (three regional adult mental health services) were assigned to the intervention. The first managers were recruited to the study on 26 July 2013 (date of informed consent).

The 424th and last employee was recruited to the study on 1 October 2013. The last baseline questionnaire was completed on 8 October 2013 and received on 9 October 2013. The 41st and last manager was recruited to the study on 17 October 2013. This late recruit was a newly appointed manager who replaced a previously recruited manager who had been promoted and was no longer managing employees in the study. Recruitment therefore ended on 17 October 2013.

The intervention began on 9 October 2013 with the first of three separate induction meetings for managers. The intervention officially ended on 20 December 2013 with the end of the scheduled e-learning timeline, although the e-learning program remained available for the managers in the intervention group until April 2014. Although according to the protocol the intervention should have lasted for 2–3 months, in practice the e-learning package was applied over 10 weeks after attending the initial face-to-face meeting and it remained available for participating managers to access for the entire follow-up period.

The follow-up data collection phase started on 1 January 2014. Sickness absence data from the organisational HR systems were collected for the period from 1 January to 31 March 2014. Paper follow-up questionnaires were sent to 73 employees who initially responded to the paper version of the baseline questionnaire on 13 January 2014; it was expected that this would improve the response rate as it was likely that employees would prefer to complete a paper questionnaire at follow-up if they had opted for this at baseline. E-mails requesting employees to complete the online questionnaire were sent on 29 January 2014 to the remaining 274 employees. Three employees were not sent a follow-up questionnaire as they had been promoted to manager status since their baseline questionnaire response and were therefore withdrawn from the study as employees. The last follow-up employee questionnaire was received on 10 April 2014. The end-of-study date was 28 April 2014, which was the date of the last in-depth interview conducted for the qualitative research.

Numbers analysed

Baseline data

The demographic characteristics of employees were broadly similar between the intervention group (three clusters) and the control group (Table 4). There was a greater proportion of women (85%) in the control group than in the intervention group (74%). There also tended to be more employees aged > 50 years in the intervention group (40%) than in the control group (31%). Similar proportions were married or cohabiting, had children or had responsibility for supporting a family member. A greater proportion in the intervention group (58%) than in the control group (45%) felt that their job interfered with their family life. More employees in the control group (25%) than in the intervention group (15%) were at band 7 or higher; however, similar proportions of employees were in charge of others.

Health at baseline was similar in employees from both the intervention clusters and the control cluster (Table 5) but the number of days of absence reported was higher in the control group than in the intervention group. It was notable that both clusters had a large proportion of employees who scored > 3 on the GHQ-12 (39% in the control group and 35% in the intervention group).

Table 6 shows the comparability of the intervention and control groups when those who did not complete a follow-up questionnaire were excluded.

Baseline population demographics were comparable to trust-wide demographics. In total, 79% of trust staff are female and this was reflected in the GEM study participant population (76%). Similarly, 38% of GEM study employees were part-time workers, comparable to a trust-wide proportion of 40%, and 27% of the GEM study population was aged > 50 years compared with 35% of trust employees.

Outcomes and estimation

Ancillary analyses

We carried out further analyses to assess whether or not the psychosocial work characteristics in the employees of those managers who were fully engaged in the program differed from the psychosocial work characteristics of those employees whose managers were not fully engaged in the program (Table 18). There was a reduction in job strain across the study in employees of adherent managers compared with non-adherent managers. There was little effect on work stress and effort–reward imbalance. Job insecurity diminished fairly substantially among employees of adherent managers compared with employees of non-adherent managers.

Finally, we wished to examine whether or not the intervention had differing effects within certain subgroups. In particular, we wished, as discussed in the original application, to compare well-being and self-reported sickness absence in employees from lower job bands with well-being and self-reported sickness absence in employees from higher job bands. In addition, we compared well-being and self-reported sickness absence in the following groups: men and women, those aged < 50 years and those aged ≥ 50 years, those working full time and those working part time and those who had worked for the NHS for ≤ 2 years and those who worked for the NHS for > 2 years. The differences were relatively small and there was little evidence of greater effects in any particular subgroup (Table 19).

Harms

No harms or adverse effects to employees or managers were reported during the study, neither to the facilitator, the qualitative researcher or another member of the research team. There were no reported adverse effects of either the trial or the intervention.

The context within which the study took place

The study took place at a time of considerable change and uncertainty within and beyond the NHS. Respondents cited pressures on NHS funding, restrictions on housing and other state benefits and the impact of the Francis inquiry57 into failings in NHS care as all affecting overall staff morale and well-being and having, in the words of one respondent, ‘knock-on effects for the stress levels of people’ (KI10).

At the local level, a major organisational change took place in the trust leading up to the period of study, which was thought to have significantly impacted on workforce stress levels (see Reorganisation, job insecurity and poor communication about change). Indeed, there was a perception among many that stress levels were higher than they had ever previously been.

At the time of the study, the trust had been engaged in a number of ‘healthy workplace’ initiatives in addition to the guided e-learning intervention. For example, several managers mentioned their involvement in ‘resilience training’, although most managers reported that before the guided e-learning intervention they had not received any specific training in managing employee stress. Nevertheless, by virtue of their employment in a NHS mental health trust respondents were likely to have significant awareness of and sensitivity to issues of stress and mental health well-being, and some mentioned that they were undertaking further study in the field of mental health. At the same time, working in a mental health trust providing services for mentally vulnerable patients was identified as having its own particular stresses and strains.

The trust has a stress prevention and management policy that references the HSE management standards for work-related stress. 32 The policy outlines the trust’s commitment to ‘identify, assess and subsequently eliminate, or reduce, work related stressors’. In our interviews with managers and employees, no reference was made to this policy.

Experiences of stress at work

Overall, the different groups of respondents (managers, employees and key informants) identified similar influences on workplace stress. Key sources of workplace stress were organisational change, job insecurity and poor communication, an increased volume of work and lack of resources to deal with it and organisational culture. These aspects of workplace stress reflect those identified by the HSE management standards32 on which the guided e-learning intervention is based. Respondents also identified some additional sources of stress: the physical environment, the nature of mental health work and the pressures of family life events and ill health. These influences on stress at work are less discernible in the e-learning materials.

In talking about workplace stress, respondents conveyed the ‘double-edged’ nature of pressure and stress: how what might be perceived as stress by one person might be experienced as pressure under which they thrive by another. Respondents thus emphasised the very individual nature of stress and how in terms of tackling stress at work the ‘one size fits all approaches won’t work’.

I actually love my job. I really enjoy it. I probably work better under pressure. I really enjoy problem solving. And I enjoy kind of working in different ways and new ways, and I’m always interested in different ways of working, and what’s going on, and how things are working.

M8

Some people deal with it [stress] in a better way. And I’ve always considered myself to be of that ilk, but knowing and seeing what I’ve gone through I’ve realised that it is a fine line that we all tread. And I think none of us quite knows which event or what sequence of events will take us to, you know, that breaking point in which, you know, for our own safety and our own sanity we’ve probably got to sort of step back. And I do think that some people are able to take these sorts of things on board a little bit better and deal with them, but it’s not easy at times.

E5

Reorganisation, job insecurity and poor communication about change

A major reorganisation within the trust during the period of the study was identified as a key source of stress for both managers and employees, with staff redeployed to different departments or reassigned to a different job role but not necessarily a new job band. Mention was made of ‘90 days to possible redundancy’ letters having been sent out in 2012, before the start of the study. Respondents spoke of the stress of uncertainty and job insecurity, of feelings of being out of control and decisions being taken elsewhere:

I don’t think in general any of us are working in a healthy environment. Because the constant changes that are happening, constant redesigns that are going on, everybody’s unsettled.

M5

And I think my concern about it is that most of the problems and stress-inducing aspects of work that we feel are beyond our control, completely beyond our control, and I don’t think that helps at all because things are foisted on us really. I think there’s an awful, to me, a yawning gulf between those who make the decisions and impose these things and the rest of us. I feel personally that voices aren’t heard . . . but it was very comforting to know everybody else felt the same.

M3, feedback meeting

As indicated in the previous quote, there was a view that poor communication ‘from the top’ had exacerbated the stress of reorganisational change:

. . . the process and the communication that people had going through that process was absolutely abysmal and it only added to the sense of worsening morale and not really knowing where the service was going. Lots of concerns about individual practitioners but also the team and how that would function, but more importantly how that was going to have a knock on effect to service users.

M9

In Box 1, one manager tells the story of the impact of reorganisation on his team and on one employee in particular.

Volume of work and lack of resources

Respondents identified a marked increase in the volume of work in recent years, but at the same time they felt that they had fewer resources to deal with the increase:

When a CPN [community psychiatric nurse] goes off sick or somebody in the community goes off sick, we’re generally not allowed to cover that with bank staff, so the team has to soak up the pressure. So in the last 12 months, we have had three staff off on long-term sick. So if you think they have an average caseload of 25 each, that’s 75 more patients that the rest of the team has got to look after.

M8

We can have say 13 referrals come in one morning and you’ve got five staff and they could be all urgent and they all need assessing and so that’s the stress for the workers and the stress for me is sort of managing that I suppose, trying to help them find solutions to manage their work.

M10

Managers spoke about how their jobs had changed in recent years, with more and more administrative demands on their time and also how more HR functions had been devolved from HR to managerial level, turning line managers into ‘part-time HR managers’:

It just seems there’s more and more and more things that have to be met . . . . The demands of having to meet the paperwork and the timelines and the guidance and the policy expectations, you’re constantly getting these performance data spreadsheets saying well, this is missing, this information needs to be updated and it’s just, you’re catching yourself coming back really. Being able to prioritise work, you’ve only got so many hours in a day to be able to do the bits you need to get done.

M9

The increased demands of documentation were identified as affecting all staff – employees as well as managers.

Overall, the feedback from managers at the facilitator-led induction meetings was summarised by the facilitator as consisting of concerns about ‘high demands, low control’.

Organisational culture and attitudes

Several respondents spoke of the negative effects of organisational culture, with one making the suggestion of a ‘blame culture’:

I think a lot of the stress isn’t so much about the service users. The stress is about the organisation’s attitude to the staff, if something doesn’t quite work out right, so you know obviously if we see 2500 people a year, a high proportion of them are people who’ve harmed themselves, so proportionally we will have more people killing themselves that come through our service than anywhere else but the trust response to that is extremely negative and there is a blame culture.

M3

Key informants from outside the trust identified broad trends in the workplace impacting on stress, with organisations becoming more and more ‘data-driven, rather than people-driven’ (KI1), performance management seen as a tool to identify employees as weak and failing, rather than as needing to be helped and supported, and ‘macho attitudes’ – ‘if you can’t take the heat, get out of the kitchen’ (KI5).

Discussion at the facilitator-led induction and follow-up meetings for managers conveyed considerable negativity about organisational culture and attitudes. The facilitator noted among managers a ‘concern that the number of investigations, disciplinaries and capability processes have increased . . . not feeling appreciated for what they’ve done/achieved – the focus is only on recognising what hasn’t been done/achieved’.

The physical environment

A common problem identified by employees and managers was the physical working environment. Common concerns were a lack of space, hot, overcrowded and noisy offices and ‘hot-desking’:

I think the stresses for the staff are rooms. Rooms are a big, big stress . . . . the availability to have rooms to work out of. This building isn’t adequate for the amount of therapists we’ve got. We’ve got people working out in satellite clinics but we also have to have, obviously, a core of staff here. They’re just juggling rooms on an hourly basis.

M6

It seems to be in vogue . . . but the actual experience of working in a hot desk room is causing lots more stress than it’s ever solved. The staff have been coming to me saying the rates of sickness are increasing, they’re unable to hear themselves think sometimes so their own stress levels in getting through things are [rising].

M9

It’s very hot, it’s very noisy, your desks, you’re literally in a row next to each other so you haven’t got any private real space, you’ve got a locker which you keep everything in basically. Being an OT [occupational therapist] I do lots of different assessments, I have quite a few OT different things and I’ve got nowhere to put them, so they’re all shoved in a drawer and I have all my clients that I’m working on in little folders, then I have to get the whole drawer out to try and find that one folder that might be at the bottom, put them back in and then think ‘Oh, I’ve got to do that’ get them all out. And it’s really had a major impact, the closeness, you can’t hear on the phone, you can’t concentrate.

E7

The nature of mental health work

It was clear from respondents that the nature of the work that they were undertaking could be inherently stressful:

The stresses had built up from an incident when I was on the night shift, when a colleague of mine was badly injured. So there were issues around that, which led to other issues with management, and I suppose I wasn’t reacting very well to the stresses, and you know, it just sort of built up and built up. So I’d a few discussions with my ward manager at the time, who was supportive, very supportive and . . . but I decided that I was okay, I can struggle on through. And I think it was the straw that broke the back. And . . . in my case it was a patient who just decided to target me for a day that finally just broke me.

E5

At the same time, respondents identified the positives of working with this client group:

The positives [of the job] for me are absolutely the clients. I am very passionate about recovery and so the people that I work with, the fact that I can work with them for up to 2 years so I really get to know them inside out. So building up therapeutic relationships there is extremely satisfying.

E8

Family pressures and personal health

One of the striking findings to emerge from manager and employee narratives of employee stress was how their accounts invariably started with a description of the employee as a family carer, as having an elderly mother or a sick relative, or as coping with a bereavement, or having grandchildren who needed looking after whilst the parents were going through a messy divorce, or having various other ‘private personal stresses’. As one respondent commented: ‘It’s not really work stress so much as it’s personal stress. But of course it does have an impact on one’s work life’ (KI13) Several of the stories of stress at work that we present here convey the complex, intertwined nature of personal and workplace stresses (Boxes 2 and 3):

[I was] very, very stressed. I kind of . . . it gets to that point when you think, I just can’t do it. And it might sometimes be personal stress and work stress and everything all happening at the same time, which it usually does doesn’t it? It’s never just one thing or another.

E8

Views on ‘competencies’ required for managing stress at work

Interviews with managers and employees illuminated the ‘competencies’ that respondents thought were required to manage stress at work. Respondents spoke of the need for managers to have an ‘open door’, be accessible, approachable, supportive, compassionate, a good listener and understanding, look for solutions to problems, give staff confidence that they could confide in their manager and have a culture of honesty and openness:

I think knowing that there’s a manager’s support in all that is worth a fortune, really, for someone. They know people are seeing their manager being aware of the stresses that they’re under and being able to express as much in the way of listening and understanding and helping within the system in as many ways as they can to support that person. I think that’s what employees really need, they just don’t want to be left with all this to try and cope because then they’ll implode, sort of break down and next thing they go off with stress.

M3, feedback meeting

I like to feel that they are open and honest. They will share things where they can share things. They are always there if you’ve got a query, a question, a problem and I think if approached in the right way will always be on board with trying to solve that, rather than taking a sort of an authoritative approach that, you know . . . if there’s maybe some aspect of work that maybe it hasn’t happened as well as it should do the approach would be a fairly informal ‘Let’s talk about this, let’s find a way forward’, using supervision, using appraisal and the like rather than . . . I know in some areas it would automatically become almost disciplinary or a capability issue, yeah. So it’s like trying to find a way forward rather than punishing you for it.

E4

In Box 3 several of these attributes are identified, as an employee describes how her manager supported her through a stressful period.

Emotional intelligence and sensitivity

You can have a commitment or a strategic goal around mental health but actually what really matters to people is when they come in on a day-to-day basis what’s their relationship like with their line manager? And if line managers don’t possess that emotional intelligence in terms of being able to spot signs and symptoms, motivate, lead, inspire, when there is an issue of course and an individual is not able to relate to their line manager and they go off sick for any reason, it’s incredibly difficult to get them back into the workplace . . . it is all about middle managers and it’s about their behaviour and about their emotional intelligence and about how they go about conducting themselves on a day-to-day basis.

KI5

The various competencies for managing stress at work identified by managers and employees can perhaps best be encapsulated in the concept of emotional sensitivity. Emotional sensitivity requires a connection both with one’s own emotions and with the emotions of others and has been identified as being critical to workplace health. 58 Yet surprisingly, the HSE management standards and the e-learning program pay little overt attention to this concept. In Box 4 a manager illustrates this ‘competency’, describing how he draws on compassion and his own ‘personal beliefs and personal values’.

‘Juggling’ priorities

Although the guided e-learning intervention was focused on reducing stress and increasing well-being in employees, it quickly became apparent that managerial stress was also a key issue (Box 5). At the same time as talking about managing stress in their employees, managers easily slipped into talking about their own stress.

A central aspect of managing stress as a manager was the ability to ‘juggle’ – to juggle between organisational demands and employee needs (Box 6). Managers saw themselves as pushed in both directions, as ‘the damp-proof course in the organisation, nothing permeates in either direction’:

I suppose you wear two hats as such because one, you want to look after the individuals and the employee, you want to do the kind of best for them, you want to make sure that they are being supported in every possible way, kind of physically, mentally. You want to ensure the other hat is a . . . the organisation is still working and that the team itself is still working, that the people are picking up his patients. So you’ve got a foot in both camps because you’re trying to be the manager and look after the welfare of everyone including all the staff, all your patients. And then you kind of, on an individual basis, you’re trying to look after that person’s mental health you’re trying to be there for them, you’re trying to give as much empathy as possible. You are always trying to kind of . . . it’s like kind of juggling two balls and all of a sudden you drop a ball.

M8

If you’re getting stuff fed down to you without really understanding where you’re coming from, my priority that day isn’t to update a spreadsheet that they need to put through to their contracting discussions, my priority is actually getting a staff member out to see that person who’s just tried to hang themselves or something. So the pragmatic things I have to get done on a daily basis to maintain a service or to reduce the risks to individuals but then there’s the pressure from people from a corporate point of view in terms of running an organisation and all the governance that goes with that and sometimes the two just don’t tally up.

M9

The ‘difficult place’ that managers found themselves in was something acknowledged by employees:

I think the manager role is a key role and I’m aware of the annual staff questionnaires that are done and I think for this trust one of the positive things that’s come out has been a lot of people have felt the first level line manager as being supportive. Where it’s fallen down has been with sort of more senior managers who maybe don’t have that relationship or whose own pressures are very different ones, increasingly financial ones aren’t they? So I see a split, a disjoint if you like between what’s happening for . . . what the senior managers are doing, what middle managers are doing, I think sort of middle managers are probably getting the worst of both worlds. There is pressure from them on the top to perform, to hit targets, to achieve financial outcomes and then from the lower levels with all the everyday aspects of running a service and everything that goes with it. So I think they are in a very difficult place.

E4

Managers also described how they had to juggle different aspects of their role, as both a manager and a practitioner:

I think getting the balance between managing my own workload and my own priorities, supporting the staff, and everything that involves with sort of the performance management and that sort of thing. And also doing clinical work as well, . . . it’s almost like having lots of balls up in the air and juggling with them.

M5

Some managers felt that they had ended up as managers by default rather than by design – ‘I was a good nurse but now all of a sudden I’ve been told “you’re good at nursing, do management” ’ (M8).

Another source of stress identified by managers was the feeling that the stress they were supposed to be managing for their employees was often out of their control, perhaps because it was something that senior managers had imposed, or because it was to do with an aspect of organisational life or policy over which they had little influence. Although many employees acknowledged and accepted this situation, for others it was a source of frustration: ‘Well if you do raise issues she’ll just say, “well there is nothing I can do about it, we’re all in the same boat” ’, said one employee (E6).

Experiences of the guided e-learning intervention

In this section we report on the views of the 13 managers in our interview sample. We were unable to elicit the views of those managers who did not participate in the study. All of the intervention managers in our sample were defined as ‘adherent’ (i.e. had completed three or more e-learning modules). Thus, our sample consisted of a motivated group, keen to engage with the educational opportunity offered by the e-learning program. One respondent described her motivation in the following way:

Firstly, as a manager, I wanted to feel more confident and competent at dealing with my workforce and learning how to do it differently, or even just reflecting on the way that I do things, so that, you know, I’m consolidating my knowledge and experience. So that was my first sort of motivation. The second motivation was to help the staff, because I knew that the staff were also going to be getting involved in terms of giving feedback about how I manage.

M5

Feedback at the manager meetings that we observed indicated that some groups of managers had been nominated to participate in the study. At the preliminary trust meeting about the study the head of HR reported that ‘about half the managers attending have volunteered and the others have been chosen to participate’. At the ‘feedback’ meeting (April 2014) one manager commented:

When I found out I was taking part in the study, it was in a ward managers’ meeting, it was like we’re the ward managers so there was no choice, just we’re doing it, which is fine for me because I thought oh I see, but I was really pleasantly surprised.

M1

Overall, managers reported many positive aspects of the study and the e-learning program. They were overwhelmingly positive about their interactions and communication with the study team and the program was considered easy to access and straightforward to use, ‘looked professional’ and was visually appealing:

It’s easy to use, it’s user friendly, it’s simple, it looked nice, there were useful tools that were in there. It was a tool I’d go back to if I could, stuff that was in there that you could draw upon at later dates. I think that would be useful.

M9

Managers generally found the material within the program applicable and relevant. A few managers commented that the e-learning program had introduced them to new concepts, such as underemployment as a cause of stress and ‘presenteeism’. Respondents were positive about the ‘layered’ approach to information in the program, which meant that they could engage with the material in more or less depth as they found helpful:

I think it gave just the right amount of information, I don’t think it blew you away with shedloads of unnecessary information and it came up with cases and it came up with, you know, the legal framework around discrimination say, so I thought that was quite good and then you clicked on things that you could elaborate on and get more information and there was other stuff that I managed to get which I thought was quite handy for my dissertation in fact. You know, when it’s the further information bits you could click on or the resources as well. So that was quite good.

M3

A tool for reflection and validating practice

For most managers the key value of the e-learning program was not the new knowledge that they acquired from it but the way in which it ‘backed up’ existing knowledge and encouraged reflection on managerial practice. In this sense, managers felt that it validated their own practice and thus helped build confidence:

It made me kind of more aware of the kind of processes and made me more aware of some of the things I was actually doing, probably was doing subconsciously, but it kind of backed me up.

M8

I suppose it wasn’t any news to me about things that people suffer when they’re under stress but then I don’t think it does any harm to do it as a reminder.

M1

I don’t think there was much in there that I don’t already apply to everyday practice. I guess it’s about reaffirming that that is what we should be doing with specific people. It’s having that toolkit almost isn’t it of things?

M2

I quite enjoyed the course because I didn’t really see things that were totally shocking to me or, ‘Oh! You should be doing that’. It reinforced that my way of doing it is all right, it’s acceptable . . . . So I found that course sort of validated some of the stuff that I already do and sort of sends a message to me that carry on doing it that way.

M4

It is a good package in that it invited reflection, and it certainly invited you to draw on past experiences yourself, which is a bit . . . I think I particularly took that bit away. Because there was one exercise, I thought, ‘Oh, this is really good’. It was where there was a question about thinking about managers you’ve had in the past and can you think of someone who you would have rated as a good manager? And what are the qualities of that person? And that was quite useful really, it really helped me to reflect, and then also the same thing for a bad manager . . . and what it was that that person was doing that was poor in terms of stress levels and that sort of thing. Now, I thought that was particularly good.

M7

A few managers, particularly those who had a background in psychology, found the information in the e-learning program too basic and would have liked more in-depth and advanced sections. Arguably, this finding is unsurprising among a group of experienced managers (respondents had been in post for between 3 and 25 years) working in the field of mental health.

I probably found [it] a little bit repetitive, or a little bit basic, probably because of my own level of knowledge that’s all. And obviously, quite a lot of it’s based on psychology, isn’t it? From that point of view, it’s granny and sucking eggs, I think.

M7

I think the content could have been a bit harder, but I suppose . . . looking back at it, I suppose it depends what or how long you have been a manager.

M8

I can’t really key into anything that I’ve picked up from the [e-learning modules] that I wasn’t already aware of or trying to use.

M9

The problem of time

Although senior managers had emphasised at the beginning of the study that the e-learning program could be studied in work time, one of the biggest problems that managers expressed about the e-learning intervention was finding the time to engage with it. Managers working in outreach settings spoke of ‘snatching time here and there’ (on a laptop in their car between clients or at lunch). Managers working on wards or in open plan offices spoke of being frequently interrupted. Some managers who did have their own private office spoke of ‘blitzing it’ – coming in to their office early, or staying late after work, and working through it in one 3-hour block, putting a notice on the door that they were not to be disturbed for a few hours. For one, time had clearly been a factor in not managing to complete the modules. At the facilitator-led follow-up meeting it emerged that ‘completing’ modules did not necessarily mean that managers had undertaken the suggested activities, with several mentioning lack of time as the reason for this:

It was finding the time in the day just to sit down and be able to do it sat at my desk without some other priority or somebody knocking at my door with another question. That was really it, it was not time-consuming or anything necessarily it was just literally finding enough time . . . I found it was useful, it was something I would want . . . I didn’t get far enough through to really be able to say actually, this could have been done differently or . . .

M9

My own initial thought was, ‘Where am I going to find the time?’ I want to do it, happy to do it. I think it’s good for me, on a personal level as well as a work level. But where was I going to find the time . . . I don’t think I gave it the time it deserved . . . I was interrupted all the time.

M6

I was interrupted. I tried to do it between . . . whenever times I wasn’t either doing supervision or going to meetings. And I think it ended up I tried a number of times, but I got interrupted so often that in the end I kind of just shut me door, locked me door, pretended I wasn’t in and blitzed, did a good 3 hours of it.

M8

The problem of senior management support

A clear recommendation from those involved in the development of the guided e-learning intervention and others involved in researching the management of workplace stress17,37 is the need for explicit commitment (‘buy-in’) of senior management. Indeed, those involved in developing the HSE management standards for work-related stress32 argue that, for an initiative to be successful, it needs to be ‘top led and bottom fed’:

One part of the process is we recommend setting up a tripartite steering group: senior management, local management and union reps to work together so that ultimately there is engagement. We came up with this wonderful phrase many years ago: it’s top-led, bottom-fed. And that’s it in a nutshell – without that, it doesn’t work. We found that through our bitter experience. If it’s all coming up from the bottom, it doesn’t work because you haven’t got the buy-in at the top to make the changes necessary. Conversely, if it’s all being fed down and the bottom aren’t engaged, they don’t participate and they don’t change their behaviours.

KI2

Senior management ‘buy-in’ included encouraging managers to participate in the intervention, perhaps ensuring that there was protected time for managers to engage with the intervention, engaging with the findings of the research and having a commitment to addressing the implications of study findings. Although the GEM study team put in a considerable amount of time and resources to try and ensure that senior management were on board in this way, and initially they seemed to be, by the end of the study it was felt that the initial support shown had not been sustained. The facilitator noted that at facilitator-led follow-up meetings for managers:

. . . there was an overwhelming feeling that there was a lack of senior management commitment to the program and the organisation had not supported them to do the e-learning as had been promised.

The facilitator (who had experience of similar initiatives in other organisations) reflected that:

. . . this is quite the norm in health and well-being programmes. They are often introduced to tick the boxes and are not followed through as recommended in good practice. Even when senior managers mean well programmes do not get the support that is initially agreed as other priorities take over.

There was also a disappointing turnout at the final dissemination meeting at the trust, with only two senior managers in attendance. We found it difficult to schedule interviews with senior managers at the trust and some interviews were brief and rushed. One senior manager reflected that arguably more could have been done to encourage greater engagement in the study:

We need to do more on selling it. So that lead-in stuff, although we did those briefings, and I was part of the briefings . . . with hindsight I think I’d do more. I think there’s a bit about how we then sell the management interest bit, ‘cos I think probably where we have not helped you . . . I think is there’s not been enough sort of day-to-day senior management attention to it and encouragement of it.

KI10

Some managers expressed cynicism about the trust’s motives for taking part in the study and concern that the trust could be seen as shifting responsibility for its duty of care to employees on to managers. Some voiced concerns about whether the trust would ignore or act on their comments or on the wider findings of the study:

I think the one thing I’d like to say is . . . I’d love to feel that senior management and the trust board and everyone do really look at this and see what comes out of it because I feel it will, like most other things, fall on deaf ears. But it would be really nice to feel that they do engage with the whole process and they do listen to some of the things their managers are saying . . . It would be nice to think the trust would embrace it as part of the core training for managers because I think managers, until fairly recently at least, in their training they were really just looking at policies and procedures, there was nothing really to do with the human resources, the people they were expected to manage. And all these human issues that do come up. Now anything like this should be trying to restore some or at least achieve some balance because really it’s the people in an organisation that is the main resource isn’t it?’

M3, feedback meeting

At the facilitator-led meetings for managers, in addition to expressions of cynicism, additional reassurances were sought that no individual’s views would be identifiable from the data, arguably reflecting a certain degree of anxiety and mistrust of senior management.

Views on e-learning

A central feature of the intervention was its primary mode of educational delivery: e-learning. Originally, the stress management course developed by APP consisted of a 2-day, face-to-face event. APP reported that, as a solution to organisations increasingly asking for more material to be included in less time, they developed e-learning modules, with the intention of delivering them alongside facilitated face-to-face meetings as part of a ‘blended learning’ package.

In the case of the GEM study, the face-to-face element consisted of two facilitator-led meetings that took place in October and December 2013. The first ‘induction’ meeting was held before the managers started the e-learning intervention and the second ‘follow-up’ meeting took place after the managers had completed a number of the modules. Further details of these meetings can be found in Chapter 3 and Appendix 3.

In general, managers were ambivalent about e-learning as a mode of educational delivery. On the one hand they identified certain advantages: the flexibility of when, where and at what pace to study and the ability to return to material:

And that flexibility, and it’s the direct stuff. I think the trouble with face to face, you might be having an off day or something like that, and you’ve trudged along to something and you’ve not really derived as much as you could out of it. And there isn’t really any time to go back, you can’t rewind and go through it again, can you? So I think the medium was very good, the presentation was good, it was nicely, clearly done.

M7

At the same time, e-learning was thought to be not as ‘engaging’ or as ‘real’ an experience as face-to-face learning. One key informant (KI1) suggested that e-learning can easily become a ‘a “click-click-click-click” exercise’ and this view was evident in some managers’ reservations about e-learning. Furthermore, the advantage of flexibility was sometimes seen as a disadvantage:

I do like the e-learning but if you go on a training session, face to face, then you’re taken away from the environment, aren’t you? You’re taken out and you’ve got that time – that dedicated time – to learn. Whereas doing it e-learning it’s easy to just get distracted.

M10

The ambivalent views expressed by managers towards e-learning echoed those of the GEM developer:

My view is that e-learning’s got the advantage you can go back as many times as you want; you can learn the stuff and you can do it in your own time, and it’s cheaper and it’s quicker. But it’s not as good in my view as face to face because people need to talk, they need to learn and support each other.

KI13

Most managers favoured a ‘blended’ approach, with more emphasis on face-to-face learning than had been the case with the guided e-learning intervention:

I think a lot of the content of it, I think if we’re looking at people trying to develop and learn, should be done face to face and then perhaps followed up by the e-learning package, almost to sort of consolidate your knowledge rather than learn it. Because it’s very different, isn’t it, learning how to recognise signs in people than it is to actually see it and you know I don’t know how you’d do it face to face, but I think people would get a lot more out of it when they’re learning how to do the job, face to face.

M5

To me it would have been nice to do the initial meeting, go away and do two modules and then get together and kind of have a brief discussion.

M8

In this sense, some managers saw e-learning not so much as a replacement for face-to-face learning but as a supplement to it. Those managers who attended the study induction and follow-up meetings with the facilitator were positive about the benefits of meeting face to face in a group. The meetings provided a forum in which managers could engage in discussion with other managers. Some managers reported working in isolation from others at a similar grade and it was reassuring for them to hear that their colleagues were experiencing similar pressures and confronting similar issues at the trust. They had also found it useful to hear others’ views of the e-learning program:

And it was quite good to hear the other people in the room were having similar things, similar issues, similar thoughts, similar concerns. And it was good to express those concerns, I suppose, in a safe environment with no people higher up from myself looking down on you and judging you. So from that perspective, it was quite . . . it felt like a safe environment, just to discuss openly some of the issues that as managers we were concerned about and had raised.

M8

A key informant reflected on the beneficial effect of this shared learning experience:

So what the intervention is facilitating is networking, which could have good effects on the managers’ well-being which in turn leads to well-being effects for the employee [but] in a sense, disentangling that from the actual e-learning intervention would be quite hard to do.

KI6

Reflections on the guided e-learning intervention as an approach to improving employee well-being

The qualitative study elicited a range of reflections about the extent to which the guided e-learning intervention might be expected to ‘work’ as an approach to improving employee well-being.

The tension between ‘blinding’ trial participants and the e-learning program’s interactive learning activities

The GEM study design involved ‘blinding’, that is managers who received the intervention were instructed not to tell their employees that they were participating in the trial. The rationale behind this was to reduce the bias that could be induced in employees’ responses to questions on well-being and their work environment at follow-up if they knew that their managers were taking part in the intervention. Interviews with employees indicated that blinding had mostly worked and that employees were unaware of their manager’s involvement (or not) in the study, although one employee reported that her manager had explained that she was being ‘trained up to manage stress with people better’ and the questionnaire was ‘part of her evaluation’ (E8). In one or two cases managers revealed that they had told their employees that they were participating in the study and in other cases, although the managers had not explicitly informed their employees that they were taking part, they indicated that some members of their teams had suspected that they were involved with the study:

The few in the team that signed up to do it knew that I’m doing something because obviously I had to give them the paperwork to sign up to do the course with me. I’ve never reminded them again that I’m doing it. Some know what it’s about. One of the lads actually said to me the other day, ‘Do you know why I keep getting e-mails off this flaming GEM place? What do they want?’ And I laughed and he went, ‘Ah . . . it’s about you, isn’t it?’ And I went, ‘Yeah’ and he went, ‘Oh, I’ll do it. I’ll do it’. He thought it was some staff survey that he wasn’t going to participate in.

M6

Alongside the trial requirements for blinding, the e-learning materials included activities to prompt managers to discuss aspects of their learning with their employees and to ask for feedback on their management style. One activity asked managers to conduct a ‘stress survey’ with their employees to identify and discuss problems and to work together to find solutions. Specifically, the activity encouraged managers to discuss the results of the survey and the causes of stress in the workplace at a team meeting. None of our respondents commented on this potential tension between the demands of the trial and suggested learning strategies, although in the facilitator-led meetings one manager mentioned that, when she had asked an employee how she felt about something, the employee asked if the manager was asking because she was in the study! The facilitator reflected that, in her view:

 . . . it is impossible to carry out the research without employees knowing some managers are taking part, and if they behave differently it is possible that their motivation will be perceived as due to the study rather than concern.

How a short educational intervention might change attitudes and behaviour

The theory of change underpinning the GEM study was that the guided e-learning intervention would lead to changes in managers’ knowledge, attitudes and behaviour, which would impact on the well-being of their employees. In our qualitative interviews we tried to capture examples of managers ‘putting the intervention into practice’, in other words, examples of how they had applied their learning from the intervention to support employees’ well-being. However, we found that in practice when managers spoke about instances of managing employee stress they drew broadly on ‘experiential knowledge’ rather than isolating any specific learning that they could attribute to the guided e-learning intervention.

Some scepticism was expressed about the extent to which a guided e-learning intervention might be expected to impact on entrenched attitudes and beliefs:

There are some old style managers who think stress is a weakness, all that rubbish, that type of thing. And I don’t know, I don’t think your module is going to address that, they’ve already probably seen what you’re talking about, I think that’s people’s beliefs and attitude and that needs to be addressed in a different way.

M4

A viewpoint that emerged during the study from key informants and members of the study team was that arguably the short timescale of the GEM study militated against the likelihood of observable effects of the intervention:

 . . . the big question is whether the e-learning package would induce long-lasting changes in the managers, behaviour change. We’re talking about behavioural change and the vast literature on behavioural change says it’s a slow process.

KI6

Managers similarly expressed some concerns about the timescales of the study. At a preliminary meeting to introduce the study to managers, one commented: ‘The period between the intervention and the second data collection is very short. How are we expected to implement things?’ (field notes from observation of preliminary meeting to launch study, April 2013).

Methods

Results

Cost-offset analysis

Cost–benefit analysis included two estimates of average intervention costs (£81 and £153) per employee (reflecting variation in the numbers of managers involved in the different parts of the intervention) and the average HR-reported sickness absence over 3 months at follow-up. The results of the cost-offset analysis are summarised in Table 29. The estimated net cost was more negative in the intervention group than in the control group for both intervention costs (£81 and £153). Analyses indicated that the intervention did not have a positive impact on the net cost.

Introduction

In this chapter we explore the implications of our results, interpret the findings and suggest future research directions. Specifically, we revisit each of the study objectives (to test the acceptability of the trial and intervention, the feasibility of recruitment and participation, the components of the intervention, adherence to the intervention and the likely effectiveness of the intervention) and identify what we have learned from the pilot study. We also review what we have learnt from the health economics evaluation, identify and reflect on the limitations of the study and present recommendations for taking the research forward.

What did we learn about the acceptability of the trial and intervention?

The study set out to assess the acceptability of the trial and the intervention by eliciting the views of managers and employees. As stated in Chapter 2, a limitation of the qualitative sample was that it included only one manager from the ‘non-adherent’ group of managers. Furthermore, as consent had not been gained for this purpose, we were unable to approach managers in the trust who did not participate in the study, although their views would have been of interest. We found that the trial and intervention were acceptable to those employees and managers who participated: no respondent identified any aspect of the research that they found unacceptable and many respondents made positive comments about the conduct of the trial and the intervention itself, as described in Chapter 4. Arguably, a further indicator of the acceptability of the trial to employees was the high retention rates achieved in the questionnaire survey (see following section).

We suggest that the construct of acceptability, although widely used in trials, has in fact limited meaning and value in that it artificially reduces complex phenomena to a unidimensional ‘yes’/‘no’ variable. In keeping with the study protocol and objectives we have reverted to using the term ‘acceptability’ to convey that none of our respondents found the intervention ‘unacceptable’; however, our narrative methodology enabled us to paint a much richer and more nuanced picture of respondents’ experiences of the intervention.

What did we learn about the feasibility of recruitment to the trial and participation in it and the feasibility of data collection?

Our study supported the feasibility of recruitment to the trial. We met recruitment targets for both employees and managers, although achieving this proved to be a resource-intensive process, as we expected. Recruitment was greatly aided by being able to use the onsite trust research team to approach participants. Many employee teams required several visits during the recruitment phase of the study for recruitment targets to be reached. Making contact with employees was not straightforward – teams were spread over a large geographical area, reflecting the dispersal of trust services, many employees worked off site and recruitment meetings were attended only by some staff. This may have resulted in reduced levels of recruitment.

Recruitment of employees took longer than expected and this meant that to keep within the time frame of the study not all employees could be recruited before the clusters needed to be randomised; however, we mitigated this by keeping staff who recruited employees blinded to the allocation.

We found no evidence of recruitment bias among employees in terms of gender and part-time working; however, compared with all trust employees, the study population was slightly younger. In terms of employee participation in the trial, there was a fall-off in the participation of employees between consent and baseline questionnaire completion, but a high retention rate between the initial questionnaire and follow-up. This was achieved by sending frequent e-mail reminders to employees to complete the online questionnaire and giving employees the option to complete a paper version of the questionnaire, which some chose to do.

There was good initial recruitment of managers: 41 out of 49 managers who were approached agreed to take part in the study. It appears that some groups of managers were nominated by senior managers to take part in the trial.

In terms of manager participation in the trial, a sizeable number dropped out at the very beginning of the study (between recruitment and the beginning of the e-learning intervention) and disappointingly only half of the managers completed three or more modules of the e-learning intervention (see What did we learn about adherence to the intervention?). Attendance at the facilitator-led induction meeting was good (with almost two-thirds of managers participating) but attendance dropped to just over 40% for the follow-up meeting. Only three managers attended the qualitative focus group, again a lower than expected level of participation.

We found some evidence that the managers who were more engaged with the study in the sense of completing more of the e-learning program were managing employees who had higher employee well-being scores both at baseline and at follow-up. One interpretation of this finding is that the group of managers who were more ‘effective’ at managing stress in their employees were more likely to participate in the study. It could also be the case that in some settings stress levels were very high, employee well-being levels were low and the managers were too stressed to complete the intervention. This latter group is a group who we would like to have engaged as they might have particularly benefited from the intervention. It is notable that levels of psychological distress were high among employees from both the intervention group and the control group compared with the rate in the general population.

Our findings on recruitment and participation need to be interpreted within the wider context described in Chapter 4: considerable organisational change and uncertainty within the trust, resulting in particular pressures on staff time and resources. Arguably this context made staff less responsive to additional demands on their time, such as the invitation to participate in the study. Furthermore, we found evidence of low levels of trust between staff and senior management and cynicism about the trust’s motivation for participating in the study. Participants also asked for reassurance that the research was unconnected to any internal trust monitoring and that the findings would be anonymised. These could perhaps have been factors affecting staff willingness to be involved in the study.

As noted elsewhere in this report, a mental health trust was a very particular organisation within which to undertake a study of stress and well-being. It could be argued that those working in the field of mental health would be more likely to be interested in our study, which would have helped with recruitment and participation; conversely, it could be argued that familiarity with mental health issues may mean that staff were less interested in joining the study, as they may feel that they know about most of the issues around work stress already.

We found that it was feasible to collect sickness absence data using the trust HR systems and that it was also feasible to obtain sickness absence data from social services when employees were employed by social services rather than the trust. We also found that it was feasible to collect data to carry out economic analyses using questionnaire results from employees, although as the employees largely consisted of a healthy population it may not be profitable to examine health service use as an outcome in a further study of a similar population.

Apart from members of our steering group, patient and public involvement involved discussion with managers and employees before the study and also two meetings to feed back our results to employees and managers after the study. Our study had no ‘patients’; instead, our participants to whom the results might be generalised were employees and managers working for the same organisation. In workplace studies the relevant portion of the general public to whom the study results are relevant are employees; hence, engagement with the wider general public would be less meaningful.

What did we learn about the components of the intervention?

We considered the components of the intervention in terms of the different modes of delivery of the educational intervention (e-learning, face-to-face meetings and telephone and e-mail support from a facilitator) and the content of the educational intervention.

We found that the managers who we interviewed were ambivalent about e-learning as a mode of educational delivery (and, as reported in Chapter 4, these managers had all completed three or more modules and so arguably were likely to be more positive about e-learning than those who engaged less). Managers identified both benefits and limitations of e-learning; most favoured a ‘blended’ approach, with more emphasis on face-to-face learning than had been the case with the guided e-learning intervention. There was a strong sense that managers liked hearing from each other at the face-to-face meetings, both about their experiences of the intervention and more widely about the problems of managing stress at work, and found this shared learning experience supportive. Managers made no use of the telephone and e-mail support offered by the facilitator, perhaps suggesting that this is an unnecessary component of the intervention.

In terms of the content of the intervention, we found that managers emphasised how they found the modules reaffirming of existing knowledge rather than necessarily providing them with significant amounts of new knowledge (arguably an unsurprising finding among a study population with considerable experience in the field of mental health). The program was considered no less useful for this. Indeed, managers welcomed the opportunity that the material provided to reflect on practice.

Given our finding that some managers dropped out of the study shortly after taking the initial quiz, it is possible that this aspect of the program was off-putting to some participants, especially as some of the managers who did not progress seemed to spend longer online on the initial modules than managers who went on to complete more modules.

An important finding was the contrast between the focus of the e-learning materials, that is on the competencies that managers require to manage stress at work as defined by the HSE management standards,32 and the sorts of competencies that managers and employees identified when recounting specific instances of stress and managerial support. In their accounts managers and employees emphasised what we have referred to as ‘emotional sensitivity’ (being compassionate, listening, ‘being human’, etc.),58 an aspect of competency not explicitly addressed in the e-learning program, although it was implicitly covered in some aspects of the program, such as in module 6 (Being proactive – helping individuals).

Furthermore, in talking about stress at work, employees and managers highlighted factors that they felt were beyond line managerial control (family pressures and personal health, the physical environment, the specific pressures of working in a mental health trust), conveying a rather different picture of employee stress from that presented in the e-learning modules.

What did we learn about adherence to the intervention?

Managers were deemed to have ‘adhered’ to the intervention if they completed at least three (i.e. half) of the six main e-learning modules. Given that only half (51%) of the managers achieved this, adherence was considered to be relatively low, although, as stated earlier, most of the ‘non-adherers’ stopped after the quiz and before the introductory module. Unfortunately (but perhaps unsurprisingly), those managers in the ‘non-adherent’ group did not take up the invitation to be interviewed and so we have little information from this group as to why they did not continue with the modules (except from the facilitator’s reports stating that in follow-up telephone calls to non-adherent managers no one gave negative feedback on the intervention itself).

What we know very clearly from those who we interviewed and from meetings that we observed is that time was a critical issue for all managers and thus is likely to have been a significant factor in non-adherence. From our monitoring of login times we found that ‘non-adherent’ managers spent more than twice as long logged in to the initial quiz as adherent managers, suggesting that they either found it more time-consuming to complete or were distracted by other duties while still logged on to the module. It seems that managers found it difficult to set aside time for the e-learning intervention in the way that they might for attending a meeting. This highlights the need for dedicated, protected time, which needs to be not only verbally agreed by senior management but also followed up and ascertained by any future study teams.

Our qualitative findings highlighted how in the group of managers defined by the trial as ‘adherent’ many spoke of not having had time to complete the suggested activities that formed part of the learning materials. This finding raises a question about the validity of the definition of ‘adherence’ adopted by the trial, in the sense of measuring meaningful engagement with the educational materials. Furthermore, there was the problem of the tension between the blinding requirements of the trial (meaning that managers were instructed not to tell their employees that they were studying the GEM program) and the learning activities themselves, which encouraged managers to engage in dialogue with their employees and seek feedback from them. There was thus conflict between the demands of the trial and the educational intervention and this perhaps militated against managers engaging as fully and successfully as possible with the educational intervention.

Our finding that there were higher levels of well-being in employees of ‘adherent’ managers both before and after the intervention implies that these managers were already having an impact on their employees’ well-being irrespective of the intervention. In this sense adherence could be considered as a marker for positive qualities in a manager.

What did we learn about the likely effectiveness of the intervention?

There was only a very small effect of the intervention on employee well-being and little effect on sickness absence. It is important to acknowledge that this was a pilot study and we did not therefore expect to test the effectiveness of the intervention definitively and the study was not powered to do so. The lack of a positive effect of the intervention on well-being and sickness absence must be interpreted in this context.

However, our study provided a wealth of learning about the possible factors influencing the likely effectiveness of the intervention and we identified the following potential explanations for our findings:

1. Seasonal effects. Data collection for the baseline questionnaire took place from June to September and for the follow-up questionnaire from January to April. There are well-known seasonal fluctuations in national sickness absence rates,64 with a peak in late winter, which could have affected our results.

2. Random variability. The CIs around the magnitude of the effect of the intervention were wide, given the relatively small numbers involved in the study, and thus our results do not provide a definitive test of the effectiveness of the intervention. Nevertheless, the size of the observed intervention effect was much less than the 3% change expected before the study.

3. Selection effects at baseline. Employees who were on sick leave at baseline were excluded from the study, perhaps leading to lower levels of sickness absence at baseline in the recruited sample than could be expected in the trust as a whole. By follow-up the recruited sample might be expected to have higher levels of sickness absence as part of the normal course of illness in an employed sample, more representative of the trust as a whole. Thus, one possible reason for the increase in levels of sickness absence at follow-up may be because employees who were off sick at baseline were not included in the study.

4. Sensitivity of the well-being measure. The WEMWBS has proved to be a useful measure of well-being and to sensitive to change in other studies. 65 However, it could be criticised as not being sufficiently work directed as it measures overall subjective well-being influenced by non-work-related factors as well as work-related factors. Nevertheless, if there had been a significant change in well-being at work as a result of the intervention it might reasonably be expected to have altered the WEMWBS score.

5. Time interval between the intervention and the follow-up employee well-being measurement. The constraints of the time frame of the pilot study meant that the interval between the end of the intervention and the employee follow-up questionnaire was relatively short – between 1 and 3 months. A benefit of this short interval was that it perhaps helped to reduce attrition and minimise the impact of organisational change (see the following point). However, it also meant that there was little time for any behavioural change among managers to take effect in terms of employee well-being.

6. Confounding effects of organisational change. The reorganisation that took place in the trust meant that one-quarter of managers in our sample changed roles during the study. This may have diluted the effect of any potential behaviour change encouraged by the intervention. Our finding that employee well-being was significantly lower in employees managed by this group of managers lends some support to our hypothesis of a relationship between manager practice and employee well-being.

7. The characteristics of this group of managers. The managers in this study all had considerable experience of mental health issues by virtue of working in a mental health trust and interviews with managers and employees indicated that many managers were ‘already doing’ what was suggested in the e-learning program. As such, it may be that this group already had relatively high levels of competency in supporting employees’ mental health and thus were less likely to be responsive to the educational intervention than a group with little experience of mental health issues.

8. The educational intervention. Educational theory suggests that for an intervention to be effective it needs to encourage ‘deep’ and ‘active’ learning. 66 Our findings that the guided e-learning intervention promoted reflection on practice (through the e-learning modules and face-to-face meetings) are suggestive of an effective learning approach. However, other findings (that managers felt that e-learning could easily become an overly mechanistic ‘click-click’ exercise and that they had insufficient time to engage with the learning activities and were in any case restricted by the requirements of the trial in engaging fully in the activities) indicated shortcomings in the educational experience, which may have limited the effectiveness of the intervention.

9. The study’s logic model. The theory of change underpinning this study was that a 7- to 8-hour guided e-learning educational intervention about management competencies as defined by the HSE would lead to changes in managers’ knowledge, attitudes and behaviour, which would impact on the well-being of their employees within 1–3 months of the managers undertaking the intervention. Our study, along with the work of others,37 has highlighted the complex nature of managing stress at work. In practice, managers and employees described the importance of ‘experiential’ knowledge and ‘emotional sensitivity’ (what some call ‘practical wisdom’) in managing stress; a non-linear relationship between changes in knowledge, attitudes and behaviour; how managers, in effect middle managers, were also employees experiencing their own stresses (particularly in relationships with senior managers); and how employee well-being was affected by many factors beyond the workplace. Given this complex picture, it was perhaps overoptimistic to expect to see clear evidence of an effect of the intervention in this pilot study.

What did we learn from the economic evaluation?

The pilot study provided estimates of intervention costs, sickness absence costs, health-care service costs and QALYs to inform the full-scale study. In terms of the feasibility of data collection for the economic evaluation, we were able to collect adequate data on intervention costs and the costs of sickness absence.

An important outcome of this pilot study was the estimation of the costs of the Guided E-learning for Managers intervention. The cost of the intervention per employee varied from £71 to £153 depending on the number of managers involved in different stages of the intervention and on whether or not we included facilitator training costs. For the purpose of the health economics analysis we included 2 days of facilitator training in our estimations. It should be mentioned that, once trained, the facilitator is able to deliver courses to a larger number of managers. Therefore, the cost of the intervention per employee is likely to be lower in the future. Given that the cost of the intervention is highly dependent on the number of employees supervised by each manager (seven employees on average in our case), we would expect the intervention to be less costly in settings with higher numbers of employees per manager.

A new organisation using the program would as a matter of course be given training and guidance on factors such as obtaining senior management commitment, the promotion of the program in the organisation and the motivation of employees with regard to using the program. This would be included in the cost of the program.

In this study we did not aim to compare costs and outcomes between the intervention group and the control group as adjusting the data for clustering effects would be problematic because of the small number of clusters. However, we provided non-adjusted estimates of costs and QALYs to inform future research. Our preliminary analyses suggest that:

• Given that this intervention is targeted towards the general population, which is relatively healthy, we are not likely to see significant shifts in QALYs or large changes in the use of health-care resources in the intervention group.

• The outcome of the economic evaluation largely depends on the cost of the intervention and productivity losses resulting from sickness absence. Therefore, in the full-scale study, preference should be given to other methods of health economics evaluation, such as cost–consequences analysis, rather than cost-effectiveness analysis.

In terms of the economic evaluation, the costs were high partly because of the small sample size. Not all of the costs that were specific to this study could necessarily be separated and thus the costs of future studies using this methodology are likely to be lower. Also, we costed for 2 days of training for the facilitator in this study but once the facilitator is trained he or she would not need further training to facilitate the program in a new organisation. Moreover, in a new study the organisation might want to provide the facilitator from in-house resources to save on costs.

Although the health economics analysis was based on a comparison between the intervention group and the control group, this pilot study was not powered to detect significant differences between the two groups.

In general, we concluded that the costs of e-learning would be reduced in a larger sample. Health economics data collection was shown to be feasible but will require a full trial for a definitive and detailed cost–benefit analysis. A recent review showed mixed results of the cost-effectiveness and financial return of worksite mental health interventions. 67

Limitations of the study

1. For reasons of confidentiality we were unable to link sickness absence to questionnaire data. This would have allowed us to explore differences in rates of sickness absence in subgroups defined by sociodemographic variables such as job band or between full-time and part-time employees. We were therefore unable to explore whether or not the intervention might have had a greater or lesser impact on sickness absence for certain subgroups. The anonymised sickness absence reports that we obtained from HR included basic employee demographics and the status of their manager’s engagement in the study, but these aspects could not be systematically analysed because this information was available only for employees who had a sickness absence in the two data collection periods and not for the employees who had no sickness absence in either period.

2. Other than the e-learning quiz as a measure of knowledge gained from the intervention, the study did not include any quantitative measure of the impact of the intervention on the managers themselves, including any change in manager behaviour, although it is possible that the intervention had some impact on manager well-being and this could have been a proximate outcome of the study. It would have been helpful to have had some quantitative measure of whether or not managers’ behaviours had changed as an intermediate outcome on the pathway to influencing employee well-being.

3. Information on the number of employees approached was difficult to obtain. No information was available on how many non-consenting employees each manager was managing. Information on managers who changed position during the study was not collected in a systematic manner, but we estimated this information through direct contact with managers and through the local research team, although it was unavailable for managers in the control group.

4. Because of a misprint there was an error in the answer codes for one of the 12 questions of the GHQ-12, and psychological distress scores therefore need to be interpreted with some care in this study. However, the sensitivity analyses showed that different methods of correction used to derive a full GHQ-12 score all yielded comparable results (see Appendix 2) and we would therefore maintain that this did not influence the analyses of whether or not the intervention had an impact on psychological distress.

5. Adherence was defined in terms of adherence to the e-learning part of the intervention and did not include participation in the activities recommended in the e-learning program. Moreover, although the intervention was acceptable to managers who took part, we do not know what the managers who dropped out or who did not consent thought about the intervention.

6. It was unfortunate that randomisation resulted in the learning difficulties service being allocated to the control group and the three acute mental health services being allocated to the intervention group. There were differences in the types of employees and the mode of organisation of services between the learning difficulties service and the acute mental health services, which might have contributed to unmeasured confounding factors in the analysis.

7. For reasons of confidentiality we were not able to obtain data on full-time equivalents (FTEs) for part-time employees. Therefore, for the purposes of the health economics analysis we assumed 0.5 FTE for part-time workers. For full-time employees who were on two different pay grades we assumed 0.5 FTE for each grade.

8. It is very difficult to fully adjust for all potential confounding factors in workplace studies. Managers’ behaviours can be influenced by their own senior managers’ behaviours, which we did not take account of in this study. Lack of consistent senior manager engagement with the project, in the context of major organisational change taking place at different times in different parts of the organisation, limited the controlled execution of the study.

Recommendations for taking the research forward

Generalisability

It is not known whether the difficulties encountered with manager recruitment were specific to this organisation or not. We have learned that e-learning is welcomed by some managers but not others. The mixed-methods approach involving questionnaire surveys of employees and interviews with managers and employees has been successful here and could be generalised to other types of organisations, such as the private sector, with different types of management structure, to test what types of intervention might be feasible and might be likely to yield productive results in terms of well-being and sickness absence.

Conclusion

We carried out a pilot study of a guided e-learning intervention for managers designed to improve employee well-being and reduce levels of sickness absence in a mental health trust. We recruited sufficient numbers of employees and managers. Managers who participated in the intervention found the guided e-learning intervention and the trial acceptable. We piloted methods for collecting sickness absence data and data for economic evaluation and found that this was feasible. There was only a very small effect of the intervention on employee well-being and little effect on sickness absence but the study was not powered to examine this definitively. The fall in well-being score across the study was much less among the employees of managers who adhered to the study. The study would have benefited from ensuring greater commitment from senior managers to the study and also greater engagement from managers and employees. We have identified several ways in which the study could be improved and have made suggestions for further research.

We conclude from the results of our pilot study that we should carry out further research to develop the blended e-learning intervention and refine the study design and methodology to improve adherence to the intervention. We conclude that it is feasible to carry out an economic evaluation of the intervention. The mixed-methods approach that we adopted in the pilot study was very valuable in illuminating the acceptability of the intervention and the reasons for adherence and non-adherence and for understanding the influence of the study context. We recommend that the future study should also take a mixed-methods approach.

Contributors to the project

The team would like to thank the following people:

• Rosemary Anderson for the provision of the e-learning program, for advice and support during the planning of the research, including the intervention timeline and the quiz module extension, for the training of the facilitator and for the practical advice on the conduct of the guided e-learning intervention.

• Gill Weston for conducting the induction and follow-up workshop sessions with participating managers, contributing to the planning of the e-learning conduct and reminder systems and following up participants by telephone and e-mail.

• Darren Hockley and colleagues at DeltaNet International Ltd for hosting and providing access to the APP e-learning program, managing access and providing manager uptake data reports.

• Pat Mottram and colleagues from the research team at Cheshire and Wirral Partnership NHS Foundation Trust for their support with regard to recruitment and data collection.

• Senior members from Cheshire and Wirral Partnership NHS Foundation Trust for their support of the intervention and the research project.

• Staff from Cheshire and Wirral Partnership NHS Foundation Trust, Cheshire West and Chester Council, Cheshire East Council and Wirral Council for their help in procuring anonymised sickness absence data.

• Mike Waring, Sandy Smith, Tom Power and Arouna Woukeu at the Pragmatic Clinical Trials Unit (Queen Mary University of London) for the set-up and maintenance of the trial database and advice on data security matters; Charlotte Clark at the Centre for Psychiatry (Queen Mary University of London) for her contribution to study planning and design; and Richard Hooper at the Pragmatic Clinical Trials Unit (Queen Mary University of London) and Mel Smuk at the Centre for Psychiatry (Queen Mary University of London) for additional statistical support.

• Amy Hoon, Anitha Manivannan and Zuhur Balayah at the Pragmatic Clinical Trials Unit (Queen Mary University of London) for support and advice on matters of quality assurance and Nancy Schumann and Natasha Stevens at the Pragmatic Clinical Trials Unit (Queen Mary University of London) for project management and administrative support.

• Amanda Fahy at the Centre for Psychiatry (Queen Mary University of London) for data management support.

• Joanna Gilbert (YourBrandedArt) for the design of the logo and newsletter for participants and Gary Schwartz (PodPros) for the design and maintenance of the study website.

• Mansel Aylward (Public Health Wales), Dame Carol Black (Nuffield Trust), Keith Whitfield and Andy Smith (Cardiff University), Andrew Oswald (Warwick University), Colin Mackay (Health and Safety Executive), Geoff Smith (National Association of Schoolmasters Union of Women Teachers) and Peter White (Queen Mary University of London) for their advice as members of the scientific steering committee.

• James Shearer (King’s College London) for reviewing the health economics evaluation.

• Emma Donaldson-Feilder (Affinity Health at Work) for her advice regarding use of the line manager competency indicator tool and 360-degree feedback.

• All of the participating employees, managers and key informants for giving up their time to contribute to this research project.

Other acknowledgements

The WEMWBS was funded by the Scottish Executive National Programme for Improving Mental Health and Well-Being, commissioned by NHS Health Scotland and developed by the University of Warwick and the University of Edinburgh and is jointly owned by NHS Health Scotland, the University of Warwick and the University of Edinburgh.

The EQ-5D^™ questionnaire is a trademark and copyright of the EuroQol Group.

Contribution of authors

Stephen A Stansfeld (Professor in Psychiatry, Queen Mary University of London) was the chief investigator. He contributed to the study design, study management, the oversight of study conduct and the initial writing and final editing of the report.

Lee Berney (Qualitative Researcher, Queen Mary University of London) conducted the interviews and focus group meeting with study participants and contributed to data collection and the writing of the report.

Kamaldeep Bhui (Professor in Psychiatry, Queen Mary University of London) contributed to the study design, the original project submission and reviewing the report.

Tarani Chandola (Professor in Medical Sociology, University of Manchester) contributed to the study design and management, the review of existing research on e-learning interventions and writing of the report.

Céire Costelloe (Statistician, Queen Mary University of London) contributed to the statistical analysis and writing of the report.

Natalia Hounsome (Health Economist, Queen Mary University of London) planned and performed the health economics analysis and contributed to writing of the report.

Sally Kerry (Reader in Medical Statistics, Queen Mary University of London) contributed to the study design and led on the statistical analysis and writing of the report

Doris Lanz (Trial Manager, Queen Mary University of London) contributed to study co-ordination, data collection and day-to-day project management and the writing of the report.

Jill Russell (Senior Lecturer in Health Policy and Evaluation, Queen Mary University of London) contributed to the study design, oversaw and reviewed the qualitative research and contributed to the writing of the report.

Disclaimers

This report presents independent research funded by the National Institute for Health Research (NIHR). The views and opinions expressed by authors in this publication are those of the authors and do not necessarily reflect those of the NHS, the NIHR, NETSCC, the PHR programme or the Department of Health. If there are verbatim quotations included in this publication the views and opinions expressed by the interviewees are those of the interviewees and do not necessarily reflect those of the authors, those of the NHS, the NIHR, NETSCC, the PHR programme or the Department of Health.

Content of the facilitator training

Content of the face-to-face manager sessions

Key informant topic guide

• Respondent’s background and relevance to project.

• Perceptions of workplace stress: key issues, causes, factors fostering and hindering efforts to reduce workplace stress and absenteeism, current systems/procedures in place (the perceived advantages, benefits and/or deficiencies of current approaches).

• E-learning interventions: experience of/attitude towards current practice. How e-learning fits in with/builds on existing strategies.

• Possible influences on (a) the delivery of the intervention and (b) the effects of the intervention.

• Suggestions of relevant individuals, organisations and literature to follow up.

Manager topic guide

Employee topic guide

• Tell me about your role. How long have you been in post? Are you responsible for any staff yourself?

• What skills does your manager need to create a healthy workplace (and to reduce workplace stress)?

• What needs to be in place at the workplace to reduce stress/promote health?

• What makes you feel well at work? What makes you unwell at work? What are the positives about the job? What are the stressors?

• Please give as much detail as possible about a specific instance of feeling stressed. How was it handled?

• Opinions on the questionnaire: content, layout, etc.

• Acceptability of the trial: the registration process, information given, technical aspects, being sent login information for an online questionnaire, token codes, views on confidentiality, filling in the questionnaire at work (when found time to complete it), contamination (did you know who else was taking part?).

• The trial blinding – did you know/suspect your manager was participating?