A multimethod study of NHS 111 online

NHS 111 services and COVID-19

In response to the COVID-19 pandemic, NHS England issued guidance that the public should call NHS 111 if they had symptoms of COVID-19 or a suspected contact with a confirmed case of the virus. In March 2020, calls to NHS 111 doubled to nearly 96,000 calls per day. A dedicated NHS 111 online coronavirus service (111.nhs.uk/COVID-19), with a tailored NHS Pathways module, was launched on 26 February 2020; 35,000 people used the site in its first weekend of operation. At the peak of the first wave of the pandemic in the UK, the online service was used 950,000 times per day. 2 Throughout the pandemic, the NHS 111 online COVID-19 service continued to evolve, with additions to the algorithm to include questions about overseas travel, updated symptom profiles, advice about recommended treatments and options for onward referrals, as well as adaptations to the URL and search engine optimisation to make the online service easier to find. In June 2020, questions about coronavirus were incorporated into the core NHS 111 online service. In the Autumn of 2020, NHS 111 online began trials of a new online booking facility that allowed patients who were directed to attend an ED to select an arrival time. In December 2020, as a way to reduce ED overcrowding, the ‘111 First’ or ‘call-before-you-walk’ scheme was introduced, initially in London and Portsmouth, which required patients to call NHS 111 before attending the ED. Some EDs have a facility that allows walk-in (ambulatory) attenders to use NHS 111 online on a computer tablet in the reception area of the ED before being seen. In November 2021,3 the NHS England’s national medical director launched the ‘Help Us, Help You’ campaign, which urged people to use NHS 111 online for urgent but not life-threatening medical issues to allow the NHS to care for more seriously ill people in EDs.

The wider digital care landscape

NHS 111 online sits within an urgent and primary care landscape in which digital technologies are increasingly being enrolled in health-care delivery, ranging from electronic health records and e-consultations to decision support systems for triage and assessment. However, evidence addressing key questions about how online systems are used, their effectiveness and their impact on wider health service demand is limited. Evidence regarding accuracy of assessments using symptom checkers is also contradictory (see Bisson et al.,4 Sole et al.,5 Powley et al. 6 and Anhang Price et al. 7), Typically, small-scale studies have evaluated symptom checkers and the results are mixed, with some noting that they may encourage users to seek care from health services when self-care is reasonable. 8,9 A more recent systematic review of online symptom checkers found little evidence of harm, but noted the paucity of large studies and longer-term follow-up data. 10 Although there is potential for these systems to support self-management, there is a widespread concern that they may drive demand towards consultations ‘with a doctor’ and/or to emergency services. 6,11 Research has also identified that some patients experience difficulties using symptom checkers: they may be confused,12 lack confidence or simply struggle to navigate software systems. 13,14 Alongside these barriers to use, there is a concern that web-based sources of health information can heighten anxiety and that the quality of online health information is variable. 15,16 These findings sit within the context of a wider literature that has raised concerns about accessibility of digital technologies and about inequalities of access and use of information and communication technologies (ICT). 17–20

Demand for urgent and emergency care services

The demand for urgent and emergency services has continually risen each year despite various initiatives that have attempted to manage and reduce demand. Between 2018 and 2019, ED attendances rose by 4% to 24.8 million attendances,21 and then reached 25 million attendances in 2019–20. This represented a 17% increase since 2010–11 and there were no signs that this upwards trajectory would change. However, in March 2020 there was an approximate 30% reduction in ED attendances, which was attributed, in part, to the COVID-19 pandemic (people were heavily discouraged from attending face-to-face services, and remote primary and secondary services were made available). 22 In March 2021, there were 1,691,000 attendances at EDs, representing a 14.5% increase from the same month in 2019, confirming that the reduction in demand was temporary. 23 Alongside EDs, the NHS offers a range of urgent care services, including urgent care centres (UCCs), minor injury units, walk-in centres and GP out of hours. These services have seen similar increases in demand. NHS England estimates that there are 110 million urgent same-day patient contacts, of which some 85 million are urgent GP appointments per year. 22

Both NHS 111 online and the telephone 111 service are seen as services that can ‘empower people to manage their own health and care’. There is some expectation that this could also help reduce demand for face-to-face urgent and emergency care services or at the very least halt the upwards trend in service use. 24 NHS 111 services form a key plank in the NHS Next Steps Forward View,25 which was designed to improve access to (appropriate) services under the banner ‘right person, right place, right time’. NHS 111 online in particular is seen as an important potential ‘channel shift’ away from face-to-face care delivery and as a way to reduce demand for the telephone 111 service by offering ‘a fast and convenient alternative’. 21 Earlier evaluations of the impact of NHS Direct services on ED attendances did not demonstrate that these services reduced demand, and an analysis of the impact of the NHS 111 telephone pilots found no evidence that these changed urgent and emergency service use. 26,27 There have been claims that the NHS 111 telephone service is too risk averse and is more likely to dispatch an ambulance or advise people to attend the ED. 28,29 Recent analysis of patient compliance suggests that many patients do not follow the advice given by the 111 telephone service and that non-clinical advisors may direct more patients to the ED. 30

Rationale for this project

This research project was designed in response to the NIHR Highlight Notice 18/77 and call for research to evaluate NHS 111 online, specifically addressing knowledge gaps regarding the impact and sustainability of this new service. It builds on our previous projects about the NHS 111 telephone service (HSDR 10/1008/10), the NHS Pathways software (SDO 08/1819/217) and on urgent care sense-making (HSDR 14/19/16). We also draw on our previous theoretical and empirical studies of web technologies and seek to contribute to the wider literature on digital health-care technologies. 31,32

There are a small number of studies on the experience of using NHS 111 telephone services. These suggest that the telephone service is acceptable30 and that users are largely satisfied with the service. 33,34 Media coverage about the service and the views of some other stakeholders, notably general practitioners and paramedics, have been less positive. 35,36 Knowles et al. 37 showed that, although there was good awareness of the NHS 111 telephone service, some groups, notably older people, men and those without longstanding illness or disability, were less likely to use the service. Given that NHS 111 online is a relatively new addition to NHS urgent and emergency care provision, there is an even more limited evidence base about this service. Because NHS 111 online service is used directly by patients and the public, without a call handler or clinical intermediary, this raises additional concerns about digital and health literacy and equity of access. The concept of health literacy considers whether or not and how people are able to understand and use health-care information and services to support their health decisions. The concept of eHealth literacy adds a digital dimension to consider whether or not and how people are able to use digital health information and services. We know that there is a strong link between health literacy and eHealth literacy. In addition, we know that people with lower health literacy and lower eHealth literacy are less likely to use online sources of health advice and care. We were, therefore, keen to examine eHealth literacy in relation to NHS 111 online.

A recent systematic review10 concluded that there was ‘a high level of uncertainty about the impact of 111 digital on the urgent care system and the wider healthcare system’. 10 This called for investigations of the pathways followed by patients using the service, and also of the barriers to the use of online symptom checkers by people who are less familiar with digital technology. Our project looks at where NHS 111 online fits in patient pathways to care and within the wider urgent and emergency care system, and provides the first large-scale examination of eHealth literacy, addressing these two areas of research need.

How this study builds on our previous work and other related studies

Our previous work has looked at the use of a computerised decision support system (CDSS) to support triage and assessment in urgent and emergency care. A study by Pope et al.,38 which was completed in 2010, examined the use of NHS pathways and the NHS CDSS used by 999 ambulance and out-of-hours urgent care services to understand how call handlers triaged and managed patients seeking help. A follow-on project, HSDR 10/1008/10,39 which was completed in 2012, investigated the work and workforce implications of NHS 111 and was able to look at the telephone service itself, the technologies used and the wider network of urgent care provided in primary care. HSDR 14/19/16,40 which was completed in 2018, looked beyond NHS 111 services to explore sense-making strategies and help-seeking behaviours associated with the use and provision of urgent care services.

The project reported here was funded alongside a ‘sister’ project (conducted by the University of Sheffield and led by Janette Turner, NIHR127655/ISRCTN5180111241), which explored the impact of NHS 111 online on the NHS 111 telephone service and on the wider urgent care system. The Sheffield project provided an updated evidence review, a time series analysis of the impact of the online service on the NHS 111 telephone service, a survey of service users and a cost–consequences analysis. The Sheffield team also interviewed 16 NHS staff across four NHS 111 sites to explore the impact of NHS 111 online on their workload and their views about how the service was implemented, used and could be developed. Our study augments this aspect of the Sheffield study, providing a larger qualitative interview study of a range of NHS staff and other stakeholders to understand the workforce impacts of NHS 111 online.

This report seeks to fill the gaps in the evidence base about NHS 111 online to inform the future development of NHS 111 services and to meet some of the research needs identified by NHS England for the evaluation of this digital service.

Chapter 2: study design and methods

Chapter 2 gives an overview of the study design, including the key changes that extended the study timeline, such as the impact of the COVID-19 pandemic. We discuss the challenges of carrying out research in areas of high health need and engaging less research active sites. We also present our data collection and analysis methods, as well as details of our embedded PPI activities throughout the study.

Chapter 3: pathways to care

Chapter 3 looks at where NHS 111 online sits in the landscape of service provision and at patient pathways to care involving use of NHS 111 online. We report our analysis of the telephone interviews in England with health-care practitioners in primary and urgent care, emergency departments and dental services, and with charity organisations representing vulnerable people. We also draw on a range of policy documents and diagrams detailing pathways into urgent care and use these to build on and inform the draft pathway diagram included in our original research proposal and to reflect on what we have learnt about NHS 111 online and pathways to care.

Chapter 4: work, workforce and impacts on the wider health system – interviews in England and Australia

Chapter 4 looks at the work and workforce arrangements and impacts on the wider health system associated with NHS 111 online in English primary, urgent and emergency care services. It also reports data from interviews in Australia about Healthdirect and our comparison of this service (including the online symptom checker) with NHS 111.

Chapter 5: is there evidence for differential access to and use of NHS 111 online?

Chapter 5 presents the findings of a survey designed to address the research question ‘Is there evidence for differential access and use of NHS 111 online?’ and asked two main questions:

1. What are the demographic characteristics and preferences of users of NHS 111 online compared with those who have not used the NHS 111 online service?

2. What is the relationship between eHealth literacy and the use of NHS 111 online?

Chapter 6: discussion and conclusions

This chapter summarises the key findings, discusses these findings in the context of the wider literature and examines the implications of this work. This chapter also includes consideration of the limitations of this research study.

Changes from the original proposal

A key change was the study timetable. Although designed initially as an 18-month study, the start date was moved to accommodate the relocation of the PI to the University of Oxford and the study was then affected by the COVID-19 pandemic. We experienced a series of COVID-19 pandemic research pauses, which meant that ethics, contracting and governance processes and data collection were stopped and had to be restarted, and the study duration had to be extended (with the agreement of the funder). In addition, COVID-19 risk management and changes to ways of working in research and in the NHS meant that we had to rethink some key elements of our research design.

The original study was conceived as an ethnographic case study, and we planned to combine the survey of eHealth literacy with substantial qualitative observation and interviews focused on case studies across eight different geographical locations. The COVID-19 pauses to data collection and the shift to remote data collection meant that we had to rethink this case study design and adapt data collection while trying to keep the study moving forward.

We discussed design changes with the Study Steering Group and worked with potential study sites to ensure that our study was feasible and would deliver on the core aim and objectives. We also worked to ensure that we protected the research team, patients and health-care workers, and minimised the need for additional work by parts of the urgent and emergency health-care system that were experiencing considerable demand and workload during the pandemic.

The key changes to the design concern the survey and the case studies, and are documented in Table 1.

The extended timeline also led to some changes to the research team. Among the co-applicants, Lucie Lleshi left the study in 2021 following her change of job and the merger of the Southampton and Hampshire Clinical Commissioning Groups. Emily Petter joined in her place to provide the perspective of a policy decision-maker and health service commissioner to the study. (Emily is senior commissioning manager – Urgent & Emergency Care, South West Hampshire; NHS Hampshire, Southampton and Isle of Wight CCG.) Within the UK study team, Jennifer MacLellan was seconded for 8 weeks to a COVID-19 research study during one of the study data collection pauses. Kate Churruca and Louise A Ellis joined co-applicant Jeffrey Braithwaite to support the data collection and analysis for the Australian component part of the study.

Research in areas of high health need and engaging less research active sites.

In the original proposal, we outlined that our purposive sampling of cases would include sites in which the NHS 111 online service was established and sites in which the service was relatively new. This included the original pilot sites, sites that had been studied in a parallel research study led by the University of Sheffield and some sites that had only recently commissioned the service. At the time of writing the proposal, some areas were ‘beta-testing’ the system or favouring ‘soft’ roll out, whereas others saw the service as a core part of service provision. We were also mindful that people who live in regions of the UK in which the burden of need is greatest are often under-served by research, and we wanted to support NIHR’s goal of bringing applied research closer to these communities. 43 We, therefore, made particular efforts to recruit sites that we had not previously worked with. Our site selection was informed by area deprivation scores and sociodemographic characteristics of the population, including age profiles and the presence of ethnic minority groups.

Selecting research sites

We wanted to ensure that our study included sites reflecting deprivation and high health need. Rai et al. 44 explored how research sites are selected for multicentre randomised trials and showed that chief investigators (CIs) and teams tended to select research active sites with established governance processes and research delivery support infrastructure that enable recruitment targets to be met within tight study time frames. However, they also found that CIs who ‘broke the mould’ and actively identified and nurtured new sites reported as much success as those returning to familiar, highly research active sites. Inspired by this work, we mapped areas of socioeconomic deprivation, high diversity, low literacy and high health need, and targeted these areas for our recruitment.

We used the Index of Multiple Deprivation (IMD) to map areas of greatest deprivation in England owing to their known association with high health need. 45 We also made particular efforts to engage with general practices, district hospitals and community trusts that were less research active, and those serving more ethnically diverse populations. We used literacy scores from the National Literacy Trust as a proxy indicator of potential barriers to the use of and access to digital health technologies. Together these scores and indicators helped us to target our site selection to primary and secondary care services in North West Coast, Yorkshire and Humber, West Midlands and South Midlands, Eastern Counties, Kent Surrey and Sussex, London, Wessex and Thames Valley.

Qualitative interviews: England

We conducted telephone semistructured interviews to explore the workforce implications and work associated with NHS 111 online. This addressed objective 3, to describe the workforce for NHS 111 online (RQ3) and assess the impact of different work arrangements on the urgent and emergency health-care system (RQ4 and RQ5). Given that the parallel study led by Janette Turner at the University of Sheffield had undertaken interviews to examine the impact of the online service on the NHS 111 telephone service, looking at changes to casemix, workload and staff morale and retention,41 we focused our English interviews on the ways that NHS 111 online impacted on the work of other (i.e. non-NHS 111) services.

We developed an interview topic guide using open-ended questions and prompts to ask about experiences of the interface between NHS 111 online and other services, as well as the work undertaken because of or associated with NHS 111 online. We were interested to know about work arrangements, everyday processes and work content; for example, whether or not GP receptionists recommended NHS 111 online to patients, how they managed patients ‘referred’ by the service and any new tasks they undertook because of NHS 111 online. The topic guide asked some introductory questions about the interviewee’s professional role, the service provided, general awareness and knowledge of NHS 111 online and experience using or receiving/seeing patients/service users who had used NHS 111 online. Subsequent questions asked about the impact on work and workforce, including the skills, time and other resources linked to the use of NHS 111 online. The interview guide closed with consideration of broader benefits/challenges of the 111 services and asked the interviewee to reflect on how the service was developing and the longer-term implications for their work and workforce.

Qualitative interviews: Australia

A qualitative interview study was conducted with Healthdirect staff and external stakeholders to explore the work of this service and perspectives on the role of Healthdirect in the Australian health-care system. These interviews addressed objective 4 to compare the workforce experience of NHS 111 online with the Australian Healthdirect service, with the aim of learning from their experience of introducing similar virtual triage and assessment services.

Survey of eHealth literacy and user scenario preferences

We used a cross-sectional survey to address the research question ‘Is there evidence for differential access and use of NHS 111 online?’. The survey was first used to understand the demographic characteristics and preferences of users of NHS 111 online compared with people who had not used the service. Second, the survey was designed to understand the relationship between eHealth literacy and the use of NHS 111 online.

To capture those who used NHS 111 online compared with those who had not used it, the survey asked questions about age group (18–24, 25–34, 35–44, 45–54, 55–64, 65–74, ≥ 75 years), sex (female, male, non-binary, prefer not to say), educational level (two levels were coded for the logistic regression analysis: no formal qualifications, one GCSE or higher) and employment status (employed/self-employed, unemployed/unable to work, retired, student, homemaker, other). Respondents were also asked if they had a long-term or chronic condition (yes/no). The survey consisted of sociodemographic and socioeconomic questions, which previous research has indicated are associated with influencing levels of eHealth literacy. 47,48 This literature is explored further in Chapter 5. We also asked respondents to indicate whether or not they had used other urgent and emergency care services previously (yes/no) and these included NHS 111 telephone service, GP out of hours, urgent care or walk in centre, ED and/or 999.

In addition to these questions, we also included a set of 10 scenarios that described common presenting conditions or urgent care needs, designed to explore awareness of the online service and preferences for online/call handling for different types of symptoms from less to more serious. Different health-care needs might motivate service use regardless of previous experience or literacy. In considering who uses NHS 111 online it is important to consider the different reasons that people seek online help because use might be driven by purpose in addition to demographic characteristics. These scenarios were developed in consultation with the NHS Digital/NHS 111 online team, including their PPI lead, and were informed by data from our previous research and the literature. They were designed to represent common urgent care and online symptom checker presentations, as well as some potentially more serious symptom sets. Because the scenarios were designed to be presented in a survey (rather than a qualitative interview), the scenarios were brief (Table 8) and respondents were asked ‘In the following scenarios, how likely are you to use NHS 111 online?’. Each scenario was rated on a five-point Likert scale from ‘very likely’ to ‘very unlikely’.

The research set out to understand if there was a digital divide in abilities that might prevent some service users accessing the online platform or result in them using it in unexpected ways. We set out to understand the relationship between eHealth literacy and the use of NHS 111 online to understand if eHealth literacy and sociodemographic characteristics are associated with the use of NHS 111 online. The eHealth Literacy Questionnaire (eHLQ), a validated 35-item 7-scale questionnaire,42 was used to examine this.

Two main eHealth literacy scales have been developed. First, the eHealth Literacy Scale (eHEALS) was developed by Norman and Skinner49 to assess people’s ability to engage with eHealth, with the purpose of informing clinical decisions and health promotion planning. Although the scale has been widely used in eHealth literacy studies, recent validation studies have cast doubts on the tool’s dimensionality. 50–52 Furthermore, the eHealth and digital landscape has evolved substantially since 2006, and use and acceptance of digital health technologies have changed. Norgaard et al. 45 developed the eHealth Literacy Framework, which led to the development of the eHealth Literacy Questionnaire (eHLQ). 42,53

The eHLQ combines digital and health literacy and considers both individuals’ competences and individuals’ experiences and interactions with technologies and services. It consists of 35 items grouped into seven domains measuring different aspects of eHealth literacy, including the use of technology to process health information; understanding of health concepts and language; the ability to actively engage with digital services; feeling safe and in control in using online services; motivation to engage with digital services; access to digital services that work; and access to digital services that suit individual needs (Table 9). Each item on the eHLQ was scored using a 4-point ordinal scale, from strongly disagree (1) to strongly agree (4). Each domain contains between four and six items, and an average score was calculated for each domain. The eHLQ is not designed to produce a single measure score across the instrument but is a measure with seven different domains of digital literacy, and these are designed to be reported separately.

The survey was administered via primary, urgent and emergency care settings and the NHS 111 online service. This was carried out in three ways:

1. The survey was made available through a mail out SMS link sent to eligible participants in participating general practices. Eligible participants were all people aged over 18 years who had consented to receive SMS messages from the general practice.

2. The survey was administered in participating emergency departments and urgent care centre sites through public display of the survey QR code on posters in waiting room areas, and in one case by e-mail to recent attenders. A hyperlink to the study survey was provided on the study web page hosted by the Nuffield Department of Primary Care Health Sciences website to support this. To aid survey completion in these urgent and emergency care settings, a research nurse was available to support people wishing to access and answer the survey, using a computer tablet provided by the study team or the site.

3. The survey was distributed nationally following an NHS 111 online consult via a dedicated hyperlink. All survey responses were assigned a reference number that encoded the administrating organisation so that we could group responses by location; no personal identifying medical information was recorded. Having piloted the survey with the Southampton Consult and Challenge Group, we estimated that the survey would take approximately 15 minutes to complete.

How PPI informed this study

Patient and public involvement in research helps to make it more relevant and useful to the end-users. 57 People with lived experience of deprivation and disadvantage have high health need but are often at greater risk of exclusion from digital systems used to access healthcare due to a lack of access to digital technologies, poor digital literacy or health literacy. 58–61 PPI members of the SSG, while reflecting different age groups from 20s to beyond retirement age, with different employment and health experiences, were all confident in using and had access to digital technologies. Thus we sought additional PPI representatives with experience of digital exclusion through the homeless health peer advocates of the charity Groundswell and members of the public through the Deep End Sheffield cluster PPI Group. We also sought feedback about the survey from the Consult and Challenge group in Southampton.

Groundswell are a charity who work with people who have experience of homelessness, offering opportunities to contribute to society and create solutions to homelessness. The Groundswell Homeless Health Peer Advocacy (HHPA) service supports people experiencing homelessness to address physical and mental health issues. They work to improve people’s confidence in using health services and increase their ability to access healthcare independently.

The Deep End Sheffield PPI group are an established PPI Panel of patients who live in communities in Sheffield with the highest IMD scores; they have contributed to a number of previous studies. (The ‘Deep End’ movement was started to support GPs working in the most deprived areas by Professor Graham Watt in Scotland.)

The Consult and Challenge group are a co-production group working in Southampton City. They are experts by experience (people with direct experience of using health and care services) and membership of the group is open to people aged 14 and over, and includes people with physical and mental health disabilities and people with long-term health conditions. They assisted us by pilot testing the draft survey and telling us about their experiences of using NHS 111 services and accessing urgent care.

We held three PPI events, one with Groundswell and two with Sheffield Deep End PPI to enhance our knowledge of people who might experience digital exclusion or other forms of disadvantage and vulnerability. We used these events to try to mitigate against the potential blind spots of the research team during the analysis and to explore experiences and perceptions in greater depth from a service user perspective. 59,60 The Deep End meetings involved five men and seven women aged between 30–65 years, three participants were Asian and two were African/Caribbean. All participants lived in areas of high social deprivation in England.

Discussion with the seven Groundswell participants involved people working with people experiencing homelessness and people who inject drugs, as peer supporters (i.e. they had shared experiences with the client group serviced by the charity). These discussions highlighted the potential utility of online health services for this group, especially for access to an emergency dentist and to look up information about a condition or medication. This prompted us to widen our English interview sample to include dentists. This group also identified the ED as the main point of access to healthcare for their client group and for patients unable to negotiate digital access systems. The group highlighted the digital gap and raised concerns about access to technology, language confidence and knowledge of the health system, emphasising how lack of hardware (e.g. smartphone) but also data services and Wi-Fi meant that many of their clients could not access NHS 111 online. These problems were exacerbated in the pandemic when some places that offered public free Wi-Fi such as cafes or libraries were closed. The group helped us adapt the survey recruitment by including provision of a digital tablet supported by a research nurse to facilitate completion by those who were unable to complete the survey unaided.

The first Deep End PPI group meeting was used to examine the preliminary eHealth literacy survey analysis, in particular the scenario questions. Two scenarios, one involving a child with an unresolved fever and another of an adult with chest pain that goes away, had been unexpectedly identified as scenarios that survey respondents would consider using NHS 111 online for. We were able to discuss the rationale behind the responses to learn the nuances of help seeking in relation to personal interpretations of risk. We also used this Deep End PPI group to explore the diagram from NHS England of the different emergency services available with guidance on how to access them and the range of services mentioned in the English interviews. The participants agreed that the wide range of services named represented their understanding of the help seeking/service landscape. They suggested that an explanation of the patient pathway/journey was needed alongside the ‘messy diagram’ to maximise understanding.

A second Deep End group meeting was convened to check the credibility of the interview analysis. We generated statements from the interview data themes and illustrated these with quotes from the interviews to show how we had interpreted what was being said. The group confirmed that people are not always sure how serious their condition is and do not always know where is the best place to go for help. Members of the group described their own health seeking scenarios and challenges they faced negotiating the interface between NHS 111 online and other services, for example, one woman was sent to ED despite saying she did not need an ambulance so the next time she experienced those symptoms she sought an out of hours GP appointment rather than phoning NHS 111. They confirmed that people are often sent to different parts of the health system before they finally get an answer to their problem. The participants shared experiences of delays in reaching the most appropriate healthcare professional, difficulties in making GP appointments and attending ED when they wanted more timely assessment or advice. The group discussed the increased use of remote consultations in the pandemic noting that not everyone has a smart phone or regular data to be able to use digital services. One participant needed a mobile phone with large numbers and a loud ring tone to make phone calls and deliberately chose not to have a smart phone, this meant she could not use online services on her phone. Other group members talked about cost barriers to using online services, especially where mobile phone contracts were ‘pay as you go’. One participant described being cut off from a conversation with her doctor because she ran out of credit. They noted that people may not understand how to use digital technologies and may lack confidence in reading and writing English which can be a barrier to using online services.

The PPI groups from Groundswell and Sheffield Deep End also discussed how best to present information from the study for patients and public audiences.

Study Steering Group (SSG)

A SSG provided oversight for the research. The group was chaired by Professor Fiona Stevenson (University College London Department of Primary Care) and comprised independent members representing methodology, primary, urgent and emergency service providers, clinical commissioners, and national NHS 111 policy/NHS Digital. The SSG met five times online during the study (additional meetings were held due to the extended timelines for the study).

Additional national NHS 111 team liaison

The study shared information and early findings from the study with key members of the NHS 111 online team in NHS Digital. Online meetings and emails were used to advise the NHS 111 team about study progress and to share updates and information about changes to the NHS 111 online service. We presented interim results for discussion and credibility checking and have plans to engage with the team to assist with dissemination and policy facing outputs. In addition to the liaison meetings the study PI was invited to attend the NHS 111 user forum meeting in Autumn 2020 and was able to meet with the PPI lead for the NHS 111 online team to discuss aspects of the survey scenario content.

Research Ethics Committee (REC)

Approval for the English fieldwork and survey was granted from the London Stanmore Research Ethics Committee (reference 20/LO/0294) with minor amendments to add Research sites and to make changes as a result of the COVID-19 pandemic. The Australian Healthdirect interview study was approved by Macquarie University Human Research Ethics Committee (reference number 52021985728536).

The (in)visibility of NHS 111 online

The most striking finding across our interviews was that just under half of our 80 respondents were not aware of NHS 111 online:

If I’m really honest I don’t think I was aware that there was a new online service until I got the information through for this study.

PC18, GP

To be honest I wasn’t aware that 111 was online.

PC22, reception manager

I didn’t even know there was a website.

CH10, peer support worker

None of the dental respondents and only one of the charity stakeholder interviewees was aware of NHS 111 online. A total of 23 (of 33) interviewees in primary care and 19 (of 27) of the urgent and emergency care interviewees were aware of the service. Lack of awareness did not appear to be linked to particular roles or areas of the country, as a mix of receptionists and medical staff from different areas said that they had not heard of the service (Table 10).

One care navigator described how she signposted people to NHS 111 online when the practice had no more appointments:

... when we have no more appointments I say call 111 or they can go 111 online, because I know there is online but I don’t actually use it.

PC09, care navigator

However, health-care professionals mentioned typically advising patients to call the NHS 111 telephone service:

I haven’t ever recommended online myself. I tend to say ring 111, it’s something [signpost to NHS 111 online] that I have not thought to do.

ED22, UCC nurse

One difficulty was that interviewees used the generic term ‘111’ to refer collectively to the NHS 111 telephone and online service. GP staff in particular tended to think of NHS 111 as something associated with patient records and ‘paperwork’, and typically referred to notifications received from NHS 111:

I don’t think we have any interaction with the online 111 beyond that it might direct people to contact their GP. The main one we get is, basically, the consultation they have had with 111 comes through to us as an electronic document, the report, that’s the main way and they [NHS 111 telephone service] have directly bookable slots that they can use every day.

PC07, GP

I don’t know if it specifies one or the other, I think it [electronic report] just says 111.

PC19, nurse

This staff uncertainty and lack of awareness of NHS 111 online was, they felt, a reflection of the way that patients and the public understood and used the services:

I haven’t heard people use online so much, normally they say they have spoken to someone, I don’t think the online is as widely used as the phone service.

ED01, ED nurse

I must say most of the time its people saying they have phoned, they have spoken to somebody from 111, I very rarely hear online. I don’t hear that very often I must confess, it’s mainly that they’ve actually spoken to somebody.

ED19, ED nurse

I think there has been maybe a handful of cases that have been online, but most of them ring 111 or try to contact the GP who has put them through to 111 and they end up coming here [to ED].

ED24 ED doctor (trainee)

Not necessarily as a separate service, I mean patients will refer to 111 and don’t necessarily differentiate, you know they say they’ve looked online with 111 and this is what I’ve done or they may say I’ve spoken to 111 but they often don’t differentiate between the modality that they have been using.

ED27, MIU nurse

People don’t really clarify, if its 111 they say they’ve either spoken to somebody to somebody at 111, they don’t necessarily say they have been online on 111... usually they tell you they’ll been online [to look up symptoms], they’ll say they’ve been Googling it.

ED15, ED doctor

Very occasionally, one of our interviewees reported that they were aware of the NHS 111 online service, often because they had used it themselves:

I personally am aware that there is an online 111 and I do let patients know as obviously rather than calling you can look at your symptoms online. I find it quite easy for personal use. It’s easy to pop in your own symptoms and keep informed quickly and easily. I think it’s really good.

PC12, care navigator

Given that our interviews spanned the 2020–21 COVID-19 pandemic, some of the later interviewees reported using NHS 111 online specifically for COVID-19 symptoms and advice:

I’ve used it when I thought I had COVID symptoms, right at the beginning [of the pandemic] to see if I needed to get tested... I’ve never heard anyone having used the online service and been told to come in no, I’ve never heard anyone come in to A&E say that they’ve been told to come in by the online service.

ED16, ED doctor

I think I’ve actually become aware of it [NHS 111 online] since COVID because the testing of COVID has been pushed in to 111 rather than GPs doing it. That’s when I became a bit more aware of the different options of 111... I can’t say I have ever heard reception signpost 111, I have heard them signpost for sick notes, but they tend to say.gov.uk [website] rather than 111. Thinking about it I’ve never heard reception [talk about NHS 111 online], and we do hear reception because when we do the triage we sit in the same room together.

PC06, GP

The introduction of the COVID-19 111 online service (initially accessed via a separate ‘mini service’ with a distinct URL, but integrated to the core online service from June 2020) increased awareness of the service for staff,62 and encouraged use by patients and the public:

I think in the past we would have known very much that its phone 111 but I think during COVID it has very much been trying to direct people to the website and that’s partly because they had access especially when testing came in, they could do all that through the 111 offer. So I think I’ve gone from probably not knowing very much about it at all to we now direct a lot of patients in the first off to do that.

PC33, GP

One of our interviewees from a charity representing people with mental illness also noted that the online service was more visible during the pandemic:

... when I was hearing about 111 online, it was about COVID, so if people were concerned about COVID they were potentially using 111 online before going to the GP or something.

CH04, national policy advisor charity

There was also a suggestion from two interviewees in primary care that the pandemic had increased awareness of the NHS 111 online service and that the service was beginning to be seen as helpful or beneficial:

I was aware 111 had a bit of information before coronavirus but I’ve been much more aware of it as it became useful to us in coronavirus such as if you wanted to get sick notes you had to go online to 111 to go that route and save some you know, unnecessary troubles for us.

PC32, GP

Using [NHS 111 online] more with the COVID has given us an insight into using and how effective it can be.

PC22, reception manager

In addition to the 111 COVID-19 algorithm ‘mini service’, which was absorbed into the core NHS 111 online service as part of the pandemic response, the NHS Digital 111 online team introduced various services to support ‘more efficient’ use of NHS resources and direct patients to appropriate services during the pandemic. This included the introduction in November 2020, with a final release in December that year, of the Emergency Department Digital Integrator (EDDI) web service, which enabled NHS telephone call handlers and users of NHS 111 online to book arrival time slots in to emergency departments. This service augmented existing NHS 111 provision of direct booking into primary and urgent care services. Most urgent and emergency care-based interviewees referred to this service as EDDI, but interviewees in two sites referred to this as the ADASTRA patient management system, which was extended to support this streaming and booking function. Alongside the EDDI service, a ‘front-door’ or kiosk-based ED streaming and redirection service was introduced at five NHS sites from December 2020, which allowed walk-in patients to complete an online triage process on a computer tablet provided before entering the ED. One of our interviewees worked at a site that had adopted this self-service triage early on and had contributed to the development of the new streaming service:

No-one was allowed in unless they had done the online tool, had spoken to a GP through the CAS [clinical assessment services] or given an emergency ambulance disposition and then they would be booked in to the emergency department. But what we found was it worked to some extent but the wording wasn’t right for us because it would say things like ‘phone an ambulance’ standing in front of the ED that’s a silly thing to say...we worked with NHS Digital and said would you make this a bespoke tool that we could use in an emergency department setting.

ED10, urgent care quality lead

This ED streaming service is currently badged as an ‘Urgent care self-service tool’ and was commissioned as part of the national ‘111 First’ initiative/programme. It was supported by a national publicity campaign encouraging people to use NHS 111 telephone or online services before travelling to the ED as part of the strategy to ‘protect the NHS’ during the pandemic. 63 Perhaps unsurprisingly this 111 First activity had raised the profile of NHS 111 particularly for interviewees based in Urgent and Emergency care services. In interviews conducted from December 2020 onwards, ED staff, both clinical and administrative, described the new facility allowing NHS 111 to book appointment arrival time slots in ED. Our search of websites and documents about the service identified a number of publicly available signposting materials, produced by various NHS Trusts, advertising this new functionality and pointing to both telephone and online 111 services (e.g. Wirral Community Health and Care;64 Isle of Wight NHS Trust;65 North East Ambulance Service,66) which we suspect had helped raise the profile of this new service.

Interviewees from the ED and urgent care services were able to describe the pre-booking facility but were sometimes unsure if this was available only via NHS 111 telephone services. They used different names for the service, including mentioning the long defunct NHS Direct service:

So recently in the department it was introduced the referrals from 111 and the ED doctor explained to us that some of them could be done through 111 online and that’s the only thing that I know.

ED17, ED nurse

... we do see patients who are sent in by 111 either as an advice for treatment in ED or we also have allotted slots, like booking, every half an hour in the afternoon which is a new thing they are starting.

ED24, ED doctor (trainee)

... initially they would say they had consulted 111 online. Then my own confusion was, have they spoken to them on the phone or had they gone to NHS Direct, you know the website you can use for looking up their symptoms and stuff. Then there was some confusion around, we had a system, I think they called it the EDDI system where 111 were able to book you an appointment so to speak in A&E... That’s my experience with it being online.

ED18, ED nurse

Although a few interviewees felt that 111 First streaming was helpful, several of the interviewees from emergency departments reported that this external booking process and allocation of an appointment time was a source of some frustration for patients and staff alike. ED18, who described 111 First in the quotation above, continued:

111 were able to book you an appointment so to speak in A&E. So if you turned up between these times you would get seen and I think that caused a bit of confusion with the patients as well because I think they thought they were coming to see a doctor at 10 o’clock for example and that they would be seen straight away and that wasn’t the case. That’s my experience with it being online.

ED18, ED nurse

Emergency department reception staff reported that this created problems for them because patients arrived with an expectation that they would be seen immediately, and this was often not possible because of the health needs of other arrivals at the ED. Clinical staff also identified that lack of trust lay at the heart of problems with this streaming, and they challenged the ability of a computer algorithm to triage these patients:

... how many times do we need these systems in how many different formats, and the 111 service is actually quite crude to certainly a GP at the end of the phone, any health-care professional, it’s that clinical acumen, listening to that patient and actually hearing what is going on and no fancy computer system will be able to replicate that. Is it helping or is it hindering is the question or is it just putting another step in an already complicated pathway when all you want to do is have a quick chat with your GP about your blood pressure for example. Nobody trusts a computer like a human.

ED09, GP in ED

Our interviews suggested that in the absence of the pandemic and the NHS 111 First initiative many staff and stakeholders in the primary, urgent and emergency care system would have little or low awareness of NHS 111 online as a distinct service. The use of the online platform for COVID-19 symptom assessment and advice had increased the awareness of this service in primary care. Similarly, the NHS 111 First initiative had raised the profile of the online triage and streaming capability of NHS 111, even if there continued to be some confusion about the name of the service and its relationship to the NHS 111 telephone services. Although a few interviewees reported benefits from using the NHS 111 online system, including positive experiences of using it themselves, the introduction of the 111 First streaming was problematic for ED staff.

Our sampling for these interviews sought to reflect a range of services, staff and stakeholders implicated in or associated with a wide network of urgent and emergency care services. We interviewed NHS staff and stakeholders in charity and community organisations who worked in roles in which they could be expected to see patients or service users who had used and/or been referred by NHS 111 online, and who might be in a position to signpost patients and the public to the service. Notwithstanding the fact that NHS 111 online was launched in 2017 and is a comparatively recent addition to the NHS 111 offer, we were surprised at the low levels of awareness of the service, particularly given that it had been used more than 1 million times by May 2019. 67 During the pandemic, usage figures increased to 30 million visits (February to August 2020); however, even after this phase of the pandemic, the service continued to have limited visibility with some of our interviewees.

To understand this lack of visibility, we explored the interview data, looking at the other digital technologies mentioned by our interviewees to see if this might help explain why awareness of NHS 111 online was low.

The crowded digital technology landscape in primary, urgent and emergency care

Our interviewees described a number of different digital technologies that they used, encountered or were signposted to in their day-to-day work (Table 11). As we have already indicated, awareness of the NHS 111 telephone service appeared to be higher than that of NHS 111 online as a distinct service. In addition, some interviewees were aware of other services that are part of the expanding NHS 111 offer, including 111 First and, to a lesser extent, the online service for COVID-19. As well as NHS-‘branded’ technologies, our interviewees mentioned a variety of generic and illness-specific sources of online health, symptom advice and treatment advice; different patient- and public-facing triage and assessment tools; and various database and patient record software associated with primary, urgent and emergency care.

Given that the study was focused on NHS 111 online, we had expected interviewees to talk about this service in relation to the telephone 111 service that predated it. Roughly three times more people in primary, urgent and emergency care sites mentioned the telephone 111 service than the online service, but none of the interviewees from dental or charity organisations mentioned NHS 111 online by name. Interviewees mentioned NHS 111 telephone services, including bespoke or specialist services. As already mentioned, several ED-based interviewees were aware of the NHS 111 First initiative, and they sometimes referred to this using the shorthand NHS 111 or by referring to EDDI, which was the record system used in the ED to manage these 111 referrals. Some interviewees were aware of other NHS COVID-19-specific services: one GP working in an ED setting named the additional phone number (119) used for COVID-19 advice (ED07) and others mentioned the COVID-19 online service. Two interviewees in the same emergency care site were aware of particular support provided locally by the NHS 111 telephone service to support people experiencing mental illness, referred to as ‘111 option 2’. Just one primary care-based interviewee described NHS-branded apps that they used personally. These included both the NHS app (which gives patients secure access to their GP record, allows them to make GP appointments, order repeat prescriptions and to access the NHS 111 service online for urgent medical queries) and the NHS COVID-19 app (used for contact tracing, venue check-in, symptom and test reporting, and community alerts). The following quotation revealed that this respondent was less aware of the embedded NHS 111 online functionality in the NHS app and understood NHS 111 online as a separate website:

I have the other NHS app on my phone. I’ll just get my phone ‘cause I can’t remember off the top of my head, I use quite a lot of digital technology. It is, [consults phone screen] NHS app, it says how are you feeling today? And then you get health advice. I’m sure it’s that one I’ve been on for symptom checking. It’s not that I’ve been ill it’s just that I’m a bit of a nerd so that anything patients ask me through the day I’ll go home and try and find out... the easiest way for them to go about things... 111 is actually a website rather than an app, it’s easier for me to have the app on my phone because I’ve got the NHS COVID-19 checker as well as an app and the C19, the COVID symptom app but you see 111 isn’t actually an app is it, it’s a website rather than an app.

PC27, nurse

Three interviewees, one each from urgent and emergency care, dental and charity-based organisations, mentioned the NHS Find a Dentist website (www.nhs.uk/service-search/find-a-dentist, accessed 16 March 2023). However, for DO3, the exact name of this site, nested in the nhs.uk website, was confused with an earlier, now defunct, website (NHS Choices):

We just tell the patient, if they have come to out of hours we tell them to go and see their own dentist, [they say] ‘Oh I haven’t got a dentist’ [we respond] ‘so go on NHS Choices, put your postcode in and find a dentist that is close by’.

D03, dentist

Alternative triage, assessment and appointment booking systems

Alongside these NHS-branded platforms and services, interviewees mentioned a number of alternative triage and assessment systems offered by non-NHS suppliers, in particular e-consultation systems used in primary care. These systems are typically integrated with practice IT systems and ask patients to complete an asynchronous web-based form that is reviewed by practice staff to ensure that those in need of immediate medical attention are identified. They also offer symptom checkers and signposting to self-care and pharmacy advice or local self-referral options, and often link to NHS 111 via hyperlinks. These eConsultation systems also allow online completion of administrative tasks by patients, such as ordering repeat prescriptions, accessing their medical record or updating their details. There are a number of different e-consultation systems available, but those mentioned by our interviewees included eConsult (supplied by eConsult Health Limited, previously called WebGP) and askmyGP supplied by (GP Access Ltd), but some interviewees simply referred generically to ‘eConsultation services’:

... we have the eConsultation service that has been rolled out so if our reception are able to triage them to use the e-consultation service we frame it as a quicker way to get a reply without coming in.

PC07, GP

Two interviewees mentioned the Doctor Link eConsultation system (supplied by Dr Link Ltd), which offers a symptom checker and access to online or video consultations:

... we have a (ask)myGP app so patients can book through the app that we’ve got... I think we are just launching something called Doctor link... it’s kind of like a symptom checker... so patients will input their symptoms and then be directed as best as they can to the appropriate service, be it pharmacy or us or possibly the same day service, or A&E.

PC22, Reception manager

Previous research found that the use of eConsultation systems was low in 2016 but has grown in intervening years. 68,69 It was clear in our interviews that this mode of accessing primary care was boosted by pandemic risk mitigation measures discouraging patients from seeking face-to-face appointments with NHS services. Although a few practices opted to offer telephone triage rather than eConsultation (e.g./PC18a rural practice, low deprivation, higher than average elderly population), other practices embraced the opportunity to introduce e-consultation systems:

COVID really offered us a massive opportunity to clear the diaries... we said, let’s really make a change here... now everything comes in through e-consult if we can.

PC13, GP

We are guiding the patients to a specific e-mail address where they can e-mail us directly you know they can look at going on the [ask]MyGP app so we have had quite a good uptake of that. Again there was resistance at first in the old-school patients who wanted to ring up and ask for prescriptions but we have guided them to contact their pharmacy, they can do it direct for you... they are doing it a lot more now and I think a lot of it is through necessity because they can’t come in.

PC28, care navigator

Interviewees noted occasional negative feedback from patients about these systems, for example about telephoning the practice and being told to fill in a form before being allowed to speak to the GP (PC26); however, more often interviewees contrasted the positive features of ‘their’ eConsultation systems with NHS 111, which they felt did not capture sufficient or accessible information ahead of booking in the patient:

So most of our patients are encouraged to fill in an e-consult, its fairly comprehensive and also has quite a lot of safety features built in to it. That then comes to us in a PDF format and we then, so we have a team that care navigates that so they look at the problem and decide this is a routine appointment, this is a routine telephone appointment, this might be a next day or this person needs seen today, this is a phone appointment this is a face to face... So we have a team of people who navigate that patient into the right appointment and 111 bypasses that. When we do see the patient, if it’s a phone call, I have the e-consult in front of me, it’s quite detailed, there’s quite a lot on it and that’s there in front of me. So that doesn’t really happen with 111, basically they go to 111 and the patient just gets booked in, you know usually on an on the day appointment with a 111 report which isn’t as easy to read or as comprehensive as the e-consults.

PC14, GP

Digital communication and electronic records

Alongside these digital systems used for triage and assessment and booking appointments, interviewees mentioned a number of digital communication systems, which were used for sending information and dealing with electronic prescriptions, and various electronic patient record systems and databases (notably EMIS and SystmOne). In primary care sites, some described the use of the practice’s own website as a route by which patients could directly book appointments. COVID-19 had increased the use of some remote communication modalities and, in line with other research, we found that our interviewees mentioned ubiquitous digital communication technologies, including e-mail and text messaging (SMS), as important tools for contacting patients and professionals. Some primary care-based interviewees spoke of particular digital products, such as accuRx, which can send SMS messages and digital documents over a secure platform. As well as these digital systems, a range of other online resources that supported patients and health-care professionals, including telemedicine/teleHealth sites that support remote care, such as the mydiabetes platform, which allowed self-monitoring and reporting of blood glucose levels, to support diabetes management, and similar systems for monitoring blood pressure.

Other resources and social media sites

NHS Find a Dentist on the nhs.uk website was mentioned by three interviewees as an online resource that they signposted for patients. As well as the nhs.uk website, interviewees signposted the gov.uk website for self-guided support, particularly during the pandemic, for guidance and updates about COVID-19 (www.gov.uk/coronavirus) and for advice about obtaining ‘fit notes’ (referred to by interviewees as ‘sick notes’) (www.gov.uk/taking-sick-leave). Interviewees based in primary care also spoke about the social media platform Facebook, which many practices used to publicise details about the general practice: these Facebook pages often featured prominent advice about using NHS 111 services (telephone and online).

Positioning NHS 111 online in urgent and emergency pathways

The interviews indicated that NHS 111 online lacked visibility in the crowded digital technology landscape of primary, urgent and emergency care, where it was overshadowed by the NHS 111 telephone service, which had greater recognition and prominence, and it competed with a number of other digital technologies. In primary care, NHS 111 online was an ‘alternative’, perhaps a less-favoured, assessment and triage system compared with the in-house eConsultation systems staff there used on a daily basis. In emergency care, the 111 First initiative and the introduction of the EDDI system blurred the visibility of NHS 111 online. In primary and emergency care, there were some issues with data linkage interoperability between NHS 111 services and other digital systems, with the result that NHS 111 online was sometimes viewed as problematic. In addition, across the health system there were a number of different digital technologies and systems in common parlance and everyday use, some of which appeared to duplicate or overlap with the functions of NHS 111 online.

Having examined the perceptions of NHS 111 online and the digital landscape, we now look at where NHS 111 online sits in pathways to urgent and emergency care. To do this we draw on the interview accounts, and on policy documents and public-facing information about the urgent and emergency care system.

Conceptualising and representing pathways to care

We enrolled ideas about ‘work-as-imagined’ and ‘work-as-done’70,71,72 to explore the ways that pathways to care are presented in policy and practice documents and in public education messaging, and to compare models and diagrams depicting these pathways with the interview accounts of help-seeking. Drawing on insights from complexity science, Braithwaite73 encourages us to recognise that health systems are non-linear; with colleagues he has delineated health-care systems as having a ‘blunt’ and a ‘sharp’ end. 71,72 At the sharp end is the front-line workforce (health-care professionals and those involved in directly delivering care and services) and at the blunt end are those who aim to shape and influence health-care and services, including national and local policy-makers and service managers. It is this latter group, at the blunt end of the health-care system, who, it is argued, create and use imagined, simplified conceptualisations of health-care work and practice. The diagrams used to represent patient pathways to urgent and emergency care, and the policy documents that describe these pathways in simplified and linear terms, are located at this blunt end of the health-care system and are, we contend, work-as-imagined. At the sharp end of care, as we will show, the work of making sense of and navigating ‘real’ pathways to care is carried out by health-care staff and patients, and this can look very different.

NHS 111 telephone services were implemented across England in 2013, replacing the NHS Direct nurse-led telephone service that had been introduced in 1997. Both NHS Direct and its successor NHS 111 sought to provide health advice to callers and encourage self-care where possible, but NHS 111 offered a clinical triage system (using the NHS Pathways computerised decision support system) combined with a directory of services that also allowed call handlers to direct patients to appropriate services. In the same year that NHS 111 telephone services were introduced, phase 1 of the urgent and emergency care review was one of the first policy documents to describe the role of telephone services to help patients access care and services. 74 The update to the review, published in 2014,75 consolidated the vision for urgent and emergency care and presented a pictorial representation of the urgent and emergency care system in what has become known as the Keogh diagram or triangle after the principal author of the review, Sir Bruce Keogh. This diagram (see Figure 1) depicts ‘in visual form, the shape and structure of the future urgent and emergency care system’75(p8) and reflects a high level NHS policy vision for transforming care for people with urgent but non-life threatening health-care needs. The diagram shows that patients can:

1. use self-care, including accessing the NHS online (web-based) advice via the NHS Choices site (since been transferred to the NHS.UK website)

2. follow an emergency ‘999’ (red) pathway to specialist emergency care (depicted in red at the apex of the triangle)

3. use the new ‘111’ telephone service (shown in blue) as a gateway to a range of non-hospital services ‘close to home’.

FIGURE 1.

The Keogh triangle depicting the urgent and emergency care system. Figure reproduced with permission from NHS England. 74

The Urgent and Emergency Care Review75 update in 2014 report was clear that the transformation of services based on this vision would take 3–5 years, but the diagram conveyed important messages about the integration of the urgent and emergency care system, and that care outside hospital settings, in, or close to people’s homes, was the ‘right place’ for urgent non-life threatening health needs. The use of arrows and colour to separate emergency and urgent care reinforced ideas about appropriate pathways or routes in the system. This Keogh diagram has been widely reproduced on websites,76,77 in local Council papers,78,79 and in academic research, including work by members of our research team. 80

Other images created by service commissioners and providers also depict the urgent and emergency care system (e.g. Figures 2 and 3). These appear as posters or web-based infographics as part of health education campaigns and advice to signpost patients and the public to services. These images retain some elements of the colour signifiers used in the Keogh triangle (e.g. red for emergency care) but not the inverted triangle shape, and they include different types of services (and sometimes different names for the same service). Variations in the names and types of services depicted in these different diagrams reinforces the sense that this landscape of care provision is constantly changing, but NHS 111 often features prominently in these images (as in the examples below).

FIGURE 2.

East Riding of Yorkshire CCG Urgent and Emergency Care poster. This image was taken from the www.eastridingofyorkshireccg.nhs.uk/. East Riding of Yorkshire CCG have since updated the image used in their materials but similar versions of this diagram appeared elsewhere on NHS and other websites within the timeline of our study. 81

FIGURE 3.

Urgent and emergency care poster. This image was taken from a site now registered to www.atmedics.com. This site does not have a valid security certificate so can no longer be safely viewed.

These visual representations of the urgent and emergency care system depict separation between different healthcare services, and some (e.g. Figure 3) indicate clearly delineated health problems or conditions that can be dealt with by these differentiated services. The use of graduated colours accentuates the boundaries between services. Some of these images suggest that patients seeking help experience single health problems, and there is an assumption that they experience little difficulty interpreting symptoms. The complex health needs, social circumstances and the myriad factors known to influence help seeking cannot be conveyed in these simple schema. 82

NHS 111 online did not feature in the original Keogh triangle which predated the launch of this service. Instead the NHS Choices website featured in the self-care ‘bubble’ at the top left corner of the diagram. NHS Choices offered an A-Z of common symptoms, a symptom checker and healthcare advice (the A-Z feature is now available on the NHS.uk website). As early as 2015 the possibility of a 111 digital platform being integrated into the NHS Choices site was mentioned by some sources,76 but NHS 111 online was launched across England at the end of 2018 with a separate website/url (www.111.nhs.uk) and the NHS Choices site was discontinued. Following the launch of NHS 111 online national and local publicity campaigns for the service have promoted NHS 111 online as a new service (e.g. Figure 4) or have advertised both the telephone and online services together under the ‘NHS 111’ banner (Figure 5). These campaigns have run alongside other campaigns and advertising directing people to call 111 rather than 999 ‘when you need help fast but it is not an emergency’, including national advertising materials used in the pandemic.

FIGURE 4.

NHS 111 online ‘is available’ poster. 83 This image is available to view on a site registered to Teigh Estuary Medical Group.

FIGURE 5.

111 call or online image. 84 This image is available to view on a site registered to BrisDoc Healthcare Services.

These images suggest that NHS 111 telephone and NHS 111 online are similar and possibly interchangeable. This is conveyed also by the NHS.uk website 111 landing page,85 which states that ‘NHS 111 is available 24 hours a day, 7 days a week’ and advises people to:

1. go to 111.nhs.uk (for people aged 5 and over only)

2. call 111.

A similar message is provided on the NHS England website86 although the text on this site (Box 1) accentuates the role of clinical and call advisors, emphasising the telephone aspect of the service:

By contrast the NHS Digital website87 describes the distinctiveness of the remote triage offered by the online service and makes it clear that users (the public), rather than clinical staff or call handlers, navigate the triage system:

NHS 111 online is a triage system available for members of the public in England and uses clinical content from NHS Pathways to enable users to go through a digital remote triage process to get to the most appropriate level of care for their presenting healthcare needs.

Together these advertising campaigns, descriptions and visual depictions variously position NHS 111 as a central component in the urgent and emergency care system, although as we have shown the telephone service is more visible than NHS 111 online in most of these.

When we began this study we drafted a diagram as the basis for our exploration of where NHS 111 online fitted in pathways to care (Figure 6). This depicted NHS 111 services and the different dispositions and onward referrals to NHS services or self-care, and, albeit in simplified form, showed some of the possible interconnections between different services and possible pathways to care, for example NHS 111 advice might be to contact primary care, but the patient might decide to visit A&E (emergency care). We used this diagram and the models and descriptions of NHS 111 already described in the chapter as a starting point to understand what our interviewees had to say about NHS 111 online and pathways to care. Analysis of the interviews suggested that this draft diagram was not a helpful model for understanding pathways to care involving NHS 111. As already shown in this chapter there were a number of other digital technologies and online services implicated in patient’s journeys to care, and the range of services involved in delivering care had increased with new or re-branded services joining those listed in our draft diagram. While the presence of multiple access points may be welcome, a clearer understanding of pathways to care was needed. To this end we explored interviewee accounts of paths or routes to particular kinds of care, and how NHS 111 (telephone or online) was implicated in these pathways.

FIGURE 6.

Draft pathways to care from NHS 111 online study protocol.

Interview accounts of pathways to urgent and emergency care

Using the interviews we identified multiple triage and potential redirection points that occur for patients navigating their journeys to see/consult a clinician in urgent or emergency care. NHS 111 online generates an advisory disposition that may result in a walk-in attendance at the ED or an urgent care service, but it might also offer a clinical call back, that could also result in these recommendations. During the pandemic this facility was augmented by the 111 First triage and assessment process, a variant of NHS 111 online that could lead to redirection or onward referral:

rather than being told go to the ED, they are told someone will come back to them we then phone them back do a little bit of chat, history etc and where appropriate we then do a video consult with them, we stay on the phone but we use their phone camera, to have a look at them and then we make a decision around do you need ED, [or] do you need self-care, pharmacy, physio, MIU.

ED06, ED senior manager

Once at the ED, a patient could be triaged or seen by a streaming clinician who referred individuals to the most appropriate service. This might include direction to a GP service located in or adjacent to the ED, or to the ‘minors’ area of ED:

the streaming nurse at the front desk along with the receptionist so they can get streamed to minors, to majors, to the GP. So for example an ankle injury. If it looks deformed obviously it is time critical so it needs to go to majors or resus. Obviously if it is injured but doesn’t look deformed, it can come to minors.

ED12, ED nurse

Streaming might also result in the ED team contacting the patients’ own GP to make an appointment, diverting attendees away from ED and creating a loop back pathway to general practice:

[people] say they called the GP and there is no appointment and we will dial the GP number and we will get the appointment for them this afternoon so go to your GP.

ED13, ED doctor

Another redirection pathway occurred for patients attending MIUs or UCCs who were re-directed to ED when X-ray facilities were needed. In one region, patients experiencing mental illness who arrived at ED might be asked to call the NHS 111 telephone service, in order to access ‘option 2’ a telephone psychiatric advice service:

[option 2 staff] generally see if they can safety net people or come up with a plan for them to avoid them attending A&E. But if [the patients] do attend A&E, we can also get them to call 111 option 2 on their own phone or we give them a phone and somewhere quiet where they can make that call because we do have an in hospital psychiatric liaison service but generally they would ask us to make sure the person has called 111 option 2 prior to them seeing because if they are able to make a safe plan with the person and they don’t have any medical needs like having taken an overdose, or they haven’t hurt themselves, then again A&E is not always the most appropriate place for them to be so we do add that at the front door as well.

ED01, ED nurse

We used these data to redraw the section of our pathway diagram involving urgent and emergency care (Figure 7). The arrows reflect formal referral pathways between services, the wavy lines reflect self-presentation of individuals to a service based on advice, without a formal referral system.

FIGURE 7.

Real life patient pathway experience urgent and emergency care.

Pathways to emergency dental care

While there were no accounts from the dentists specifically implicating NHS 111 online in patient pathways to dental services, a few interviewees based in dental services described direct referral routes from NHS 111 telephone services to emergency dental care. In one region this path, when the call was out of hours, offered a direct route from NHS 111 to a booked dental appointment, however interviewees in EDs noted that patients with emergency dental health needs were often sent by NHS 111 telephone services to the emergency department, only to be advised to find an emergency dentist:

I have a particular issue with dental problems and 111 because I see a lot, I seem to see at least 1 or 2 every single shift and that is through 111 saying ‘oh 111 told me to come because they can’t get me a dentist appointment.

ED09, GP in ED

What we have started saying to them is that they have to go back to 111 and ask for dental advice because we have nothing here.

ED08, ED nurse

Interview accounts of pathways to primary care

Drawing on the pathway descriptions provided by our interviewees in primary care, we identified extra steps, hurdles and consultation circuits navigated by patients in their journeys to consultation in primary care. We have split the presentation of these data into general practice and pharmacy care.

NHS 111 and pathways to the GP

As we have already shown there were a number of alternative online symptom checkers and eConsultation access to general practice, sometimes these linked to NHS 111, and sometimes they were described as alternatives to the NHS 111 service. Use of these systems increased dramatically in response to the pandemic during 2020–2021 and many practices expected to continue to use these systems post pandemic. Some eConsultation systems interfaced with NHS 111, for example advising the user to contact the NHS 111 telephone service, although as this interviewee explained patients bypass this advice:

[in] e-consult, if you score over 6 for pain, it automatically calls you to ring 111 and so we have loads of patients, a lot of them use their acumen and say actually I just rang the practice cos I realised I didn’t really need to ring 111.

PC13, GP

The majority of practices offered the option of online booking of a selection of appointments via the practice website and the online eConsultation systems also directed users to 999 or ‘to phone the practice’ PC26 GP. On occasions when there was no capacity in the practice patients could find themselves completing the online assessment or phoning the practice only to be told to telephone NHS 111.

if we were absolutely full for the day, and the on call doctor was full and they needed help that day again, give 111 a call.

PC16, Receptionist

The most direct path between NHS 111 and general practice was via the NHS 111 telephone service, where call handlers could book patients directly into a GP appointment slots during practice opening hours. Patients who were not booked into an appointment might still be advised by NHS 111 telephone or online services to make an appointment with their GP. This might result in them completing an online eConsultation assessment, sometimes duplicating questions already asked by NHS 111.

Patients directed by NHS 111 to contact their GP were sometimes triaged by the reception team using in-house care navigation manuals and this introduced further opportunities to redirect some patients to alternative healthcare providers, such as pharmacies:

we speak to them first on the phone and put them in touch with the most appropriate clinician so it might be that they get added to a triage where either an advanced practitioner or a GP will call back depending on what the problem is...we will signpost, if it’s something like they have a painful ear, we’d potentially say, depending on how long they’ve had it, see if there is anything you can buy over the counter first, if not then come back.

PC24, Care navigator

One practice used a doctor first model of triage where patients were able to directly speak to a triaging GP, but the interviewee explained that they had to ask reception staff to call patients referred by NHS 111 to tell them to call the dedicated triage line.

NHS 111 and pathways to Pharmacy services

The increasing use of pharmacy services for primary and urgent care services added another pathway where NHS 111 could be involved. While the majority of patients ‘still walk into the pharmacy for advice rather than go through the 111 service’ PC02 some are told to seek pharmacist advice by NHS 111 telephone or online services. In these cases the Pharmacist may see the patient and offer advice, but if a prescribed medicine is indicated, or if the patient has another health condition or are taking other prescription medicines that might interact, this prescription must be reviewed by a GP, necessitating redirection back to the GP or urgent care. This was not always welcomed by GPs or clinicians in urgent care:

people think trivial illness is easy to define and everything presents in a very basic, simple way but it’s not like that in real life is it so they go to the chemist with a simple, with a rash but if they are 80 and have got other things going on even a simple consult is never that simple because the treatments might be limited by the other things.

PC32, GP

the pharmacists are not confident enough to deal with those minor ailments so they take a look at something and say oh I think you’d better go and see your GP with that.

ED27, MIU nurse

Referrals from NHS 111 to pharmacy care were not straightforward, and resulted in complex pathways for the patient, as one pharmacist interviewee explained:

Someone used 111 online to request an urgent supply of controlled medication so once we have seen it we say we can’t do anything, so we call 111 handler, and 111 handler can’t access the online system...so what they did was, they re-raised another 111 referral for me to refer it back to them and then refer it to an out of hours doctor.

PC05, Pharmacist

Consolidating these data about primary care services and NHS 111, we were able to redraft this part of the care pathway (Figure 8). The arrows reflect formal referral pathways between services, the wavy lines reflect self-presentation of individuals to a service based on advice, without a formal referral system.

FIGURE 8.

Access to primary care patient pathway.

Work associated with NHS 111 in English Primary Care settings

Thematic analysis of the 33 interviews with staff and stakeholders based in primary care identified four types of work, or potential work generated by interactions with NHS 111 services. These themes began with efforts of primary care staff, notably administrative staff, in relation to reports and records generated by NHS 111 (deciphering work) and the work by clinical staff to clarify patient details and repeat clinical assessments for patients referred from NHS 111 (duplication work). Alongside these two themes, we examined claims that NHS 111 was used as a means to deflect or redirect work to or from primary care and finally we noted potential opportunities for work by primary care staff to signpost or direct users/patients to NHS 111.

Deciphering work

The theme ‘deciphering work’ includes examples of work that staff in primary care described doing to make sense of reports and records generated by NHS 111. Initial coding for these data included emic (participant view/term) code ‘paperwork’ as when asked about NHS 111 online, several interviewees immediately referred to ‘paperwork’ that they received from NHS 111 and the difficulties they experienced dealing with this. On probing it became clear that ‘paper’ more typically referred to digital (eHealth) records received from NHS 111 services, but the ‘work’ included reading, making sense of and interpreting these records, hence our subtheme of deciphering. Difficulty making sense of these reports meant that interviewees were unsure if referrals originated from a telephone or online 111 interaction:

on our system we can’t tell if it’s from a telephone handler or from an online system. It just comes on as a referral.

PC05, Pharmacist

whether [the patient] had actually called or gone online is sometimes hard to differentiate, honestly, I’d say the majority had probably called 111.

PC04, GP

One of our jobs everyday as duty doctor is that we get documents coming through... from 111, and they do come quite timely, it’s quite well integrated as it comes into our EMIS system. I’m assuming that must come from the online service as well?

PC06, GP

Clinicians reported that NHS 111 reports, which often ran to several pages, were laid out in a format that did not always seem logical to the recipient. They needed to be interrogated and deciphered as it was sometimes difficult to understand or locate important details, such as the initial presenting problem:

I think what’s difficult with the 111 report is that you have all of the criteria they have to go through, are they breathing, are they this, are they that, are they hot and sweaty, so you have those to try and overlook but you get to the nuts and bolts of it quite quickly, sometimes that’s more difficult than speaking to the patient.

PC10, Nurse

we get [a] document that you have to search for ages, the 111s are the worst because you have to search for ages to find out and actually derive what the [patient] wanted.

PC19, Nurse

we get all of the automated questions listed and its really difficult to pick out what they even called about. If they are triaged to speak to a clinician at 111 then that bit of the report is clearer because it’s at the bottom, but the rest of the report is difficult because it’s all mixed in and it’s impossible in the time that we have to look through it.

PC07, GP

These clinicians explained that this work was time-consuming and could be frustrating. Practice staff also described how the lack of integration of record systems meant they had to process 111 reports and manually move data into local eHealth record systems. This included scanning, or ‘cut and pasting’ between the different record systems.

There were ebbs and flows in the workflow associated with reports from NHS 111, and administrative staff noted that the majority of this ‘paperwork’ arrived on Monday mornings, as might be expected after the weekend closures of general practice. Some mentioned that they had adjusted staffing numbers and rotas to accommodate this ‘extra work’ [PC28 Care navigator]. The additional work required to process NHS 111 reports also fell on primary care staff not based in general practice. Pharmacists providing minor illness services reported that some patients seeking urgent care were increasingly referred to pharmacy services and they too had to process and deal with NHS 111 reports:

there is more admin work because obviously we have to log all the consultations, all the outcomes and everything just for the patient safety and for our own safety as well and for the doctor to access them and to see what’s been discussed. So it is slightly, a bit more admin work to log the whole thing.

PC05, Pharmacist

Deciphering of information from NHS 111 was closely associated with work by clinical primary care staff dealing with patients referred by NHS 111, which formed our second theme of ‘duplication’.

Duplication work

Clinical interviewees (GPs, practice based nursing staff, and other healthcare professionals working in general practice) described work they performed to clarify the contents of the 111 reports in their consultations with patients. Like deciphering work this was described as ‘extra’ or ‘cumbersome’ work and it appeared to stem from a lack of trust in triage performed by ‘others’:

I don’t want to rely on somebody else, on somebody else’s, I don’t know if they’ve taken down their history correctly, I don’t know if they’ve documented it correctly... I don’t know if they’ve covered all the medical things.

PC03, GP

While interviewees did not explicitly identify concerns about the NHS 111 online, it may be worth noting that if the ‘other’ in this context is the (non-clinical) patient working with a computer algorithm, lack of trust in the assessment may be even more of a driver to duplicate the process.

One impact of the duplication of questions that the patient had already answered in the NHS 111 or eConsultation assessment processes was patient frustration, and this could create extra work for staff to manage this:

the receptionist puts them through for a [telephone consultation] and [patients get] a bit irate saying “Well they told me to fill in this form, so I filled in the form and then it told me to try and phone the doctors, which is what I tried to do in the first place!” “Okay, sorry!” So then we try to say that is quite useful because it tells us that you genuinely need to be spoken to, we try and explain it. But that’s what we get.

PC26, GP

NHS 111 deflecting work to or from primary care

Interviewees based in primary care did not perceive significant additional demand for primary care coming from NHS 111:

we tend to get very few through,... we get either none or possibly one at most two booked in from NHS 111, so it doesn’t really impact on our day to be honest.

PC26, GP

One GP suggested that NHS 111 was ‘great at signposting people to A&E away from primary care’ [PC13 GP] but staff based in primary care did not corroborate the view that general practice staff were frequently using NHS 111 to deflect work away from their services. One member of a general practice reception team described how their practice automated telephone system (used when lines to the practice were busy) might encourage some patients to use NHS 111:

I only really refer people to 111 when we are open if it’s coming to the end of the day and... we don’t have any appointments left. We say that if you feel you need to be seen then you can ring 111 after half six, but then you’ll find that some people ring our phone lines and find out that maybe they are 15th in the queue so then they just phone 111.

PC24, Care navigator

There was a suggestion from some interviewees that the risk averse nature of NHS 111 was driving demand for healthcare in general, and specifically, for emergency care:

a lot of these technologies are risk averse and I understand why, they’ve got to be safe and can’t miss anything but I think that probably pushes more into the health service than it should.

PC07, GP

111 seems over cautious basically, so they often will put someone in for an emergency contact that day when they are really not.

PC14, GP

This claim was reiterated by interviewees from urgent and emergency care settings as we will show in the section about this setting below.

The final theme identified in the primary care interviews looked at how primary care directed patients to NHS 111 services, particularly to NHS 111 online.

Directing primary care patients to NHS 111 services

Every site in our primary care case study had implemented a local solution to care navigation for patients as part of the Digital First Primary Care strategy,88 and their websites typically signposted patients to online digital symptom checkers and self-care advice, including NHS 111 online. Practices also used care navigation manuals and Standard Operating Practice guides, often locally developed, to support the direction of patients to appropriate care and this could include direction to NHS 111 services. The latter was only possible where staff had a good understanding of the telephone and online services, and, as we showed in Chapter 3, this was not always the case. Several interviewees confessed that their own knowledge of NHS 111 online was poor, making it difficult for them to advise or direct patients to this service. However, where interviewees were aware of NHS 111 they would direct patients to these services. Typically interviewees described telling patients who might need advice out of hours to ‘ring 111’, but some specifically mentioned that they had directed patients to the NHS 111 online service. One practice based nurse with a background in health visiting described advising parents of young children to use either the telephone or the online NHS 111 service:

[the] population now that have young children, they are coming in with their phones and we text message them and they are quite computer savvy. So saying to them, go on the 111 website or phone them, [that] covers both angles...they can access it online and find out what they have to do.

PC27, Nurse

As the online service is designed for patients over five years of age, this advice would not be appropriate for younger aged children, but this is an example of how primary care staff might signpost to the service. With the increase in the use of NHS 111 online during the COVID-19 pandemic, staff in primary care could increase their awareness of the service and be more adept at directing patients to NHS 111 online:

we were promoting it quite a lot really when everybody was presenting or phoning at the time with COVID symptoms, we were directing them to use NHS 111 online as first port of call and only if they didn’t have web access were we saying for them to use the phone service.

PC26, GP

These themes of deciphering, duplication, deflecting and directing overlapped with some themes in the urgent and emergency care interview analyses, but there were some subtle differences between the primary care and urgent and emergency settings, as the next section shows.

Work associated with NHS 111 in urgent and emergency care settings

A total of 27 interviewees were included in the urgent and emergency care case cluster. These interviewees also spoke about ‘paperwork’ associated with NHS 111, and while they did not describe deciphering work they did report additional administrative work linked with NHS 111. Like their counterparts in primary care, clinicians in urgent and emergency care settings repeated parts of the assessment process for patients referred by NHS 111. Interviewees in urgent and emergency care felt that NHS 111 directed more work to their services, and raised concerns about the appropriateness of the cases referred. They also highlighted additional work associated with managing expectations of patients sent to them via the 111 First system. Our thematic analysis of the urgent and emergency care data identified an overarching theme labelled as ‘extra work’ and within this, sub themes distinguished administrative work, re-assessments, and demand associated with 111, as additional to work expectations in this care setting. We report these data using these three sub-theme headings: extra administrative work, re-assessment and extra demand.

Extra administrative work

As in the primary care settings, interviewees from urgent and emergency care described working with eHealth record and information systems that lacked interoperability. They described extra work checking that records were copied correctly between information systems, and the delays this introduced:

you are doing double the work cos you are checking cos then you have to arrive the patient on ADASTRA but once you’ve done that all the details disappear so you’ve got to wait until MIU have accessed the system before you can accept that patient, so its purposely delaying, but we can’t find any other way.

ED26, MIU receptionist

Reports from NHS 111 services could not easily be integrated into patient records held in urgent and emergency care settings: administrative and clinical staff often had to copy data to different information systems, or scan reports so that the data could be appropriately stored.

Re-assessment

Clinicians from urgent and emergency care settings reported a lack of trust in NHS 111 assessments, reinforcing the accounts from primary care settings. They too felt the need to re-assess patients referred by NHS 111:

it’s that clinical acumen, listening to that patient and actually hearing what is going on... Nobody trusts a computer.

ED09, GP in ED

[if] I see a patient who has got one of these, or one of these letters is printed off, I will always discuss with the patient why they are there and see if it matches up. I would never just trust that 111 paperwork.

ED19, ED nurse

you are never going to provide decent healthcare unless you can sit the patient in front of you,... I’ve seen patients embellish or lie about symptoms to get seen quicker because of their expectation of what the service can do...111 is trapped as to what they can do because unless you can see a patient in front of you and see how they are actually breathing or what colour they look, you have to assume the worst because you have the risk of being negligent.

ED18, ED nurse

In addition to these negative views about NHS 111 clinical assessment there was a recognition from one interviewee that NHS 111 triage whether by telephone or online, served a useful function in capturing patient demographic details, and that using ‘expensive’ clinical staff to collect these was a ‘complete waste of time’ and resources [ED06 Senior manager].

Extra demand

Interviewees spoke about a lack of capacity in primary care and the risk averse nature of the NHS 111 assessment algorithm as sources of the extra work they perceived as coming to urgent and emergency care. Sometimes this extra demand was identified as stemming from GP or patient behaviours,

The GPs almost play it up to get the patient to go to ED because they maybe don’t have the confidence to say they don’t need to...there is a lot of inability to manage things which GPs in the past would have done.

ED14, ED doctor

because [patients] can’t access where they should be going, they see A&E as their only option.

ED02, Receptionist

But interviewees also argued that NHS 111 telephone service was sometimes overly risk averse and led to ‘inappropriate’ referrals to urgent and emergency care:

111 does some fantastic work in reducing the number of people that come the ED but it is not seen, and what you see... are those that come that really are inappropriate and that is because they are sent by a [health advisor] who is really risk averse and you think why is this patient here?

ED06 Senior manager

As well as the extra volume of work perceived as coming from NHS 111, interviewees expressed concerns about the demand coming via 111 First. Of particular concern was the extra work associated with receiving patients sent to the ED with an arrival time. This new practice introduced a ‘double queue’ made up of people sent by 111 First and walk in attendees; this sometimes led to frustration for patients and staff. Patients sent by 111 First often perceived their booked arrival time as an appointment and did not understand that they might have to wait to be seen. Staff described the work entailed in having to manage these frustrations:

people come in and think it’s an appointment but it’s not. [111 First] just give them a time to come in. Today a lady came in and said “I have an appointment at half past one”. “No actually it’s not an appointment it’s a time to come in”. “No she definitely told me it was an appointment”. That’s when the problems arise.

ED20, Receptionist

Alongside the primary, urgent and emergency care cases we also explored the impact of NHS 111 on work of dental, and charity services, and the chapter now looks at themes from these interviews.

NHS 111 and dental services work

We interviewed nine dental practitioners working in England and none were aware of NHS 111 online. However they did report interactions with the 111 telephone service, offering examples of work associated with referrals by NHS 111. The referral pathways from NHS 111 for patients with dental health needs varied. In some places NHS 111 telephone services referred on to an out of hour’s provider, or to another telephone number that directed patients to a call centre where triage of the dental emergency was performed by dental nurses. Some interviewees expressed concerns about the triage offered:

In hours it’s not triaged by a dental nurse it’s just triaged by 111 and some of the stuff that’s passing would never get past the dental nurses on a weekend.

D03, Dentist

It’s hit and miss who you get, most of the time you don’t get anyone who is dentally qualified and because of that there’s a lot of people who are ringing up here who are just being told inappropriate by 111... they won’t have a swelling but they’ve got some pain, and 111 is telling them they need to be seen within the next 12 hours. That’s just completely unrealistic.

D02, Emergency Dentist

In some areas of the North of England, NHS 111 telephone services triaged and booked patients directly into emergency dental care but in areas without this service patients experiencing dental emergencies might be referred by NHS 111 to emergency care. The latter was a source of ‘extra’ (potentially inappropriate) demand. Where NHS emergency dental appointments were directly bookable via NHS 111, demand often exceeded supply. When there were no bookable emergency appointments 111 would often give patients telephone numbers of local dental practices offering emergency services so that the patient could seek treatment. This might result in re-assessment and triaging by dental practice based staff, echoing the duplication described earlier. This could make interactions with reception staff difficult:

the patients then ring up with an expectation and then they have a go at you on the phone and we try our best to help... [receptionists/triage nurses] get so much abuse on the phone. I’ve had my dental nurses who are triaging, in tears when they put the phone down because the patients have been so rude. I do think that there is something to be said for having a dentist at 111.

D02 Emergency dentist

Risk mitigation measures introduced during the COVID-19 pandemic appeared to have led to more interactions with NHS 111 telephone services and an increase in emergency dental work for practices at a time when routine dental work was not being undertaken. There were a number of additional, short term work flow initiatives in some areas to address this extra patient demand but it was unclear how this demand would be dealt with as routine work began to be re-introduced.

Charities supporting vulnerable groups

We interviewed 11 representatives of charities supporting people experiencing homelessness, mental ill-health, literacy challenges and refugee women. These charities played a role in supporting vulnerable people to use health services, as well as offering services provided by the charities themselves. As such they were potentially well placed to signpost and direct their clients and user groups to NHS 111 online but only one of these interviewees was aware of the NHS 111 online service. Like the dental interviewees these participants had little to say specifically about the work or workforce arrangements associated with NHS 111 online but they did express concern that the NHS 111 telephone service presented an additional barrier to access for disadvantaged and marginalised groups. They felt that digital first policies and increased use of online services might disadvantage some population groups, those without digital access, and those who lacked reading and language skills:

people that have never even looked at a computer let alone had an email address, the ability to remember passwords, how do we expect people who are rough sleeping or moving around a lot and that don’t have access to the technology when they need it [to use online services].

CH01, Charity national policy-maker

if they have language barriers and all they can do is wail on the phone so it would be nice to have front liners who say “what language do you speak, we get interpreter”. That would make a lot of things easier.

CH02, Charity national policy-maker

Charity based interviewees expressed concerns that the unintended consequences of a shift to online services could be delayed presentation and increased, possibly inappropriate, use of emergency services by the vulnerable groups they represented.

The chapter now turns to the interviews staff and stakeholders associated with Healthdirect.

The Healthdirect workforce

The workforce that delivered and supported the digital triage services differed substantially across the Healthdirect and After Hours GP telephone helplines, and the online Symptom Checker in terms of size, composition, distribution and management, and was different from the workforce supporting NHS 111 telephone and online services. The administration of both telephony services was contracted out to a private company that organised the recruitment and training of nurses and doctors, their management, clinical governance, and the telephone and CDSS infrastructure. Call agents worked from home and could be ‘answering calls from around Australia’ (HD04). The Healthdirect telephone services were clinician led and interviewees contrasted this nurse-led approach with the model of using a predominantly non-clinical workforce in the NHS 111 telephone service:

We employ nurses which means they’re frightfully expensive. And I think that’s one of the comparators. If you find that these services are similar then we should be thinking of using non nurse staff because the safety of those algorithms has already been tested by the NHS.

HD41, Policymaker

shouldn’t you just have non-clinical call handlers and then it goes up to a doctor, why do you have to have [nurses] in the middle who kind of half try to do both jobs...that was from a financial commercial perspective it was much more efficient.

HD19, GP

With both Healthdirect telephone lines there were challenges in recruiting and retaining a sufficient clinically qualified workforce to work on the phones:

the main limitation is getting staffing and, which isn’t an issue for a non-clinical helpline, but as soon as you need a lot of nurses or doctors... there’s workforce issues that come into play.

HD03 Healthdirect senior leadership

These workforce challenges were exacerbated in the COVID-19 pandemic, where there was a sudden and large increase in demand for virtual triage. The pandemic led to a reconsideration of Healthdirect’s strategy of procuring telephony services from a single provider as that provider had not been able to rapidly scale up the workforce as demand had increased.

The Healthdirect Symptom Checker was staffed by a much smaller workforce that included a development team to handle coding, a few clinical staff members to oversee the currency of content against the evidence base, and a content team that worked across COVID resources as well as the Healthdirect website. This workforce is similar in size and composition to the workforce supporting NHS 111 online.

Awareness of Healthdirect

As with the English interviews described in Chapter 3, interviewees associated with Healthdirect were typically more aware of the telephone Healthdirect and were less knowledgeable about the Symptom Checker, despite it being in use for seven years at the time these interviews were undertaken. These interviewees also highlighted a lack of public awareness of the services, both among consumers and health professionals:

Healthdirect is one of those resources that’s out there. And to be perfectly frank not well known...I bet if I rang or contacted 50 people around the country that I know through various networks and so on and said, ‘Have you used Healthdirect?’ I can guarantee. I would get the ‘huh, what’s that?’ response... I don’t think we actually sell it as a resource, and I don’t think we reinforce its availability as a resource.

HD05, Consumer and patient representative

The average GP had never heard of Healthdirect before COVID... I’d be like I worked for Healthdirect like ‘who’s that? Never heard of them. Don’t know what they are.’ Been around for 10 years. No idea.

HD19, GP

They suggested that there was a reluctance among shareholder governments to promote the telephone service in particular because this had potential to increase demand for Healthdirect, increasing costs borne by government funders. The online component of Healthdirect, including the Symptom Checker, was not so constrained by these funding peculiarities yet there were still challenges to public awareness:

... it is actually a really difficult service to promote because it is every single health issue that you ever heard of, promoting it to single Australian... It’s not like... The Heart Foundation where you are only talking about heart troubles to a certain proportion of Australians who might be susceptible to heart trouble... It’s very generalist, so finding topics that would be media worthy, or headline worthy, actually can be tricky.

HD10 Healthdirect operational staff

Increasing the public and professional awareness of Healthdirect’s online provision was made more difficult as there were several other competing online symptom checkers and assessment services in use. These included local after hours provision, international health websites, and private video consultation services provided by GPs and emergency doctors, many of which were funded and organised through PHNs who had a remit to improve access through afterhours care. In the same way that NHS 111 online was confused with NHS 111 telephone services, there was some problem of brand separation for the online Healthdirect Symptom Checker. This Symptom Checker was just one of a number of Healthdirect ‘digital health solutions’ that included state and federal virtual health programmes for Pregnancy, Birth and Baby; Carer Gateway; My Aged Care; and the NSW Palliative Care After Hours Helpline and video consultation platform. While some GPs had become more familiar with Healthdirect as a result of using the latter during COVID-19 they were often still unfamiliar with the online Symptom Checker services offered:

I think the I think the brand is, is a familiar name to people within practices to staff in practices but again, the scope of what it can do, is not necessarily, no.

HD34, Healthdirect senior leadership

Interviewees compared these service awareness issues in the Australian context with their perceptions of how much more straightforward it must be for the NHS 111 in the United Kingdom. They suggested that the NHS was a ‘national brand’ without parallel in Australia, where the health system was far more fragmented. They felt that their NHS counterparts had been more successful in differentiating and promoting NHS 111 services:

... one’s envious of them, in the sense of having a national brand. And just one health system, although I know it split up into the, the county-based commissioning agencies but it’s one brand, nationally, rather than have state and Commonwealth and people who don’t understand, which is which. So having a single brand. And that concept that triple zero [in Australia], their triple nine over there, is only for emergencies and if you’re absolutely really not sure it’s that that ring this alternate. And this is such a powerful positioning. So I think that that would be a wonderful thing to have.

HD08, Healthdirect senior leadership

A primary care clinician who had previously worked in the UK reiterated this point:

... when [known previously as NHS direct] was first started, everyone knew about it, and it was embedded in, in culture. And whereas I have to say, in Australia, not so much Healthdirect... I’ve been here 15 years, so you think I know a little bit about it. And I don’t, and I don’t know if that’s a reflection of, and I expect it is, the way the health system is set up in the UK, it’s much more connected... here it’s very much disconnected.

HD22, Paediatrician

Service integration

Service integration was a major issue that interviewees raised in attempting to understand Healthdirect’s virtual triage services and their role in the healthcare system. Interviewees explained that the different services were often not able to share information among themselves and this resulted in a similar duplication of assessment to that described by the English interviewees:

I think one of the issues and that we’re all conscious of is the fact that they don’t work together at all well. There’s this recognition that there’s a lack of integration between them, and we, I mean, if you’re go on the Symptom Checker it will send you to the telephone triage... It’s an interesting question about integration because... one of the problems with Healthdirect is its lack of integration with everything and everywhere.

HD41, Policymaker

The work of integrating services often fell to the users themselves: for example users of the online Symptom Checker were provided with a reference number which they were required to use if they had to subsequently call the helpline and this allowed the earlier record to be accessed by staff. While this was useful because it allowed the nurse handling the call to see the user’s Symptom Checker details, it was described as ‘a clunky system’ (HD04). Similarly the information users provided did not, in most instances, follow them or get stored in their health records. As an exception, the subset of people who used the After Hours GP service had the option of having a summary of their encounter uploaded to their My Health Record or sent to their regular GP.

Service demand

There was broad consensus on the demographics and composition of users across the different services, and that those using the online Symptom Checker were younger and presented less serious conditions. As part of regular monitoring, Healthdirect recorded the main symptoms for which people used the services and were able to make comparisons between phone and digital triage. Top conditions included abdominal pain, bites and stings, chest pains, cold and influenza, and, particularly for the Symptom Checker, sexual health and mental health issues. One interviewee suggested the self-guided and anonymous nature of the online service accounted for this:

it makes sense to us because those are the sorts of things that you might not, certainly sexual health and mental health might be things that you don’t actually want to talk to somebody about until you know whether you really should.

HD07, Healthdirect operational staff

Rationalising the tendency for health complaints to be less severe on the digital platform compared with the phone, this interviewee also suggested online users often used the Symptom Checker out of curiosity, rather than because of a genuine concern their condition was urgent:

I think that there’s probably a lot of people who use the Symptom Checker just because they’re interested, rather than because they’ve got a problem whereas when you ring the Health Line you’ve actually got something and you want to talk to somebody about what you should do. I think a lot of people potentially did use the Symptom Checker, just as a, should I be worried about this. What could it be? What should I do? How can I manage it myself? Rather than necessarily following the direction to do stuff.

HD07, Healthdirect operational staff

HD11 (Healthdirect senior leader) noted that Healthdirect was often ‘not the first step’ in a care pathway and users typically accessed a range of other online services prior to using Healthdirect. Healthdirect’s call data also confirmed that for many who used the triage service, it was to seek clarification or advice about care or treatment initiated in the health system, as HD10 explained:

[the call agents] will still tag what the caller symptom was that they were calling about... the top category was always this really ambiguous category... effectively people who had been to see their GP and been diagnosed and had, you know, got the medication... but then we’re calling Healthdirect, to check something to do with their previous visit to the GP.

HD10, Healthdirect operational staff

Like their English counterparts some interviewees felt that Healthdirect triage services (telephone and online) were risk averse and sent ‘everyone’ to ED, and that a proportion of this demand was inappropriate. Overall healthcare provider interviewees had mixed views about whether Healthdirect triage services reduced healthcare utilisation or whether that was even a worthwhile goal, while Healthdirect staff were more likely to cite research demonstrating some reduced service utilisation.

Work associated with Healthdirect in general practice and emergency care settings

Owing to the limited integration of Healthdirect triage services with the wider health system, it created less work for primary and emergency care staff; there was no administrative burden related to managing patient records equivalent to that described by the English interviewees. The re-assessment work, which always occurred according to the process and guidelines of the setting, would also be invisible to providers if their patients did not volunteer that they had used Healthdirect prior to presenting. Indeed, in Australia, the work of virtual triage was largely accomplished by the patients themselves, who were required to input their details or answer the same question several times for different platforms or service providers. Interviewees working in both general practice and EDs highlighted additional strains on patients related to use of ‘risk averse’ triage services. For example, it might lead a mother to unnecessarily attend an out of hours general practice clinic at a difficult time with a sick child, or give a patient a skewed impression of how serious their condition was and therefore how quickly it would be seen to in the ED. This created an additional layer of work for providers who were then required to manage patient expectations or educate them on how to respond in the future:

Yeah they are usually having been told to see your GP within this many hours and it’s... I’ll be polite and do the needful medically and reassure or maybe have a management plan with the mom... then they’ll go “I thought that was a bit of an overreaction I really just wanted some reassurance and for someone to say I could see my GP tomorrow or the next day... we knew it was fine, we didn’t have to drag the baby out of at 9 o’clock at night when they had just gone to sleep”.

HD16, GP

As a triage nurse hearing that story, “Healthdirect says I had to come in here urgently” my heart would sink because I’m like well if you can tell me that whole story, you’re gonna be waiting. And then it’s a process then of navigating that person through the disappointment of how we’re going to fail every single one of their expectations from that point on.

HD37, Policymaker/clinician

Primary care

Work deciphering records and reports from NHS 111 services fell largely on reception and administrative staff, but sometimes on primary care clinicians. This latter group also described duplication or extra work entailed in their re-assessment of patients referred by NHS 111 services. We found little evidence in the interviews to corroborate a view that NHS 111 was deflecting work to or away from general practice, but there was a perception that NHS 111 was driving upwards demand for emergency care. There was a suggestion that general practice websites and automated telephone answering messages may send patients to NHS 111, both the telephone and online versions of this service. And further, there was a potential role for staff in primary care to direct or signpost patients to NHS 111 services, including NHS 111 online. However this was only possible if staff understood the scope and remit of the services, and this was often lacking.

Urgent and emergency care

Interviewees from urgent and emergency care settings described ‘extra work’ that they perceived as coming from NHS 111, this included extra administrative tasks associated with moving information from NHS 111 reports between information systems and clinical re-assessment work. They also raised concerns about the increased numbers of patients attending ED and perceived that a significant proportion of this extra demand was driven by the NHS 111’s risk averse triage and assessment. There was a strong feeling that ED care was not the appropriate level of care for many of these patients.

Dental care and charity providers

Lack of awareness of NHS 111 online by dental service interviewees means that they were unable to provide examples of work associated with the service. However these interviews highlighted the potential for NHS 111 telephone and possibly online services to refer and book emergency cases directly into dental services. This work was happening in a limited number of sites as part of local arrangements with NHS 111 telephony services and appeared preferable to the practice of sending these patients to EDs. In a similar vein there were some potential pathways from NHS 111 services to care provided by charity and non-NHS organisations, for example in the area of mental health.

Healthdirect

The interviews focused on Healthdirect identified similar concerns to those voiced in English primary, urgent and emergency care staff about the lack of integration between parts of the health system. However there was less evidence that these staff and stakeholders perceived that Healthdirect’s virtual triage created additional work for their services. While there was a suggestion that Healthdirect virtual triage services inflated demand for emergency care, this was tempered by the suggestion that the users of its services may be augmenting other care/help seeking and, particularly for the Symptom Checker, that these were less serious presentations. We conclude from this that any additional work associated with assessment, re-assessment and navigating the health system is borne by patients and users, rather than the healthcare workforce in these settings.

Pulling these themes together we suggest that there are three negative impacts to note, these are:

1. increased work associated with records, largely falling to administrative staff in primary, and to an extent also in urgent and emergency care in English settings

2. clinical re-assessment performed in primary, urgent and emergency care which duplicates work already done by patients (and staff in the telephone service) within the NHS 111 and Healthdirect systems

3. perceived additional demand and potentially inappropriate referral associated with NHS 111 and with Healthdirect’s virtual triage, felt by staff in urgent and emergency care settings.

Notwithstanding these concerns the pandemic has shown that online and virtual triage were an invaluable source of advice and assessment and that patients, the public and staff are willing to do the work required to engage with such services. We have also identified some opportunities to redistribute some work within the English system, in ways that improve care, notably by exploring how emergency dental services could be better integrated with NHS 111 services. This seems to be an area where NHS 111 could also play more of a role, especially if it could link to the gov.uk Find a Dentist directory service. Finally it has been helpful to be reminded, by the comparative work with Healthdirect, that the NHS branding provides a focal point for service recognition and that from outside the UK NHS 111 has been successful in differentiating and promoting its services.

The use of digital technologies and sociodemographic characteristics

Many healthcare systems including the NHS, have increasingly employed digital technologies to deliver healthcare with an implicit expectation that individuals will have the motivation and/or the ability to find and use health information, make decisions and navigate the healthcare system. On the one hand, these technologies potentially offer greater availability of services, convenience and improved access to health professionals/services. However, they risk increasing health inequalities if they create barriers to accessing and using care for some service users (‘the digital divide’).

The relationship between health literacy and health outcomes has been well established showing that limited health literacy is associated with reduced access to services, poor self-management, increased hospitalisation and decreased life span. 92 The use of technology in healthcare has complicated this picture by further adding the need for some degree of digital competence and access to engage in healthcare systems. 42 It remains unclear how this may impact on service use.

There is evidence that the pandemic has increased internet access, with 94% of UK homes having internet access by the end of 2021, up from approximately 89% in 2019. Additionally, 92% of adults in the UK report themselves as ‘recent users’ of the internet. However, while there have been significant gains in enabling internet access and usage, there remain some groups where these rates are much lower, for example, in 2020, 6.3% of adults had never used the internet at all. Among these, 46% of those aged 75 years and over and 19% of adults with a disability had not used the internet. Such a divide in use might result from a range of individual, socioeconomic or cultural barriers, including health or eHealth literacy (see next section), competency/knowledge of digital technologies, English as a second language and disability. There may be further structural barriers, including geographic isolation, broadband capacity, and access to technical hardware. 93 Furthermore, personal attributes such as agency, motivation, personal life experiences, and values have also been identified as barriers to adopting or using digital health interventions. 94

Approximately 35% of adults use the internet for symptom appraisal, but this proportion varies between 23% and 75% depending on both sociodemographic and disease-related factors. 15 Nölke et al. suggest there is still a lack of consensus about which social and demographic characteristics are associated with online health information seeking. 95 However, lower use is associated with sex, age, education levels, ethnicity and self-reported health status. There is evidence to suggest that women are higher users of online health related information. Alvarez-Galvez et al. conducted a large study of 28 European countries,48 which aimed to describe sociodemographic factors related to the use of online health information. The authors reported higher rates of use of online health information for women, and similar findings are reflected elsewhere. For example, Tennant et al. reported that women were ‘nearly three times more likely than men’ to use the Web for health information. 96 In relation to age, studies show consistently that younger age groups are more likely to access online health information. 48,97,98,99 For example, Alvarez-Galvez et al. reported that 81.3% of people aged 25–34 years used online health information compared with 57.5% of people aged over 65 years. 48 Research suggests that in older age groups there is ambivalence towards eHealth services, with some viewing it as a ‘solution to a non-existing problem’, as well as reporting a lack of digital skills and lack of trust in such technologies. 100 Previous research has suggested that lower educational level is associated with lower use of digital technologies for health purposes. 101,97,98,48,99 Alvarez-Galvez et al. for example, reported that the likelihood of using the Internet to look for health-related information increased with the number of years in education. 48

Higher self-rated health is typically associated with higher rates of using online information or digital technologies related to health. 99,102 Moreover, people with better self-reported health status also indicated greater digital skills, so that people with ‘good’ reported health status were rated as better at general health information searching as well as having a better understanding of outcomes of care. 103 However, searching for disease specific information has been associated with poorer health status. 103 Similarly, Alvarez-Galvez et al. found that poor health status was associated with a higher use of the Internet for health purposes but only for people having chronic conditions. 48 Wong et al. also found that people reporting fair or poor self-rated health, or a chronic health condition, were significant predictors of online health information seeking. 104 While people with a chronic condition or people with poor health status are higher users of health information, a review which focused on people with chronic conditions,98 suggested that higher users were younger, had a higher level of education and were more digitally literate. The authors concluded that eHealth is least used by persons who need it most.

The association between eHealth literacy and the use of the internet for health purposes

Health literacy is defined as an individual’s capacity to obtain, communicate, understand, appraise and apply healthcare information and services to make appropriate health decisions. 105 The concept of eHealth literacy extends this to focus on an individual’s competency in engaging with digital health services, accessing health information from electronic sources and applying the knowledge to address or solve a health problem. 106 Evidence suggests that there is a substantial correlation between health literacy and eHealth literacy. 107,108

eHealth literacy is related to the use of online health information, so that higher eHealth literacy is associated with increased use of the internet for health information. 104 There is a wealth of studies that suggest that adults with lower health and eHealth literacy are less likely to use the internet for health-related purposes compared to people with adequate eHealth literacy. 96,109,110,111 Tennant et al. for example, reported that those that used the internet reported higher mean eHealth literacy scores,96 using the eHealth literacy scale (eHEALS). 49 Studies have also found that people with higher levels of digital literacy ‘are more active consumers’ in reporting more comprehensive and sophisticated use of the internet. 112 Respondents with higher digital literacy in this large survey of 4286 people in the USA accessed a wider range of information on the Internet, used more search strategies, and appraised the information more carefully compared to respondents with lower levels of eHealth literacy.

eHealth literacy and use of online technologies for health information is affected by sociodemographic, environmental, and contextual factors. 108 Lower eHealth literacy is associated with a range of demographic and socioeconomic characteristics, similar to the utilisation patterns we outlined earlier. We would perhaps expect this given that internet usage is likely to be a crude proxy measure for eHealth literacy. Lower eHealth literacy is associated with older age104,113 lower levels of education,112,113 and lower income. 96 In a study of eHealth literacy in younger age groups, two studies using the eHealth Literacy Scale (eHEALS) reported a correlation between eHealth literacy and academic level that did not appear to be related to age. 114,115 However, a Sri Lankan study found no association between academic level or age and eHealth literacy. 116

There are some conflicting findings about the association between eHealth literacy and socioeconomic status with some findings suggesting a correlation between lower socioeconomic status and eHealth literacy. 113 Health status appears to be linked to eHealth literacy. For example, Neter et al. found that respondents that reported that they were chronically ill had a significantly lower eHealth literacy score. 112 Wong et al. using a multiple regression analysis and identified better self-rated health as a significant predictor to higher eHealth literacy. 104

There is some early evidence to suggest that the COVID-19 pandemic has disproportionately affected lower socioeconomic status groups, those in overcrowded living conditions and have further accentuated existing socioeconomic inequalities. During the COVID period, a study of a Hong Kong population reported that those with higher socioeconomic status had higher eHealth literacy and sought more web-based information on COVID-19. 117

It is important to note that much of the literature focuses either on teleHealth such as remote consultations or about using the internet to find and use health information, rather than specifically the use of online symptom checkers or services more similar to NHS 111 online, which is our focus here. It is also important to note that eHealth literacy skills are not static and evolve over time, and change within social contexts. The literature drawn upon here is from a number of different countries that differ substantially in healthcare systems, culture, wealth and context. It is also not clear how disparities in eHealth literacy may influence use of online services in the context of COVID-19. It is important to understand eHealth literacy in the UK context of using an online service designed to direct people to the right service for their needs and/or get advice about self-care.

The use of digital technologies and eHealth literacy in the context of urgent and emergency care

The goal of telephone and digitised health services in the context of urgent and emergency care is to increase the availability and accessibility of services as well as manage demand for face-to-face services, such as the ED, and reducing demand for the telephone 111 service. 10,25,118 Services such as NHS 111 telephone and online can provide initial assessment and direct users to the most appropriate service or offer self-care advice. However, such initiatives rely on patients knowing about, and being willing and able to use online diagnostic tools such as NHS Online or the national telephone helpline NHS 111. Consequently, access to urgent and emergency services in the UK (via 111 online or telephone) predicates that firstly, people are ‘health’ and ‘health service’ literate and secondly, they are digitally or technologically literate. It is unclear what this means for people who are unable to take advantage of technologies such as NHS 111 online.

Services such as the ED are often used by people who may struggle to access other services, for example, people from more deprived communities. 119 People with lower health literacy (rather than eHealth literacy) have higher rates of use of EDs,120 and out-of-hours primary care services. 121 People with low health literacy (rather than eHealth literacy) have also been shown to have difficulty understanding clinical risk and therefore use primary and emergency services for minor health problems. 122 To date, it is unclear whether digital health literacy similarly affects use of urgent and emergency services where technology facilitates access to these services.

Considering eHealth literacy and access to services is particularly important at the current time, where there has been an acceleration in the use of digital and teleHealth/eHealth technologies as a key strategy to provide services during the COVID-19 pandemic. To some extent, teleHealth has moved from being an optional service to an essential one. Prior to the pandemic, in the UK the NHS App had already enabled patients with smartphones to undertake a range of health-related activities, including checking symptoms, booking appointments, ordering repeat prescriptions and accessing their medical records. More recently, the App has added the facility to show a ‘COVID-19 Pass’ indicating vaccination status and recent test results (www.nhs.uk). General practice in particular has seen a transformation in ‘rapidly adopting digital approaches to triaging and consulting at a speed that would have been unthinkable just months before’. 123 Face-to-face GP consultations were largely or wholly replaced in many practices with telephone or e-consultations. 124 The use of NHS 111 online increased at the start of the pandemic in the UK with NHS 111 online usage increasing from 10 thousand users per day prior to the COVID-19 outbreak to 548,245 users per day in March 2020. 125

While there is some previous research that has described sociodemographic characteristics, access to and use of digital technologies (most notably the use of the internet to access health information), no previous studies have explored the use of digital health services in the context of urgent and emergency care, and whether digital literacy levels are associated with use. We seek to understand who uses NHS 111 online, if there is differential use of NHS 111 online, and understand the relationship between eHealth literacy and the use of NHS 111 online.

Who uses NHS 111 online and why?

Self-reported likelihood of using NHS 111 online for a range of health problems

Preferences for using NHS 111 online for 10 different health scenarios for users and non-users of the service are presented in Table 16. In the survey we asked people to rate how likely they would be to use NHS 111 online for a particular scenario (very likely, likely, neutral, unlikely, very unlikely). For the purposes of ease of comparison in these results, we have grouped responses into likely or not likely (removing the neutral category).

Comparison across scenarios shows two for which both users and non-users were especially likely to use NHS 111 online; ‘young child with a temperature and crying’ and ‘severe chest pain that goes away after a few minutes’. In contrast, the percentage of users and non-users reporting that they were likely to use NHS 111 online were lowest for mental health (tearful and not sleeping), injury (scalded hand) and common symptoms that could be self-managed (cough/cold, headache, diarrhoea and vomiting).

Comparison across user groups revealed that the percentage number of respondents in the user group reporting that they were likely to use NHS 111 online was higher for all health problems than for non-users. This aligned with them already having used the service for some other purpose. Chi square analysis of the difference between users and non-users for each service was conducted using Bonferroni adjusted alpha levels of 0.005 per test (0.05/10). These differences were significant for the scenarios ‘itchy bite or sting’, ‘young child with a temperature and crying’, ‘scalded hand’ and ‘painful when urinating’.

Of more interest is that a sizeable proportion of non-users who reported that they would be likely to use NHS 111 online, particularly for seeking advice about young children (76.3%) and severe chest pain (69.3). Additionally, nearly half of non-users reported that they would be likely to use it for an itchy bite or sting (42.5%), pain when urinating (50.9%) and a headache for several hours (43.3%).

Understanding users and non-users in terms of eHealth literacy

Table 17 shows NHS 111 online users compared with non-users for each of the seven eHLQ domains. A higher mean score indicates a higher (self-reported) health literacy score (on a scale of 1 to 4). Across almost all domains, NHS 111 online users had a higher mean score than non-users suggesting a higher level of digital literacy overall. Analyses of the difference between users and non-users for each service were conducted using Bonferroni adjusted alpha levels of 0.007 per test (0.05/7). Significant differences were observed for all domains except domain 4 (‘feeling safe and in control’) and domain 6 (‘access to digital services that work’). Effect size calculations revealed differences between users and non-users were largest for the domains of ‘being able to process information’, ‘being comfortable using digital services’ and ‘feeling that engaging in digital services will be useful for managing their health’.

Further analyses compared mean eHLQ scores across the key sociodemographic variables of education, age, and the presence of a long-term condition where prior research has reported differences in health or eHealth literacy within these groups.

What is the impact of NHS 111 online on patient pathways of care?

This component of the study addressed the first objective: to describe the pathways of care and services used by patients who access NHS 111 online. We set out to understand where NHS 111 online is positioned in the wider system of urgent and emergency care, and to describe the different way that users of NHS 111 online can navigate this system. We used qualitative telephone interviews in England with a range of people connected with primary and urgent care, emergency departments, dental services, and with charity organisations representing vulnerable people to collect primary data. We also drew on a range of policy documents and diagrams that provided models, diagrams or descriptions of pathways to care. We used these documentary and interview data to describe and create new visual depictions of pathways to care.

Despite increased use of NHS 111 online during the pandemic the service had limited visibility among staff and stakeholders in the wider health system. We found that 23/33 interviewees in primary care, 19/27 urgent and emergency care interviewees, none of the dental services interviewees and only one charity stakeholder interviewee were aware of NHS 111 online. There was greater awareness of the NHS 111 telephone service, and growing awareness of the 111 First initiative, which uses the NHS 111 online assessment and triage system to manage access to EDs.

We explored the reasons for this lack of visibility for NHS 111 online and posited two interconnected explanations. Firstly, NHS 111 online sits in a very crowded digital technology landscape. There are a number of alternative triage, assessment and appointment booking systems, notably different eConsultation systems used in primary care that compete with NHS 111 online. Some of these link to NHS 111 services, referring to or receiving referrals from these services, for example by offering a hyperlink to the NHS 111 online website. Alongside these systems there are a variety of digital communication and electronic record systems that offer remote communication modalities (such as text messaging) but also sometimes provide online self-care support and resources for patients, overlapping with some of the functionality of NHS 111 online. Beyond these services we also identified that other web-based resources and social media platforms, including the nhs.uk and gov.uk websites, and Facebook pages for general practices also offer users and patients access to a range of health advice. Among these many digital technologies, NHS 111 online does not stand out, it is simply one of many systems that users, patients and staff may come across. This finding confirms the conclusion of the Sheffield study that NHS 111 online has added an additional access point for urgent and emergency care in the NHS and this may not have made access routes clearer or simpler for patients and service users.

Secondly, NHS 111 online is part of a wider service landscape offering a range of different primary, urgent and emergency care services and this too adds to confusion about where patients should go. We used the conceptual distinction between ‘work-as-imagined’ and ‘work-as-done’ to understand the gap between policy and health education messaging and representations of the service landscape, and the experiences of service users/patients and healthcare workforce of pathways to care. We concluded that pathways to care are confusing and difficult to navigate and that the role and position of NHS 111 online in the service landscape is not always clear.

Is there evidence for differential access and use of NHS 111 online?

Chapter 5 presented the findings of a survey examining the evidence for differential access to, and use of, NHS 111 online. Our findings revealed that self-reported eHealth literacy scores and a number of demographic characteristics of the sample population were associated with previous use of NHS 111 online. Although we did not find a significant effect of sex in logistic regression, a greater proportion of NHS 111 online users were female. This finding is consistent with prior research. Younger respondents were more likely to use NHS 111 online and were more likely to have higher eHealth literacy, confirming the findings of previous research. 48,97,98,99 Among previous users of NHS 111 online, those in the youngest age bands and those with at least some form of qualification were proportionally more likely to report having used the service than older people (who perceive themselves as lower skilled with digital technologies). Previous users were also found to have significantly higher levels of eHealth literacy than non-users on all domains of the eHLQ with the exception of domain 4 (‘feeling safe and in control’) and domain 6 (‘having access to digital services that work’). These differences in eHealth literacy were present despite the COVID-related adjustments made to data collection in which almost all participants completed their survey online suggesting some degree of digital competence.

Additionally, of those with a LTC a higher proportion were found to have used NHS 111 online than those without a LTC. This finding is consistent with previous research on internet use that has found that those with chronic health conditions show higher rates of information seeking but only where they have higher levels of digital skill. 98 In the present study, while presence of a LTC was also found to be correlated with lower scores on the eHLQ, grouped analysis of LTC and non-user/user found that those with a LTC who had used NHS 111 online had equivalent eHealth literacy to those without a LTC. Of note here is a group of people with a LTC who had not used NHS 111 online and whose self-rated eHealth literacy scores were lower than other groups. This raises the possibility that feelings of digital competence may be a barrier to accessing urgent care via this route. Our finding that people with no formal qualifications are less likely to use NHS 111 online is consistent with literature elsewhere. 48,97,98,99,101 Together these findings are in line with prior research which has examined eHealth literacy in specific patient groups,113 but represents the first research that has specifically looked at eHealth literacy in relation to the use of urgent care services.

NHS 111 online users were also more likely to have used other NHS urgent and emergency care services, such as NHS 111 telephone, urgent care centres, GP out of hours, 999 ambulance and the emergency department. They also had higher cumulative use across these services compared to non-users suggesting they are using these services more diversely. While the data cannot reveal the reasons for this, possible explanations include, that NHS 111 online acts as a gateway to other services, and/or that greater awareness of the range of services available, or a higher burden of need, encourages use of more services.

Responses to the questions about the scenarios in which people would consider using NHS 111 online suggest people are unlikely to use it for cold symptoms and headaches, but that they might use it for potentially more serious chest pain symptoms. This may reflect a difference between what people say they might do in response to a hypothetical scenario, and what people do when they are actually feeling unwell. Our work with PPI indicates that this may be the case, but nonetheless there may be a need to clarify the differences between the NHS 111 telephone and online services and encourage greater awareness of the most appropriate services to use.

What are the workforce implications of introducing NHS 111 online? How do workforce arrangements (e.g. staffing, skillsets, task allocation), vary within different types of NHS 111 online services? How do variations in these workforce arrangements impact on the wider health and social care system?

Notwithstanding the lack of awareness of NHS 111 online in England, and Healthdirect’s virtual triage in Australia, our interviews with staff and stakeholders have identified some of the kinds of work, workforce arrangements and impacts associated with online triage and assessment and the telephone versions of these services.

Staff and stakeholders in primary, urgent and emergency care settings perceived that NHS 111 services created additional demand for ED services. Interviewees felt that the triage and assessment algorithm was risk averse and resulted in ED attendances by people who could or should be managed by other non-emergency services. While the statistical picture about whether actual increases in demand are associated with NHS 111 service use is more nuanced,28,130 this narrative continues to be a powerful negative message about NHS 111 services and one that could gain further weight as use of NHS 111 online grows. There is one UK study suggesting that clinical call handlers decreases referrals to ED,131 and this is supported by evidence about Healthdirect that suggests that clinician triaged calls achieve a higher level of appropriate onward referral. 132,133 The interviews did not suggest that there had been a significant shift of demand for other services related to NHS 111 in general, or NHS 111 online specifically. This confirms a finding from the Sheffield study (p 139),129 and counters other commentary. 134 However it is worth noting that this may be partially related to the pandemic risk mitigation measures, and the use of NHS 111 to try to reduce demand for face to face care.

In primary care, interviewees described additional administrative work deciphering reports from NHS 111 services, and clinicians described duplication of assessment of patients referred by NHS 111 services. There was some overlap here with the urgent and emergency care interviewees who also noted some extra administrative work dealing with reports, and clinical re-assessment. While there was little evidence from Healthdirect that significant additional work was generated for healthcare staff by the Symptom Checker, there was nevertheless a suggestion that triage and assessment was often repeated, with this additional work falling on patients who had ‘already completed the questions’ (making this work less visible to staff who were repeating the assessment). While studies in the UK and Germany have shown that computer assisted telephone triage can be safe, other research has argued that self-assessment systems may be more risk averse than GP assessment of urgency in primary care help seeking. 9 This perception, combined with high profile cases where coroners have judged that NHS 111 telephone triage has failed to identify health risks accurately may encourage this risk management behaviour by clinicians. 135,136

Our interviews with other potential stakeholders associated with NHS 111 online included dental service providers and representatives of charities that work with vulnerable people. Both these clusters of interviewees represent groups who may seek and need urgent healthcare advice or treatment. The lack of awareness of NHS 111 online meant that these respondents had nothing to contribute to the evidence about work generated for their services or client groups by NHS 111 online. However they did identify potential areas where greater knowledge and use of NHS 111 online might benefit their patients/service users. There appeared to be opportunities for NHS 111 services, including NHS 111 online to support patient access to NHS emergency dentistry. At present patient access to such services via NHS 111 is different in different areas of the country, and some of this demand was referred to EDs where it was often seen as inappropriate.

How does UK NHS online workforce compare with the Australian Healthdirect service?

The workforce supporting the NHS 111 online service and the Symptom Checker element of Healthdirect are similar, and comprise small teams of service managers/operational leads, developers and clinical staff who review evidence and clinical guidance and oversee content. The key area of divergence is that Healthdirect has retained a clinician led service for its telephone triage and there were mixed views about the differential benefits of this. NHS 111 telephone services can offer transfer to, or call back by, a clinician. The wider network of healthcare providers implicated in patient pathways to care in both countries includes a range of primary, urgent and emergency health services staff. The work performed by these staff associated with digital triage and assessment is more visible in the English context. Table 21 shows how the two services compare.