After Vodafone was fined £4.6m by Ofcom for failing to handle customer complaints properly, citing outdated technology and poor training, its chief executive Nick Jeffries vowed to tackle the issue head-on by completing IT migration critical to customer support infrastructure, consolidating call centres into core hubs and in relevance to my work; ramping up customer service using artificial intelligence, voice/biometric ID, Alexa skill and iterative improvement of help and support tools.
Picture the scene. You’ve just bought a new phone on a Vodafone contract. You get it home, tear away the packaging, slide in the sim and… it doesn’t work. After reading the instructions for the millionth time, you decide you’re not to blame: Vodafone is.
You decide to head online to see who can help and how this can be resolved as quickly as possible. After briefly reviewing customer FAQ’s without success, you become increasingly impatient. You experience a further 10 minute wait to speak to a customer services agent via Live Chat, who manages to troubleshoot the query with you and get you on the move again.
Not a good start, right?
This is just one of the user stories we have been designing for in Vodafone Group as part of our vision to improve the customer experience across sales and service queries.
A big part of Vodafone’s promise to improve this customer experience is TOBi; the IBM Watson AI-powered ‘chatbot’ will assist customers making enquiries online, deflecting contact centre enquiries whilst allowing users to easily access account information and complete customer tasks. TOBi was already being deployed as an MVP in both Italy and UK when I joined Vodafone, geared to respond to a selective few support questions.
A-Synchronous messaging pilots in Ireland and Egypt, aimed at enabling Vodafone to communicate with customers without the need for either party to be continuously engaged (live) in conversation were also under way, but without a consistent framework for experience design.
The project goal was to describe the optimal experience design and interaction model for the “Chat” product suite across Vodafone group, and how it would integrate with other help and support tools in My Vodafone App and web. My tasks included;
- Design of a pattern library for conversational UI that delivers the user experience via familiar, easy to understand paradigms
- Establish best practice UX guidelines for Vodafone Group and local markets capable of implementing chat tools using customer acceptance testing to increase business confidence in design
- Plan, prototype for and conduct multiple phases of customer acceptance and usability testing across 6 international markets.
- Provide Q&A to all international markets implementing “Chat” into their help & support tool suite.
I joined as a UX Consultant with BAE in October 2017 and continue to work on this project, and other Vodafone work streams.
The selection and priority of the query/user journey release plan for TOBi was driven by data from current live chat intent capture, customer call centre drivers, and technical complexity in creating the ‘natural language’ modelling in local markets - in other words, as a business we prioritised less technically demanding journeys of similar assumed customer benefit.
Service additions - TOBi
An effective bot is one that is able to understand the user’s language and the context of conversation throughout the chat. While comprehending the language of user requires text and linguistic analysis i.e parts of speech tagging, phrase analysis etc. analyzing the context of conversation involves modelling the entities, intents, and actions of the user.
A collaborative, user-centred, iterative design process in Agile Scrum was used to guide this project from kickoff, design and successful development in 4 international markets: UK, Ireland, Italy and Egypt to date.
Early collaboration and ideation using cross-discipline workshops were key to setting our core design principles, and capturing the most fundamental user requirements.
However, user data including interviews and behavioural analysis of existing help & support tools would be the driving force behind our experience design and tone of voice creation. Ongoing feedback followed by iteration helped to deliver a set of products that delivered on core business needs, but also received consistently improving task completion and satisfaction rates.
Discovery & Planning
As a first step, it was important to gather internal and external data that would help frame the problem, contextualise the landscape and help garner a shared team vision of what we’re trying to create.
Despite chatbots having been around in some capacity for fifty years, the recent rise in their usage can be largely attributed to two key developments:
A. Messaging Services Growth
Messaging technology has spread rapidly over the past few years, more recently driven by the inclusion of features such as payments, ordering and booking, which otherwise would require a separate app or website. So, rather than having to navigate through separate channels and services in an end-to-end experience, users can buy products and ask questions all through their favourite messaging apps.
B. Advances in Artificial Intelligence (AI)
Artificial Intelligence has also made significant steps forward in the past few years. Techniques such as machine learning and deep learning take advantage of vast amounts of data and cheap processing power to dramatically improve the quality of understanding and decision making. For example, using techniques such as deep learning, image recognition rates have fallen to a level that exceed average human performance levels.
Reviewing existing design paradigms
Users will enter our service with a mental model; influenced by their interactions with similar looking and feeling messaging experiences.
In a customer service context, we can think of chatbots and A-Syncrhonous messaging as the combination of three parts:
- 1. User Input
- Text and Voice input are combined with Quick replies to facilitate task completion
- 2. Status
- Activity - TOBi is thinking, Agent is typing - provide indication to user that there is activity on Vodafone's side of the interaction
- Who you are talking to - TOBi, Customer Services Agent or Group Chat
- Availability - E.G: Typically replies instantly (TOBi only), Typically replies every X to X minutes (A-Synch and Live Chat)
- 3. Messages
- Messages from different actors in the conversation are clearly distinguished.
- Cards and in-page draw modules are often used to provide additional CTA's in-context
What are customers doing when visiting help and support areas on app and web?
- Top Help & Support queries by market e.g Ireland (helped to prioritise user stories)
- Top call drivers
- FAQ's via Social media chanels
- Search items
- Help & Support landing page click map
Contextual interviews with customers
We carried out a phase of contextual interviews with recruited, existing customers to help us understand their thoughts, feelings and experiences when using the current TOBi pilot and other similar chatbot services. Our key insights were to;
Avoid using chatbots in cases where the customer is likely to be sensitive or stressed, such as dealing with complaints. Trained human agents are better able to express the empathy to deal with the situation. We can use Natural Language Processing to identify these cases and divert them to human agents.
Clearly distinguish the ‘bot’ identity versus the ‘human’. When reviewing task completion using the existing UK pilot, users were prompted to describe who they believed they were talking to after an initial exchange of messages. Several customers were shocked to find it was in fact automated, resulting in a negative feeling of mistrust and caution about discussing sensitive information moving forward.
Ideation & Synthesis
Design thinking techniques were used in various capacities to generate and synthesise ideas, extract and leverage domain knowledge and key insights from business/product stakeholders and the core team, whilst establishing shared enthusiasm for the design task ahead.
UX Ideation Workshops
To kick the project off, several group (multi-discipline) ideation and brainstorming exercises were conducted with a view to building upon insights learned during research, by creating design principles, an archetype for our chatbot TOBi to guide our tone of voice and an interaction model to guide our experience design.
Rapid Prototyping Workshops
Once this documentation had been created to give a shared view of what the desired user experience ought to be, regular workshop sessions then shifted priority on to concept a potential solution design - thinking about what technology options enable and constrain us as designers.
This opened up conversations with wider business and local market teams to understand what technical considerations we had to consider that would potentially expand our detail of GRD - as various scenarios for solution design were to be given based on back-end integration methods.
This provided a unifying base from which the team could use to move into user requirement definition, prioritisation and sprint planning through to our first ‘Chat’ GRD release.
User Experience Design
Chatbots matter to customer service for two main reasons:
Although a chatbot cannot handle all customer queries, it can be used to deal with many of the routine queries that typically make up most service requests.
In the UK pilot run by Vodafone using TOBi on a set of common customer queries it resolved 82% of queries by itself, rising to 88% of interactions when combined with live intervention by a human agent. Effectively designing for this handover between bot and agent would be key to our experience design.
Chatbots make it easy and fast for customers to reach you using the same messaging services they use daily. Chat is also an easy medium for most people to use, as it is a much more natural way to interact.
TOBi as primary customer help & support channel
As we cover more use cases in design and TOBi’s AI reaches a high percentage of customer intent coverage; the business objective is to reduce reliance on costly upkeep of support channels through the customer services centre, and increase diversion through to TOBi as a customer self-service tool.
8 Channel’s for Help & Support
However, due to varied market technical capability and TOBi’s capability in customer use case current coverage being low, the Vodafone help & support ecosystem would service the following help and support tools alongside each other, with options presented depending on customer intent.
- Community / Social
- Call us
- Find a store
- Live Chat
- Message Us
Our GRD would seek to give interim guidance to markets implementing TOBi and Chat tools alongside all existing help and support channels.
Under the guidance of our design principles, for GRD we would design ‘Chat’ as an ecosystem of help & support tools that would give clarity to the user on the benefits and value of selecting one channel over another.
To facilitate a shared understanding of the experience strategy across the core team, I created a low-fidelity interaction model that reinforced the current positioning of TOBi’s role in customer support, as this would be the initial paradigm we would be describing in design.
Localised AI Messaging System Flows
The illustration of the unique triggers and corresponding messaging flows with TOBi within the system were to be defined by the local markets, with GRD aimed at describing optimum experience design for baseline user journeys, and providing local markets with a component library to rapidly implement the solution design with localisation.
Agile UX/UI design
In a typical sprint for UX, I would begin by discussing the user stories with a business analyst, gaining a brief on the business context from our product owner and then validate the user journey before progressing on to low-medium fidelity wireframes.
Iterative Wireframe Design
Medium fidelity wireframes, in combination with formalised user testing that would dictate each monthly release to global markets, were used throughout the design process to get our ideas in front of users for feedback.
Our early prototyping would be to use Axure, as this was the most commonly adopted tool for Vodafone, but after a battle for development resource we began to code the UI components and integrate a test-background AI in order to replicate more realistic interactions.
This rapid iteration allowed us to gain confidence in the optimal architecture and design of both the interface, and the conversational modules we were providing to local markets.
Screen Structure & Content Modules
The interface structure drew on many familiar interaction paradigms and visual cues that would purposefully lean on predicted mental models.
Formalised User Testing
User testing at Vodafone is formalised within the market delivery cycle. This afforded design the opportunity to regularly engage with customers already knowledgable and confident users of similar chat tools, as well as those who would be much more anxious about communicating with a bot and more likely to request handover to a (human) customer services agent.
We investigated user perceptions and understanding of Tobi across multiple markets; including the UK, Greece, Italy and South Africa. This was explored through user interviews, multiple user journeys and task completion exercises - such as solving a complex billing issue and completing a simple Sim-only sales.
Sample Research Objectives
Evaluate ease of use of key Tobi chat bot interactions including accessing the service, logging into your account, interacting with visual elements in the chat stream, handing off to a human live chat operator and completing sales processes.
Evaluate user perceptions towards Tobi chat bot including the mental model customers build of Tobi as a service and the influence this has on their interactions in the chat journey including responding to Tobi and (when needed) transferring to human operators.
Sample Key Findings
KPI tracking for Chat
Having metrics to measure and visualize the performance of chat tools is important. Just like we track and monitor the performance and metrics in other key areas of our business, we need to monitor our chatbot metrics properly if we want it to perform better.
As detailed in the following model, the 3 most important indicators for TOBi's performance are:
- 1. Containment rate
- % of chats handled and resolved entirely inside TOBi.
- 2. Handover rate
- % of chats handled and then handed over from TOBi to a (human) customer services agent.
- 3. Failure rate
- % of chats with a system error that prevents the customer from completing their intended task, or resolving their query.
After a spike in focus on "Chat" as a customer service tool, our work in GRD would shift in to other areas. We would continue to collect insight from local markets on performance and provide guidance on best practice until we re-visit the next phase of TOBi's UX development.