Dear Bot, may I speak to your manager?
Dear Bot, may I speak to your manager?
It's popular to propose using AI to deflect front-line CX messages and offer support. What are you giving up and what do you gain? Read: "Everything Starts Out Looking Like a Toy" #160
Hi, I’m Greg 👋! I write weekly product essays, including system “handshakes”, the expectations for workflow, and the jobs to be done for data. What is Data Operations? is a post that grew into Data & Ops, a team to help you with product, data, and operations.
This week’s toy: a quick way to test your memory and see how well you can draw … try to copy the Mona Lisa. What’s interesting about this? Getting a sense of how much our brains fill in for us on a regular basis, even when we think we’re paying close attention. Edition 160 of this newsletter is here - it’s August 28, 2023.
If you have a comment or are interested in sponsoring, hit reply.
The Big Idea
A short long-form essay about data things
⚙️ Dear Bot, may I speak to your manager?
It seems like almost every company with a large volume of customer requests is looking into AI assistance to help them deflect (in lay terms, avoid) initial conversations from customers so that they can do more with less.
Using AI, the thinking goes, will make the experience better for the customer by presenting relevant resources to them and better for the company by waiting until the last minute to drop the team member into a chat with the customer. Ideally, that team member will have excellent context due to AI helping them understand the summary of the customer’s request and their past experience. It might even hint that the customer is kind of annoyed given a sentiment analysis of the text and the comparative speed of the keystrokes that are happening.
But are we missing the opportunity to think at the same time about improving the customer experience of typing to (and perhaps, in future, speaking to, or seeing a virtual) bot?
From a product perspective, this falls into three broad questions to answer:
Should a bot answer truthfully when asked if it is a bot?
What’s the benefit of offering a way for a human to ask for escalation?
How will we measure the success of using an automated bot to handle initial requests?
Satisfaction = Perception - Expectation
When you ask a question and you get a middling to poor or evasive answer, how do you feel? You’re probably frustrated or wondering how you can find a better answer, or might want to escalate to speak to a manager.
Compare this to getting a comprehensive answer that’s different than what you expected or wanted. You might feel frustrated, but less likely to escalate because you are pretty sure nothing will happen.
Finally, when you get a pretty good answer with details and it solves your problem, your emotions range from satisfied to pleased.
All of these results are related to your expectation – how you thought the situation would go – and the perception of how it actually went.
I’d argue that all Conversation Bots should have a feature that you could enable that would clearly enable that bot to show that it is a bot. Whether it’s using a 🤖 emoji or stating that it’s automated, bots need to make it easy for humans to know they are automated.
There are two primary reasons for making it clear that a bot is doing the answering:
Bots can do so much more than humans, but humans need to know how to talk to bots to get the right answer. (Skeptical people might say this sounds like a phone tree, but I think it can be better with generative AI tools that allow you to use plain language to ask what you want.)
Humans can emote much more than bots, so it’s best to flag that there is a human on the other end of the conversation who has more emotional capability than a bot.
Why offer an option for a human to request a non-bot?
The Toyota Product Method profoundly changed manufacturing, infusing methods of quality deeply into the production process. One of the key tenets of that system is the idea of an andon cord, a way for plant workers to stop the line by pulling on a cord and prevent defects from moving later into production when they are spotted.
Prospects and customers need an andon cord equivalent to stop a bad bot conversation. When the results or the tone of the conversation are wrong, we need to be able to ask for a human, or route the conversation to a human automatically so that the prospect is less frustrated.
Is this less efficient? Probably on the individual conversation level, but it’s key to retaining brand equity. Brands that offer a more human experience will differentiate themselves against brands that offer only bots.
How do you measure bot success in a CX context?
Using a bot in a CX context is valuable if it does the following:
answers more questions faster and more consistently at a higher quality and satisfaction rate
helps team members to answer more complicated questions in an effective manner
doesn’t lower the overall customer satisfaction
This means you’re going to be watching the service level (during a time frame, the number of questions answered / number of questions offered), the CSAT (are prospects and customers satisfied with the experience), escalation count (how many times do people pull the virtual andon cord), and sentiment (are there generally the same proportion of happy, neutral, and unhappy messages).
Does all of this tell you whether or not to hire a bot to answer your inbound CX messages? Nope. You’ll need to decide whether you want your brand to human or robotic. By the way, another way to consider this is to think about how to help your team behind the scenes. Bots are amazing at classifying and summarizing information - this can be a bigger lift to the front line than simply handing off the answering part.
What’s the takeaway? It’s likely that you will think about using AI to save costs and improve your CX process. But simply placing a bot on the front lines to answer questions might give you a lousy customer experience. Why not make it possible for customers to ask for a human, or simply prioritize the humans while providing AI help and summarization for them behind the scenes?
Links for Reading and Sharing
These are links that caught my 👀
1/ Pizza and 💸 - How can you fail to make great pizza with $442 million dollars? The story of Zume is a cautionary tale of how building technology before you nail the problem you are going to solve can fail. One unexpected tidbit of trying to make pizzas while moving: the cheese falls off before it cools.
2/ Add AI to your Figma - The team at Figma has introduced Jambot, a plugin for Figjam. The idea? Use Jambot during a Figjam to get quick iteration, answer a question, or get unexpected results. This works well when you’re building a structured process and don’t know what’s next. It’s not quite as effective if you want to explain an existing process and make it better.
3/ Python in Excel - If you haven’t heard, Microsoft announced Python support inside of Excel. This is a giant step to bring low-code programming into spreadsheets and to open up this underappreciated code surface. What will Google do to match this? Seems like a no-brainer to allow Python to do the same in Google Sheets.
What to do next
Hit reply if you’ve got links to share, data stories, or want to say hello.
Want to book a discovery call to talk about how we can work together?
The next big thing always starts out being dismissed as a “toy.” - Chris Dixon
There have been times where a bot has absolutely solved my problem. One example was Sunbasket (meal prep service). They forgot a key ingredient and handled the whole complaint/process seamlessly. I didn't feel as though I needed to talk to a human. They clearly thought through all of the different scenarios.
On the other hand, most are terrible and do not give you a good way to get to a representative. Uber Eats is a brand that comes to mind. They make it infuriating to get to a human, but even when you do, they are not skilled at understanding context on why the problem was that bad. Peloton is also bad at this.
Absolutely - think it's important to call out situations where sometimes, talking to a bot is better. The key of the experience? Knowing you're talking to a bot ;)