Understanding the emotions of users through natural language processing

Winning customers is hard, but keeping them is harder. The best companies invest in proper customer support because they know that a good CS team is more than just a team of firefighters: it’s a treasury of knowledge about your product, your users and what those users are thinking.

A customer usually contacts your company when things go wrong. The more critical the issue is, the more emotional that message will be. Often these messages are the first indicator we get that a critical bug exists in the system, or that we’ve made a poor UX decision, or that a feature is causing more problems than it solves.

Customers also contact us to convey positive sentiments, whether that’s via the App/Play Store, Trustpilot or directly via email. It’s important that we understand and are able to sort the sentiment of these messages as well, so that we know what we’re doing well. Both kinds of feedback are crucial to understanding our users and building the right product for them.

Azimo now moves hundreds of millions of dollars around the world every year. With that kind of scale, it’s not easy to identify and sort all of our customer feedback so that we can refer to it effectively in customer research. As there is nothing more helpful than people’s feedback, we decided to tackle the challenge by integrating sentiment analysis into our customer support tools.

The core of our support system is Zendesk platform, this is where all of our customer communication happens. Zendesk handles:

Each new message is analysed and dispatched to the right team and the agent to handle it. However, besides internal automation (agent assigning, automatic replies, SLA policies), Zendesk supports external service plugins through webhooks.

One of our webhook implementations for Zendesk is a service we built called Ticket Enrichment. When a new message appears in Zendesk, a call is made to the Ticket Enrichment service, which then analyses the content and parses data through various internal functions, like Natural Language Processing among the others. When all data is ready, it sends back the enriched ticket to Zendesk via their API.

One of the features of our Ticket Enrichment service is text analysis, which helps us classify what the sentiment of the incoming message was. Thanks to Natural Language Processing, we can tell whether the message was:

Thanks to all these features, we can analyse thousands of tickets every week. We now have a much better idea of which features people like, which are giving them trouble, and which critical bugs need to be fixed.

And this is just the beginning. Natural Language Processing shows a lot of promise and its accuracy is increasing all the time. Today we can identify and understand a user’s problem programmatically before a customer service agent even reads the message.

Right now an engineer still needs to triage the problem and deploy a fix, but we’re hopeful that in future we will be able to fix a user’s issue, or at the very least provide them with accurate information while they’re waiting, automatically and immediately.

For the rest of cases, text classification will substantially reduce response time, and allow us to assign a ticket to a customer service agent with the appropriate expertise to deal with the customer’s issue. No more “let me just pass you on to the best person to deal with this.” 😎

As our users’ privacy is crucial for us, we won’t share any real conversations. Instead, here are some test cases we wrote which show how sentiment analysis works. In the cases below, we would have been able to identify not only sentiment but also the nature of the problem/compliment.

“Hey, I have a problem with your app. It is constantly crashing on my device. I cannot login and I’m not able to do transaction. It is useless. And I cannot contact with you. For sure this is very last time when I’m contacting with you.”

“I don’t like your service! You are slow, you don’t respond my questions. Your app doesn’t look very good on my phone. I won’t recommend you to anyone.”

“Hey Azimo!
I love your app! I’m your client for years now and I have never had any issues with transactions. Money is delivered on time, very quickly. Thank you for doing your hard work, you are the best company!”

“Your app is awesome. It makes transferring money to my home country easy and fast. I can’t ask for more. I love it.”

“Dear Azimo. I’ve been trying to contact you, I have questions about my transfer. Can you tell me what is the best way to talk with you?”

While sentiment analysis is powerful technique that helps to understand your customers, for us it’s just the beginning. The state of the art in Natural Language Processing gives us tools to not only classify emotions hidden behind a message but also find its deeper meaning. By extracting keywords, categories and sentence structures it is possible to build solutions that automatically analyze all communication channels in real-time, suggest trends (what people like or dislike the most in our product), and track it over the time (has the last direction in building UI made people like our app more?).

With growing scale, it is extremely important to understand your customers. So what can be better than listening their feedback? With Natural Language Processing we are able to hear everyone who wants to speak. No matter when and how much they want to tell us.

It would have never be possible for us, humans.

Related posts:

An amalgamation of Haikus for Dead Poets Live. “Elements of Life” is published by Ashwini Dodani in Dead Poets Live.