How not to create disappointing chatbots?

The chatbot revolution 

Even the most skeptical amongst us will have a hard time denying that chatbots are taking the world by storm. Social listening data shows an explosive rise in popularity – discussions about chatbots went up significantly over the last two years: from 110,000 detected mentions in May 2017 to 830,000 in May 2020.

A SentiOne mention graph showing the increasing volume of mentions for the term "Chatbot" (and related terms) over the last two years.

Clearly, chatbots provide a significant value proposition to brands and businesses – and the market is only happy to deliver, with new software solutions and offerings appearing on a daily basis. 

Various market analysts forecast a bright future for chatbots – Business Insider, for instance, stated that by 2020, 80% of enterprises will employ a chatbot solution in some manner. Surveys, such as the one conducted by Spiceworks, showed that 40% of large businesses (as in, employing more than 500 people) planned to invest in chatbot technology in 2019.

However, you don’t need to be an analyst to see why chatbots have taken off in such a dynamic way. They are simply the natural fit for the current marketing and customer service landscape! 

Increasingly, our customer bases are primarily composed of millennials and Gen Z – the so-called zoomers. These are people who grew up immersed in the internet and social media. They are, by and large, tech-savvy and accustomed to receiving instant results. They also value convenience – and when it comes to getting in touch with a brand? There’s nothing easier than reaching them on social media or over instant messaging.

Chatbots, then, are a natural fit when it comes to satisfying the expectations of our youthful clientele – taking a closer look at the data, however, reveals something even more interesting.

Many of us would assume that just as the younger generation is heavily skewed towards using social media, older generations heavily rely on more traditional ways of communicating with brands – such as phone calls.

This may not be the case! According to this archived report published by Tidio in 2019, 44% of online shoppers aged 45-60 prefer using chatbots over other methods when purchasing online. This goes to show that chatbots as a technology appeal to wide swathes of consumers – not just the zoomers and millennials!

Chatbots in practice

Knowing all this, then, it comes as no surprise that increasing numbers of businesses are deciding to take the plunge and deploy a chatbot of their own. The results, however, are often mixed, as we’re about to learn.

When taking a closer look at chatbots as a whole, it would be prudent to examine some of the longest-standing examples on the market. SkyScanner introduced their much-hyped chatbot solution all the way back in 2016 – and as one of the largest flight search engines, surely they’re leading the industry by example?

Well, not so. We tried messaging their chatbot and were told to use the website contact form instead.

A disappointing experience with SkyScanner

As it turns out, Skyscanner’s chatbot – something that should be available at all hours of the day and night – is only online when their customer support team is on call. This honestly boggles the mind!

Once that hurdle is cleared, however, you might still be treated to a slow and unresponsive experience – which was a common point of contention in reviews of the SkyScanner chatbot as late as a year after it launched. Admittedly, things have changed for the better – but this isn’t an isolated experience.

Expectations versus reality

All too often chatbot users complain about poor responsiveness, misunderstood queries or simply a lack of functionality. A survey published in early 2018 by sums these complaints up quite effectively. Customers hate it when chatbots “get stuck” – and they especially loathe having to repeat themselves to human agents if their ticket gets escalated.

These surface-level complaints, however, often disguise much deeper problems with the implementation. In our experience, these problems are most often caused by a lack of understanding of both the underlying technology and customer expectations.

A lot of brands attempt implementing a chatbot without providing it with enough historical data to learn from and act surprised when their results are less than satisfactory. In fact, the necessity of providing a training dataset at all often catches brands off-guard! It is crucial to understand that chatbots are not a one-size-fits-all solution. Every use case is different and requires a personalised approach.

What compounds the problem is the fact that many companies decide to deploy simpler, rule-based solutions instead of more robust chatbots powered by a natural language understanding engine. This severely limits the bot’s ability to learn and adjust to customer behaviour.

Another common misconception that leads to poor results is that chatbots are a one-and-done affair. This couldn’t be further from the truth – chatbots thrive only when their performance is constantly monitored and optimised. The amount of effort involved in analysing and fine-tuning a chatbot to its intended use case leads a lot of brands to committing the grave error of just not bothering – facing customers with a sub-par solution. It’s important to understand that it’s simply not possible to have a chatbot with a 100% intent detection rate right out of the gate.

Falling into these misconceptions will result in your brand releasing another sub-par bot onto the internet, leaving the customers with an unsatisfactory experience. It is therefore crucial that the people responsible for the chatbot – managers, heads of IT and customer experience, chief innovators – do their homework and understand what releasing a chatbot really entails.

How to do chatbots right

Don’t let all of this discourage you from investing in chatbots, though! Where there’s a will, there’s also a way, after all! Our years of experience creating chatbots for a diverse selection of clients allowed us to gain a lot of valuable experience and insights into what truly makes chatbots a success. We’re not going to keep these a secret – here’s what you need to know to create a truly great chatbot.

Automation is a marathon – not a sprint

Chatbots are a long-term commitment. Like everything in automation, it depends on continuous, iterative development. Start small, figure out what works and what doesn’t, and improve accordingly.

Chatbots aren’t everything!

Customer service automation isn’t just chatbots. AI-powered solutions can help in many diverse areas of the customer service process – from automatic ticket routing and load balancing, through AI-generated response templates to incident detection!

Human + AI = 🔥

AI exists to assist humans – not to replace them. Unless a giant technological leap straight out of science fiction occurs, even the most cutting edge technology available will never be able to replace a human being. Instead, the best results are achieved when AI and humans work together.

The five steps to a succesful chatbot, illustrated

The five steps to a successful chatbot

  1. Gather as much relevant data as you can. You’re looking for historical data from every single communication channel and customer touchpoint you have. Then, segment that data into different categories: billing, sales, complaints and so forth.

    Once you’ve done that, you can narrow down which of these processes you’d like your chatbot to handle. Chatbots excel at handling monotonous, repetitive tasks which quickly tire out human agents. The processes for which you have the largest amounts of data are also a good candidate – after all, preparing a solid training dataset is vital for your chatbot.
  2. Prepare a training dataset relevant to your industry, customer requests, processes and intents. Training a chatbot based on your data is simpler than you think – simply provide it with examples of questions it will be exposed to and the answers to those questions. You may not realise it, but you likely have a lot of training data in your system already. Ensure that you have a very wide, very diverse set of variations for each question to make sure the bot becomes as accurate as possible.

    Once the model is trained, it will be able to identify requests even if their wording doesn’t precisely match its training dataset. That’s the true power of AI.
  3. Decide which KPIs define how successful your chatbot will be – after all, simply creating a chatbot shouldn’t be your end goal. What challenges do you need to overcome? What’s truly important for your business?

    Some example metrics that can be helpful in evaluating your chatbot’s performance are the average time a customer has to wait before being handled, the average ticket resolution time, total ticket volume and the percentage of tickets that were successfully resolved, and more.

    The performance of voice bots can additionally be measured using metrics such as the average call time, the percentage of abandoned calls and the average time until abandonment – which is defined as the customer hanging up without their issue being resolved.
  4. Now that you have a proper dataset and you know what exactly you want to achieve with your chatbot, it’s time to take a look at your natural language understanding (NLU) engine. Optimising your NLU engine for your specific use case is a necessity – the bot should be able to recognise the various keywords and phrases used within your specific niche.

    Teaching the bot to recognise more than just patterns can further improve your intent detection accuracy and vastly improve your users’ experience with the bot.
  5. Let’s face it – only the biggest tech companies can roll their own chatbot solution. If you don’t want to devote half of your company’s resources to reinventing the wheel, you’ll likely choose to work with a chatbot provider.

    Your choice should primarily be based on the provider’s fit within your niche. At SentiOne, we’re focused on developing a state of the art NLU for customer service automation. To that end, we have built robust conversational datasets using data gathered from social listening. These datasets can then be fine-tuned to fit a specific business niche, which significantly improves intent classification.


Keep these steps in mind when deciding whether or not you should commit to a chatbot solution. As is the case with many other emerging technologies, it does take a lot of effort and research to truly understand what makes chatbots tick and what’s required for them to be successful – but believe us, it is well worth the effort.