Traditional chatbots were universally dull conversationalists. Early versions relied on rigid, rule-based scripts (user says X, chatbot says Y), making interactions feel robotic and frustrating. But thanks to advancements in artificial intelligence — particularly conversational AI and agentic AI — chatbots have the capability to evolve beyond simple Q&A tools into dynamic, context-aware assistants.
These days, companies are embracing chatbots. The chatbot market is expected to reach $7B USD this year, and grow to nearly $21B by 2029. Applications like WhatsApp, WeChat, and Facebook Messenger are driving chatbot adoption, enabling businesses to deliver more responsive, data-driven customer interactions.
But even with these advances, most chatbots remain reactive — they answer questions but rarely take action. Enter agentic AI: the next evolution of chatbot intelligence. Unlike traditional AI chatbots that rely on predefined workflows, agentic AI can make decisions, automate multi-step tasks, and proactively assist customers, making digital interactions more seamless and intuitive.
This shift marks a turning point. Chatbots are no longer just answering questions — they’re becoming autonomous digital agents, capable of driving real business impact. But to harness this potential, companies need to rethink their chatbot strategy. Let’s explore how.
Conversational AI & Agentic AI: The Next Generation of Chatbots
Conversational AI has transformed chatbots from simple, rules-based programs into intelligent, human-like assistants. These AI-powered systems use natural language processing (NLP), natural language understanding (NLU), and machine learning to interpret queries and generate more relevant, contextual responses. But while this has been a major step forward, many chatbots still struggle with context retention, decision-making, and complex problem-solving.
That’s where agentic AI comes in. Unlike traditional conversational AI, which primarily reacts to user inputs, agentic AI can autonomously make decisions, take initiative, and execute multi-step tasks without constant human prompting. Think of it as the difference between a chatbot that simply answers a question and one that proactively solves an issue — for example, troubleshooting an account problem.
However, implementing effective, secure, and reliable AI-powered chatbots is still a challenge for many organizations. Data fragmentation, lack of contextual awareness, and reliance on outdated rules-based systems often prevent chatbots from reaching their full potential.
Ethan Mollack, a professor at the Wharton School of the University of Pennsylvania, believes these barriers won’t last forever. He writes:
“Many things that once seemed exclusively human will be able to be done by AI. By embracing this principle, businesses can view AI’s limitations as transient, ensuring they stay competitive in a fast-paced digital world.”
With conversational AI and agentic AI working together, chatbots are moving beyond simple back-and-forth interactions. They’re evolving into autonomous digital agents, capable of delivering smarter, more proactive, and frictionless customer experiences. But the key to success lies in how well companies integrate high-quality, unified data into their chatbot systems — because even the most advanced AI is only as good as the information it has access to.
What’s Wrong With Chatbot Technology Today
What marketers and organizations are promising, and the reality of what an AI chatbot can realistically deliver, are two very different things.
As AI-driven chatbots and chatbot conversations increasingly become part of how companies interact with customers and deliver proactive customer service, it’s important to keep the following in mind:
Emotionally intelligent chatbots don’t exist yet
Chatbot technology is part of the broader AI story because it’s so tangible. People ask chatbots like ChatGPT or Google Assistant a question and the bot responds with a cheerful, “Sure! I can help with that.” But an emotionally aware chatbot that understands the nuance of human emotion, can interpret emotional cues, and then generate the right type of empathic AI response?
Well, we’re not there yet.
It’s true that machine learning and natural language processing help chatbots understand and interpret human language. This is what makes them better at mimicking real human interaction and going beyond the “user says x, chatbot responds y” functionality. But although AI driven chatbots can respond to customer queries with what seems like real human emotion, they aren’t yet nailing human conversation the way we need them to. An AI bot still struggles with emotional responses and empathy.
That said, companies like Hume AI are working on building emotional intelligence with technologies like empathic voice interface (EVI), sentiment analysis, facial recognition (to better understand facial expressions), and emotional tone recognition. All of this works to improve chatbot response and empower more appropriate emotional artificial intelligence.
Many bots still lack contextual awareness
Customers expect your AI assistant or chatbot to be as wise and all-knowing as Google, but this can only happen when chatbot solutions have access to all your data repositories so that it can connect users to the right information or answer their questions.
Seamless integration of enterprise data is needed for a chatbot to fully understand user context and solve customer problems. Without this, chatbot solutions have limited ability to answer questions or provide the same level of customer care as a human agent.
And while some chatbots can connect to multiple data silos, complex connections and data governance pose hurdles. Thus, even in 2024, chatbots often fall short of the seamless, context-aware experience that existing customers and new customers expect.
Relevant reading: Why Unified Search Is a Must-Have for AI Innovation
They don’t work in a vacuum
If your data is not accessible because, for example, it’s stored in a system that’s not connected to your main data ecosystem or it’s offline, then your AI agent can’t access it. This goes hand-in-hand with the point about contextual knowledge. AI bots often suffer from “death by data silo” where their access to data is limited.
If a chatbot is “chatting with” a customer, they not only need to access the contextual data of their customer but also have access to every place where the answer to the customer’s question may reside.
Product documentation sites, customer communities, and different websites are all places where that answer can be. Exceptional customer service — and customer success — rely on access to this needed information.
They miss the mark when it comes to user experience
The bottom line: chatbots frustrate your customers if you are viewing them as a replacement for your customer support team. Be transparent about AI use. Program your bot to recognize its limits and smoothly hand off to a human agent when needed.
If your chatbot suffers from any of the issues above, you’re probably creating a poor customer experience overall and an angry phone call to a poor unsuspecting call center rep.
The real value in AI chatbots is in their potential to improve customer service and provide unique answers to customer queries without the limits of rules-based workflows. Generative answering, agentic AI, and conversational AI are converging with another important feature – unified search.
Where Most Companies Go Wrong with Chatbots
What will a lot of companies regret a year from now when they review their chatbot project? For the most part, each of these recommendations comes down to not setting clear expectations and goals when discussing AI chatbot development with your team.
Chatbots are not a panacea and will not cure all of your customer experiences issues. If you’re looking for that, you need to examine every customer interaction and undertake a top-to-bottom overhaul of your customer journey. A chatbot is only one part of that customer journey, but can become a crucial touchpoint to increase the overall satisfaction of your customers and their likelihood to be loyal to your brand.
Just don’t make these mistakes:
#1 No defined chatbot purpose or personality
Having no chatbot is better than having a poorly functioning chatbot. In a Gartner survey, less than 10% of consumers said they’d used a chatbot for their most recent customer service experience. Voice bot, voice assistant, generative AI chatbot – whatever the tool, without a clearly defined purpose, your chatbot risks doing more harm than good when it comes to customer satisfaction.
Chatbots also have the potential to be really helpful, provided that your customers have realistic expectations of their capabilities, are aware of what the limitations are, and when they’ll be carried over properly to a live agent.
Even if your customers know they’re engaging with a chatbot, they still want a human-like experience. Look at your current customer service interaction to define what conversation flows look like, branching, etc. The biggest mistake would be to treat the project as a technical one rather than a customer-centric one focused on meeting customer needs.
Pro tip: think “conversational interface” not “bot”.
Relevant reading: How Conversational AI Humanizes Chatbot Personality
#2 Expectations set too high
Future chatbots may be able to do everything for your customers, from solving complex tasks to taking orders to coordinating delivery times. But present AI technology is not quite there yet.
For now, and for your first chatbot, keep your expectations limited to conversational queries grounded in knowledge. Shopping assistants, for example, can provide needed answers about specific products, company policies (e.g., shipping, returns) and act as a virtual sales associate.
Amazon’s generative AI-powered shopping assistant, Rufus, launched in 2024, is a great example of the evolving capabilities of chatbots in retail. Rufus, available to U.S. customers in the Amazon app, answers questions as customers shop.
Rajiv Mehta, VP of Amazon search and conversational AI shopping, told Digital Commerce 360 that customers have already asked Rufus tens of millions of questions. Mehta reports that customers are using Rufus to better understand product details, read customer feedback, receive recommendations, compare options, and stay updated about new products.
Relevant reading: The (Realistic) Impact of GenAI on Ecommerce
Relevant reading: Debunking 10 Generative AI Myths: What Enterprises Need to Know
#3 Treating chatbots as a separate customer interaction
For your customers, the interactions with your chatbot, website, call center, mobile app and more are one continuous experience that they have with your brand.
If you’re still viewing each of these interactions as separate and not sharing the knowledge and insights gained between these channels, you’re losing the contextual awareness each of these channels need to provide a more personalized experience.
Your customer has recently purchased a product, is viewing page after page trying to resolve an issue with that same product, and now opens up a chat. If your chatbot doesn’t immediately understand that they are most likely resolving issues related to that purchase, and personalize the results to their questions based on their online behavior, you’re creating more friction in your customer experience and frustrating your customer.
Relevant reading: Create Effective Customer Journey Map (+ Free Template)
#4 No enterprise chatbot roadmap
What’s the long-term view for your chatbot? How will it grow within your customer experience? If you’re just doing this as a one-off project to solve an immediate issue, don’t waste your money. You’re talking about investing a lot of resources and that investment needs to align with your strategic priorities and goals.
Focus on building a roadmap with small, measurable milestones that translate to quick wins for your customer experience strategy. Be realistic about how the technology is going to mature and where you need it to go for your company.
#5 Not using the insights gained from your chatbot
This is the most common issue that we see with AI powered chatbot investments. This technology is a goldmine of customer inquiries, knowledge, and insights; where else are your customers directly asking for what they need and voicing their concerns? If you don’t have a plan in place to use every single insight to make your products and experiences better, you need to re-evaluate why you are creating a chatbot in the first place.
As you start to see the questions that your chatbot cannot answer, or the ones that generate more questions from the customer, you can use that data to create a content strategy to address those “content gaps” on your website to create a more seamless experience.
Relevant reading: Search Analytics: Know What Works and Why It Converts
The Future of Chatbot Technology
Voice bots, virtual assistants, conversational AI chatbots — whatever chatbot usage looks like for your company, the technology is poised to connect to everything, from websites to CRMs to your company’s intranet and Slack channels. Rather than going through different websites and speaking to various different chatbots, the virtual assistant will be the platform for finding the answers you need.
In our 2024 Commerce Industry Report, 70% of 4,000 consumers surveyed said they’re expecting generative AI to enhance their shopping experience. All roads point to a future of “conversational commerce” where consumers chat with AI-powered agents as they move through their digital shopping journey.
But as technology advances and companies like Walmart, Amazon, eBay, and Ikea test the limits of AI-powered assistants and bots, there are still some roadblocks to navigate. Consumers remain wary of these tools, preferring to speak with human agents over robotic assistants, particularly when resolving complex issues or questions. Companies may face legal consequences or harm their reputation if the chatbot provides faulty or incorrect information.
Oh and by the way: We’ve been hard at work on some interesting projects at Coveo, one of those focusing squarely on the world of chatbots. We’ve leveraged our insight engine, and enabled it to work within the confines of your preferred chat tool: the power of Coveo, in chatbot form.
Coveo’s latest offering is a chatbot that uses advanced AI to improve user experience. Our intelligent chatbots use your company and customer data to provide reliable, accurate, and safe information grounded in user context and your own knowledge base.
Don’t let your chatbot be a frustration point. Get AI-powered accuracy today.
Dig Deeper
As exciting as chatbots are, they’re not the only technology that can help take your customer self-service to the next level. Check out our free ebook, 6 Proven Techniques for Customer Self Service Success, and learn how you can reduce contact center case volume and increase customer satisfaction.