Reading Gartner Magic Quadrant for Insight Engines1 to wrap your brain around what’s best in enterprise search?
Data-driven decision-making is a must in today’s business world. Yet the information needed to make those decisions is all too often locked up in silos.
A potential solution is an insight engine, which can centralize content and help users reach the answers they need. For employees, this means a single interface to search instead of four or more content repositories. Ultimately saving them from wasting half the day just trying to dig up the information they need to do their jobs.
For customers, providing relevant content can increase your website conversion rate by as much as 55%. For ecommerce companies, it can increase the average revenue per visit by 25%.
Let’s dig into what exactly an insight engine is, and what features to look for to choose the right fit for your business.
What Is The Core Function Of An Insight Engine?
An insight engine combines keyword search, data connectors, UI components, and machine learning to provide customers and employees alike with more relevant information and personalized experiences.
Where a database tries to structure data to answer a few select questions, an insight engine uses structured and unstructured data to answer an almost infinite number of questions.
An insight engine is not just “search” but search, data, and intelligence. The difference between a search engine and an insight engine is the difference between a wheelbarrow and a self-driving electric pickup truck.
And now that you know what an insight engine does, let’s talk about the elements that the best insight engine should have.
12 Features Of A Good Enterprise Insight Engine
1. Connectors And Text-Preprocessing
It’s 10 p.m. – do you know where your data source is?
Most organizations have data everywhere, in many different systems and many different formats. Loading and transforming data might not be the most attractive feature of an insight engine, but it’s a critical foundation.
If you can’t get data from your systems, you can’t get the answers you need from your insight engine.
Choose an insight engine with pre-built integrations that can seamlessly connect to the content sources of your choice. Coveo has 50+ to choose from.
2. One Unified Index
Think about the last time you spoke with a large healthcare or service provider.
They often put you on hold while they look something up or transfer you to a different person. Often, their websites aren’t navigable or searchable because the information you need is in many places.
Better, more relevant answers require a unified index – that is, centralizing and unifying data in one index.
Entropy is the nature of data. Data disperses and becomes less coherent. While most search is contextual, supporting cognitive search requires looking in multiple places, including answers from data in multiple formats.
3. Instant Results: Customers And Employees Can’t Wait!
When was the last time that you went to a website, used the search bar or any interface component, saw it spin, and you waited?
Oh, you left?
Technology users don’t wait either. Your competition is just a browser tab away, giving your customers plenty of alternative options. Moreover, along with being hard to measure, the cost of digital workers’ wait time drains dollars and cents out of a company.
In a modern business, answers should be both instant and correct.
An insight engine combines search results into a single index and uses machine learning to suggest queries to users as they type. It does not require going back to the source system to provide instant gratification.
4. Facets And Filters
Facets and filters may not be as new or shiny as artificial intelligence, but any insight engine should allow searching, sorting, and filtering on categorical data called “facets.”
Filtering on a facet is how digital workers limit their search to a particular domain (i.e., Human Resources Documents), or a customer searches on a specific product category (i.e., Shoes) or customer support topic (i.e., Coveo Headless Tutorial).
Most searches are pretty straightforward, but your most dedicated employees and customers likely dig deeper, and these tools make their life simpler by offering them relevant results.
5. ‘Know What I Mean?’ NLP And Intent Detection
For a person or an insight engine to answer a question, they need to know what the question is and what the person means by the query they type into a box.
Natural language processing (NLP) goes beyond mere keyword matching and derives actual meaning from the question. NLP finds synonyms and matches phrases.
People don’t always say what they mean or ask complete questions. According to Coveo customers, most searches are two words or less.
A modern insight engine needs to not only understand the language or just match keywords but understand user intent based on user behavior.
6. Automatic Relevance Tuning: Art Of Relevance
Perfect results may not come from matching what users asked for with the data at hand. An insight engine is smart enough to notice when users aren’t getting the best search result.
Tuning a query isn’t even a one-time fix – the meaning of a search may change over time. For example, consider a general ecommerce retailer. In October 2019, the search query “mask” should return a different result than it did by March 2020 at the onset of the COVID-19 pandemic.
Modeling, interpreting, and tuning to user behavior must be automatic in the modern world. The days of poring over logs and getting user feedback to tune results are over. In most cases, it would take too many users looking at too much data on too massive of a scale to make this possible. An insight engine should see this behavior and adjust itself.
Learn more about Automatic Relevance Tuning (ART).
7. Rules
“No one knows how to spell HIPAA,” said Eric Immerman, practice director for Perficient, while working with Humana. While an insight engine might eventually notice this and tune itself, sometimes users are creative with spelling.
Moreover, even the best AI algorithm requires a lot of big data before it learns. Sometimes you only need a simple rule to tell the insight engine, “when they say ‘HIPPA’ or ‘HIPPO,’ they mean ‘HIPAA.’” Rules have a lot of usages beyond spelling.
Sometimes you want a higher margin product or this year’s new employee benefits manual near the top even though something else had more clicks.
See how Coveo manages ranking expression rules.
8. Question Answering And Chatbots
In the early days of the net, every question was answered with a search engine result page (or SERP). These days, if you ask Google, Siri, or Alexa any number of questions, you get a direct answer.
If you type “What is the weather” or “Who was Milton Freedman,” you get a direct answer. Insight engines use a combination of NLP and other techniques to search results AND actual answers.
Question answering technology is useful for everything from giving employees answers in an enterprise search application (like smart snippets) to handling support requests.
However, not all questions are merely a question and an answer. These days chatbots can answer questions that have “branches” or need more context.
For instance, if you ask, “Where can I download a driver for my webcam?” it might ask for your model number and which computer you have. If you have questions about what your health insurance covers, maybe it asks which policy you have or where you live.
An intelligent chatbot guides a user to answer beyond what search can provide.
9. UX Components And Tools
Most search applications have search bars, query suggestions (sometimes incorrectly referred to as type-ahead, which is different), and a slew of user interface and user experience traits.
Redeveloping them for every application is wasteful, which is why an insight engine provides prewritten components that can be composed using JavaScript or TypeScript.
Get started with Javascript Search Framework.
10. Deployment Flexibility Across Clouds
The world is moving to the public cloud. But not everything lives there yet, and some information requires more careful handling. An insight engine is only useful if it can handle data from both public and private clouds.
A cloud-based insight engine that adheres to robust enterprise security protocols, provides the flexibility to connect and index content both within and beyond the organization’s internal data sources.
Needing to index and unify content that lives on an arcane internal file server, SharePoint, and Google Drive is typical for most enterprises.
11. Security
Most businesses have a team of infosec professionals trying to keep employees as well as customer data safe and systems secure.
However, your security is only as good as your weakest link. An insight engine must preserve ownership and permission information. It must encrypt data in transit and at rest.
To maximize the value from an insight engine, you need the confidence that comes from knowing a third party has audited and assessed vendor practices and procedures as well as the software itself.
See how Coveo keeps interactions secure and businesses protected.
12. Contextual Personalization
Being responsive to user requests and giving them what they want, is the base level of search. Insight engines go much further: suggesting what other content will be helpful to them – despite not being asked. We call those “recommendations” – and anticipate what visitors will need next.
You need to understand the context of what people are searching for, what they have already done, and what others like them have already found useful.
Doing this provides the basis for personalizing relevant results and recommendations that meet their immediate needs, and guides them to what they are going to need next. This is all delivered using specific machine learning models for each purpose.
You also need analytics that provide actionable insight into what your users are searching for. This tells you what they’re finding, as well as what’s working and what’s not – which queries are resulting in content gaps? Which turns them into content opportunities for you.
Beyond The Best Insight Engine
These are really the minimum for a modern insight engine. You can get somewhere on a bus, but if you can take a high-speed electric train and get there faster and cheaper, why would you go the more challenging route?
While modern software can be complex, configuration and administration shouldn’t be.
Ease of configuration and administration allows you to better use, tune, and get better results from your insight engine. It also saves time and money. If DevOps professionals have to jump through hoops, there is a cost.
Simple configuration and administration are also security features. While bugs can happen in any software, a quick perusal of the news reveals that many if not most major security breaches have had something to do with poor configuration and how hard it is to administer the software.
Simple configuration is a great feature. All of these insight engine tools are helpful, but life, search, and artificial intelligence are not standing still.
Ideally, the team developing your insight engine is always thinking about making you and your customers’ life easier. Ideally, there is ever more R&D going into AI, chatbots, and implementation practices.
Learn more about the latest developments from Coveo’s AI Labs.
Insight Engines Are Where Intelligence Meets Search
While full-text search is at the core, as you can tell, an insight engine contains a lot more than just a search engine. These 12 features might be essential aspects of any insight engine, but Coveo goes much further.
For Coveo, developing an insight engine is about doing everything necessary to make people’s lives easier, whether competing in the marketplace, searching for employee benefits, elevating the customer experience, or the security or DevOps professionals responsible for tuning and keeping it all working.
No matter what your interests, Coveo is here to help make your digital experience a great one.
Learn more about how intelligent search and enterprise search are finding the answers customers and employees need.
1-Gartner, Magic Quadrant for Insight Engines, Stephen Emmott, Anthony Mullen, 17 March 2021.
All statements in this report attributable to Gartner represent Coveo’s interpretation of data, research opinion or viewpoints published as part of a syndicated subscription service by Gartner, Inc., and have not been reviewed by Gartner. Each Gartner publication speaks as of its original publication date (and not as of the date of this report). The opinions expressed in Gartner publications are not representations of fact and are subject to change without notice.
GARTNER is a registered trademark and service mark of Gartner, Inc. and/or its affiliates in the U.S. or internationally, and is used herein with permission. All rights reserved.