I confess, developing search engines is one of my favorite technology initiatives. I can totally relate to chief technology officers, product managers, architects, software developers, and test engineers that see the opportunity to develop an amazing search experience.
I spent a decade of my career building search engines for newspapers, magazines, travel websites, data providers, market research companies, B2B financial service businesses, and nonprofits. My startups worked with media companies like Hearst and Scripps, and later in my career, I was the CIO at two McGraw Hill companies. Today, my company StarCIO guides transformation leaders on digital transformation, and we work with development teams on agile, data, and low-code centers of excellence.
Here’s how we developed search engines back then.
How to Enable a DIY Search Experience
We bought a search indexing technology and went to work. Loading the data from several sources only took a couple of development sprints, but cleansing the data and iteratively improving the search taxonomy required multiple releases.
We debated how to query the search index using all the available facets and search terms, only to realize that it needed ongoing tweaking to answer all the stakeholder input around relevancy. Even then, one query became a rules-based algorithm aiming to avoid null result sets and to narrow down others that returned too many results.
I admit that even though we had a team, and sometimes multiple teams, dedicated to improving search, the business outcomes and end-user experience improvements came slowly. Every sprint and release had development work dedicated to tweaking the algorithms, improving data loading reliability, increasing performance, or addressing a scalability bottleneck.
Why DIY Search Falls Short
And unfortunately, we barely scratched the surface on personalization, A/B experimentation, recommendation engines, and other machine learning capabilities. Few technology capabilities are more important than search when providing a great digital user experience to customers, employees, and supply partners. Yet, as noted above, implementing search is often ripe with complexities, from loading in the data, developing the search algorithm, ranking the results, and optimizing the end-user experience.
And search engines have a lot more capabilities today to help architects, software developers, product managers, and user experience designers to create experiences that delight end-users.
Customers, employees, and other end-users have high expectations when entering queries into a search box, clicking on dynamic filters, receiving personalized emails, or getting alerts on their phones. To implement a world-class search capability today, agile development teams must consider faster, easier, and enhanceable implementations.
What Capabilities Are Needed for Minimally Viable Search?
The search capabilities that ecommerce and media companies pioneered over the last two decades are now common features applied in many customer and employee-facing applications. For example, many businesses are focusing on the future of work and looking to improve employee experiences, especially now that many companies are shifting to hybrid work.
Financial services companies are looking to provide customer self-service capabilities, manufacturers are establishing new digital sales channels, and health insurance companies are looking to provide the most relevant information to people seeking medical attention.
What, then, are some of the search capabilities that enable today’s search experiences? Here is my list of what should be in a minimally viable search:
- Dynamic facets and filtering to aid end-users who are learning and browsing
- Natural language query search boxes to help people ask detailed questions
- Machine learning-enabled re-ranking and personalization to present the most relevant results
- Recommendation engines that provide similar results based on end-user behaviors and popular results from related searches
- Tools that empower subject matter experts to define customer segments and perform A/B and other experiments to optimize search results
And those are just some of the capabilities enabling the end-user experiences.
To deliver customer-facing search experiences, development teams also need efficient ways to integrate content from multiple sources (i.e., unified search), including content management systems, CRMs, learning management platforms, social networks, and ecommerce platforms.
For employee experiences, add in all the primary knowledge repositories, including file systems like OneDrive and Dropbox, community tools like SharePoint and Slack, and enterprise platforms like Jira, Salesforce, and ServiceNow.
And let’s not forget the non-functional criteria critical to SEO and customer satisfaction, including query performance, data loading reliability, cloud agility, and information security.
Today’s Search: Powered by Low-code, Pro-code, Self-Service, Machine Learning, and Integration
To accomplish all this, technology teams must select smarter, faster, and more innovative ways to plan, implement, establish feedback loops, and enhance search capabilities. Delivering world-class search functionality requires a top-down plan and architecture focusing on business outcomes, user experiences, and collaborative practices.
Technology leaders accomplish this by selecting search partners that handle the many lower-level details. Because if tech teams elect to develop experiences on top of a low-level search index, they own many underlying infrastructure, data plumbing, and query tuning challenges.
Here are five things I look for in today’s search experience platforms:
- Provide out-of-the-box integrations with common SaaS and enterprise platforms so that dataops is less of an issue.
- Enable self-service and low-code UX capabilities so that subject matter experts are more hands-on developing experiences for different customer segments.
- Support a pro-code headless library to customize search experiences and also have JavaScript and API development options.
- Deliver robust machine learning capabilities so that technologists apply best practices to improve user experiences and outcomes instead of investing efforts in building models and tuning algorithms.
- Manage the cloud infrastructure for high-performance, reliable, secure, and data-compliant search experiences so that the ops and security teams can sleep at night.
By letting the platform handle many lower-level technical details, agile teams that include business, product, technology, and data science leaders can focus on the user experience and business outcomes.
Back in the day, I was one of the heroes managing all the complexities in developing the search engine. Today, the challenge is up-stacked, enabling digital transformation champions to select search platforms to handle the plumbing and then guide their businesses with differentiating search experiences.
Dig Deeper
Wondering where to get started when looking to evaluate different enterprise search platform options on the market? Check out our Buyers Guide for Enterprise Search Platforms for the inside scoop.