Why Enterprise Search is so Difficult

Oct 16, 2024

11

min read

Workplace search is seeing rapid growth.

Mainstream adoption of Large Language Models has brought about a wave of productivity tools that address the challenges of enterprise knowledge management. At a basic level, these platforms enable a business to search an index of their own files and data.

The more recent addition of chatbots has expanded search to include analysis and answer generation. This combination of search and Generative AI chat is commonly known as RAG (Retrieval Augmented Generation).

But despite a ton of investment and effort, we have yet to see mass adoption of search at a business level. Have you personally used an enterprise search tool today? Maybe not. But chances are you've used Google at least once.

We often get asked by business leaders why does it seem so hard to adopt this tech? The answer is a mix of technical challenges and public perception of search.

Key Takeaways

• LLMs have breathed new life into search software, especially for B2B use.
• Technical hurdles include data governance, inconsistent search quality, and an overwhelming amount of bad data.
• For business leaders deploying this tech internally, focusing on specific pain points and narrowing the use cases can greatly accelerate adoption.

Search Sounds Easy

The "Google for B2B" trope isn't new.

It has been pitched for years with buyers being told that all you need are a few key ingredients:

Language Model + Database + Data Connectors + User Interface = Enterprise search

As it turns out, getting search to run well is harder than people think. And it's not just the nerdy stuff. There's also the parts about selling, marketing, supporting, securing, maintaining, and building an actual business around the tech.

Technical Challenges

There's too much data

Good search requires clean, structured data. This is why Google mandates that webpages meet stringent requirements around machine readability. It must be easy for a web crawler to understand the data before it can retrieve anything.

The problem here is that within (most) organizations there's too much data at any given moment. Most of it is very poorly structured. Some of it is completely irrelevant.

This creates a roadblock for workplace search tools. Don't believe me? Just take a look at the G2 reviews for business search tools. A common complaint is that it's difficult to find what you're looking for.

This is where the "Garbage in = Garbage out" saying makes total sense. You either need to clean up your data or be highly selective about what goes into your search index.

3rd party connectors create headaches

Business data is scattered. That's just life.

Think of how much work-related information you have sitting on your desktop, Google Drive, email inbox, Sharepoint, Slack, or any file storage system.

Search sits on top of these existing systems. That means a search tool needs to ingest the data, keep it synced, and ensure that it's always pulling quality information. Reading all of this material requires 3rd party integrations that can introduce new technical pains.

Constantly building and maintaining connectors. Keeping a team dedicated to maintaining these integrations. Accounting for changes made to the connectors. The list goes on and on.

Data governance is painful

Controlling access to files is hard enough. Introducing AI into the mix has opened a new can of worms for businesses. Just scroll the CIO subreddit and you'll find no shortage of complaints around AI data governance.

We're not just talking about controlling access to files. LLM search systems have introduced a new challenge of controlling for answers sourced from outdated content, sensitive documents, and even opinions generated by an AI.

Imagine sensitive financial information is ingested by your search tool. A user conducts a search that erroneously answers their question using this document.

Who is to blame?

• The AI search tool
• The user looking for this information
• The administrator who failed to remove this outdated document

No one is to blame. This is a bi-product of using enterprise search without strict governance controls. In a search context, the sensitive data is not just exposed to a group of users. It's made available to anyone interacting with the tool.

Good search quality is a moving target

This is the number 1 challenge of automated knowledge management. Good results are subjective. They may vary from company to company, and even at the department level.

There are many different approaches to search. The two most often discussed are:

• Dense vector search focuses on semantic meaning between words. This approach creates a modern search experience where users can lookup large bodies of text. It can also lead to unsatisfying results when users are looking for specific terms.
• Sparse vector search relies more heavily on keywords using exact term matches. Think of how Google worked in the early 2000s, when keyword frequency was still a ranking factor. It's great for keyword search but can't support user behaviors search as phrase search.

As a business, your employees could be searching for 3-letter terms or for long complex phrases. Which search method will yield the best results?

There's also the always present problem of garbage in/garbage out. Results depend heavily on user-generated content, so you're fighting a constant battle to keep good data in the system.

One example of this is when searching a company chat in Slack or MS Teams. You would think that these message logs are a treasure trove of quality data. Sometimes they are. But the amount of noise that you find in these data sets can introduce incorrect answers, unintended opinions, or private conversations that you never wanted to be searchable in the first place.

Business Hurdles

High costs = barrier for smaller businesses

Great workplace search platforms come at a high cost. Tools like Coveo and Elastic are complex systems with deeply technical teams. Given the problem space, it's no surprise that they focus on big businesses as their target audience.

There's a "so what" mindset

Is workplace search a real problem for most businesses?

On paper, of course it is. But speak with business leaders and you'll hear mixed opinions. Urgency varies at a department level.

Some folks feel like the need to make everything connected and searchable is a nice-to-have for their organization. In rare cases you might find that mature internal tooling has enabled a team to search for knowledge effectively.

Our take is that the true benefit of enterprise search lies in connecting these disparate sources and allowing for cross-department collaboration. How much of a problem that is remains up to the organization and can vary based on stage, size, and maturity.

High bar for implementation

There can be a lot of effort involved in getting a search system off the ground.

Take a 1,000 person organization as an example. There could be upwards of 50,000 files to index across multiple departments.

Your system needs to pull from all sources, index the data, make it searchable, handle the complexities of data governance, and finally – get people to adopt it. This is a high bar for setup which can take multiple departments weeks or even months to complete.

User Habits Delay Adoption

This is the part no one likes to discuss because you can't really force it.

Users who are satisfied with their current approach to retrieving knowledge will have no incentive to use a new system, even if it's noticeably better.

Where Workplace Search is Seeing Success

Despite the technical and business challenges, enterprise search is a $12B TAM that's growing rapidly.

That's because the underlying pain is very real. There is urgency to solve it. Here's where businesses are seeing immediate value from these tools:

Knowledge Management

This is the traditional use case where teams are trying to make knowledge accessible internally.

Questionnaire Automation

This is becoming more recent and more popular among sales, IT, security, and compliance teams. An automated questionnaire is really just a series of search + answer loops.

Document Analysis

Using AI to search and analyze documents is a huge use case.

Customer Support

This is probably the most obvious use case for search.

Search Infrastructure

Beyond the application layer there are a number of successful infrastructure tools such as Algolia, Coveo, and Elastic.

Sales Enablement

Enablement is a broad category of sales tools, but in the context of knowledge retrieval we're seeing modern search tools displace legacy players.

Narrowing Focus to Maximize Adoption

Going "Narrow" on a problem set is a faster path to adoption and clear ROI.

Conclusion

There are many reasons to be optimistic about the future of search within businesses. We think focusing will go a long way for businesses adopting this technology.