How search works: Advanced

The goal of search is to get readers to relevant content. There are many things happening behind the scenes to help make this happen. As much as possible, our goal is to provide a search that works really well out-of-the-box while also giving you visibility and control over your search results.

The heart of any search engine is the index, which is where (and how) your content is stored for retrieval. KnowledgeOwl automatically indexes the most important fields of your content for our keyword search:

• Title (also indexed by Semantic search)
• Permalink
• Body (also indexed by Semantic search)
• PDFs
• Meta description
• Search phrases

You don't need to add tags or keywords for search to work. Any words you use in the indexable fields will automatically be searchable.

Here's a quick overview of what is and isn't indexed for search:

Control your keyword search scores and results through a few different methods. Refer to Optimize search for a detailed walkthrough of the strategies we suggest!

The search index is where we store your article content and metadata after we process and optimize it for searching.

When you save an article, we automatically digest and analyze your content for the index. This process involves:

• Stripping out HTML
• Breaking up the text along word boundaries (tokenization)
• Breaking up tokens into small fragments, called N-grams, for partial matches and autosuggestions
• Removing punctuation
• Lowercasing all tokens
• Stemming all tokens based on your chosen primary search language
• Adding the words from your Synonyms page for matching tokens, if available

Reindexing

The initial index is automatically created when you save an article. If you make a change to your search settings that affects how the data is processed, your knowledge base will need to be reindexed. Reindexing is the process of reanalyzing and redigesting all of your content. 

Changes that require a reindex are:

• Changing the primary search language
• Adding, removing, or editing synonyms
• Turning semantic search on or off

During a reindex, search is disabled. A message will be displayed to indicate that an update is in progress; this message can be customized under Customize > Default text. Reindexing normally takes less than a minute, but larger knowledge bases or knowledge bases with very lengthy articles could take significantly longer.

Stemming is the process of reducing words to their base or root form. For example, "searching", "searches", and "searcher" can all be reduced to the word "search".  This is called the stem. In our keyword search, stemming happens automatically in the background as part of the indexing and reindexing processes.

Stemming helps make search smarter. It's how search engines can find the results you want even when you don't type the exact words. 

Stemming is a core task of Natural Language Processing (NLP). NLP is a branch of artificial intelligence (AI) that focuses on how computers process and analyze natural human language. It is not related to generative AI like ChatGPT.

Effective stemming depends on setting the correct primary search language. Refer to Primary search language for more information.

When a search is performed, relevance algorithms are used to weight the results. The algorithms assign a score to each matching search result, and the search results are ranked by score.

The following are simplified explanations of the types of algorithms used by our keyword search:

1. The exact match 
   All words in the search term must appear in a searchable field for it to be considered a match. This algorithm has no tolerance for typos, but it allows for some variation in proximity of the search terms (words don't have to be in the exact order and can have a few words between them). Any matching result gets a significant boost in score because it exactly matches what a reader typed into the search. You can trigger an exact match search by placing your search phrase "in quotes."
2. The mostly match
   75% of the search term must appear in the searchable fields. This algorithm has no tolerance for typos. Matching results get a slight boost in score.
3. The somewhat match
   50% of the search term must appear in the search fields. This algorithm is tolerant of typos. Matching results receive the lowest boost.
4. The maybe match
   Some of the search terms much match. This algorithm is tolerant of typos. No boost is applied.

In order for an article to appear in search results, it must be considered a match by at least 2 of the algorithms. Scores from all algorithms are combined to determine the article ranking.

The algorithms consider three main factors when calculating scores:

1. Term frequency
   This is how often the search term appears in an article. The more times it appears, the higher the score.
2. Inverse document frequency
   This is how often the search term appears across all your articles. If the search term shows up in many articles, it will have less weight in scoring. If the search term only appears in a few articles, it will result in a higher score.
3. Field length
   This is how long the searchable field is. If it's really short, like an article title, matches will have more weight since it's more likely to be significant.

A note on algorithms

As noted above, search uses several algorithms. Those algorithms are quite complex.

While you can impact your search scores and results by customizing your search weights, optimizing your content, and adding synonyms, the exact calculations and behavior of the algorithms is not something our support team can fully explain.

We added semantic search to our built-in search options in 2025, but it's still a setting you can turn on or off (unlike our original keyword search, which is always on). This page outlines what semantic search is and what it does and doesn't do.

Types of search

First, here's a quick primer on what the different types of search are:

• Semantic search performs some natural language processing to handle typos, synonyms, and other concepts organically without you having to touch any settings, and it will return results based on your reader's intent, even if they don't know the exact words you use in your knowledge base. For example, if you search for something like "How do I log in?", it can surface an article called "Sign in to your account" because it calculates that logging in and signing in are the same thing.
  • Article fields indexed: Title and body
  • Used for: Full search results
• Keyword search directly matches words/stems within your content by searching a large text index for those keywords (or the root/stem of the keywords). You can dial in the Fuzziness (typo tolerance), search fields and weights, and more.
  • Article fields indexed for full search results: Title, body, permalink, summary, search phrases, body, and embedded PDF content
  • Article fields indexed for typeahead search results: Title, permalink, summary, and search phrases
  • Used for: Typeahead search results and full search results

As you've likely guessed from these details, semantic search helps most often with new readers who aren't familiar with your knowledge base or folks who don't quite know how to ask for what they need, whereas keyword tends to really shine for readers who know what to search for. Running both types of search in your knowledge base can give your readers the best possible odds of finding what they want. That is what happens when you enable semantic search in KnowledgeOwl.

What semantic search does

Semantic search allows readers to phrase searches conversationally (for example, "why won't my password reset" instead of guessing the exact article title wording of "How to unlock my account").

It works alongside your existing keyword search, not in place of it, so you don't risk breaking your current search experience.

When semantic search is turned on, here's what you can expect:

• Typeahead search: Typeahead search will still use keyword search, indexing on the article's title, permalink, summary (formerly meta description), and search phrases.
• Full search results: Full search results will use a combination of keyword and semantic search, trying to put the most relevant results first. Keyword search weights will be reduced by a factor of 10 to slightly prioritize semantic search results.
  • If your reader begins their search with a question word like "can," "does," "how," or "why," semantic search results are more heavily prioritized since these are signals that they're searching using natural language rather than keywords.
  • If your reader enters a phrase "in quotes," this triggers an exact keyword match search and bypasses semantic search entirely. Only keyword results for that exact match will be displayed.

What semantic search doesn't do

Turning on semantic search won't turn on the AI Chatbot. You need semantic search enabled for the AI Chatbot to function, but you do not need to turn on the Chatbot in order to use semantic search.

It also won't replace or disable keyword matching, search weights, or synonyms. Those are all settings used by keyword search and it doesn't touch them. Semantic search works in tandem with your existing search settings, adding onto them instead of replacing them.

It doesn't require you to add tags, keywords, or restructure content. (Neither does keyword search, for that matter!)

Semantic search doesn't use an LLM or send your data to any AI provider. We run this technology in our existing infrastructure alongside your existing keyword search. Your data is as secure as it has always been with our keyword search.

How to turn semantic search on or off

Turn semantic search on or off in KB settings > Search (or Search and synonyms) in the General settings tab.

Refer to the instructions in Semantic search for more details.