Why Your AI Shopping Assistant Is Only as Good as Your Search

Think of an AI shopping assistant as a fluent intern who only knows what you hand them. Retrieval is the handing. Whatever search puts in front of the model is what the model talks about, and when search hands over the wrong products, the intern sounds just as sure.

How an assistant uses search

The assistant does not look through your catalog. It sends the shopper's question to a retrieval step, which returns a short list of products. The model then writes its answer from that list. This pattern is called retrieval-augmented generation, and it is how most grounded shopping agents work, including the reference design we follow.

Redis's engineering team calls retrieval failure the most common accuracy problem in these systems: the right document exists, but the system does not surface it (Redis, a database vendor, in its RAG accuracy guide). Once retrieval misses, the model has two choices. It can say it found nothing, or it can reach for something nearby. Fluent models often reach.

Plate 1 · Where search breaks an assistantFive stages, four failure points

Four ways search fails an agent

1. The question is misread

Shoppers type “runing shoes”, “kettle induction” and “something for a wet hike”. A system without typo tolerance, synonyms or any sense of intent returns nothing useful, and the agent has nothing to work with.

2. The right product is never retrieved

Vector search captures conceptual similarity well, but it can miss documents where exact terms matter, such as part numbers, codes and proper nouns. Keyword matching has the opposite problem: it misses related content that uses different words than the query (also per Redis). In a store, that means vector-only retrieval can stumble on a part number like V10-1234, while keyword-only retrieval can miss “warm jacket for a wet hike” because no product says exactly that.

3. The right product is ranked too low

The agent reads the first few results. A product at position 9 might as well not exist. Redis notes that documents ranked 8th or 9th can be more relevant than those ranked 2nd or 3rd, and that models pay more attention to context near the start of their input. A reranking pass fixes this, and Redis says mature implementations report 10 to 40 percent precision improvements, depending on the baseline.

4. The filters let the wrong thing through

Sold-out items, discontinued variants and products that do not ship to the shopper can all come back from search. An agent that trusts the list will recommend them. This one overlaps with product data, which is why the data audit and the search check belong together.

Our own illustrative specimen shows the pattern. A query for “tea kettle induction” drops the induction-compatible kettle out of the top five on a replacement engine. A shopper sees four reasonable kettles. An agent builds an answer from them and never mentions the one the shopper wanted. See how we score that kind of miss.

Hybrid retrieval, in plain terms

Hybrid retrieval runs keyword and vector search together and merges the results, often followed by a reranking step. The point is that each approach covers the other's blind spot.

Plate 2 · Keyword, vector and hybridWhat each handles well

Published results back the idea, with a caveat. Research on hybrid RAG systems found that combining keyword and vector search improved retrieval recall by 3 to 3.5 times and raised end-to-end answer accuracy by 11 to 15 percent on complex reasoning tasks. Microsoft's Azure AI Search benchmarks found hybrid retrieval improved NDCG scores by roughly 10 to 20 percent over single-mode approaches (both as reported by Redis). Those are general text-retrieval results, not shopping catalogs, so do not expect the same numbers on yours. They explain why hybrid is the first thing to try.

Test retrieval before you test the agent

If you test the assistant first, every failure looks like a model problem. Test retrieval on its own and you will see how many are not.

1. Pull 100 real queries from your search logs. Keep typos, SKUs and long descriptive questions.
2. Write an answer key for 30 of them. Which products should a good result contain?
3. Measure three things: how often an expected product appears in the top five, the share of queries that return nothing, and whether filters behave (stock, size, price).
4. Sort the misses into typo, SKU, concept and filter failures.
5. Fix the biggest group and rerun the same queries.

This is the replay method our Search Lab was built around, and it is the first step of the Agent Snapshot.

What to fix, in order

Work upstream first, because every downstream fix is wasted if the data above it is broken. Start with product data quality. Then add typo handling and synonyms, then hybrid retrieval, then reranking. Tune the assistant's prompts last. Teams that begin with prompts spend weeks polishing the intern's wording while the briefing folder is wrong.

Does an AI assistant replace site search?

No. Keyword search stays the fastest path for a shopper who knows exactly what they want, such as a model number and a size. An assistant earns its place on open-ended asks like “a lightweight sweater for a beach vacation”, where search returns a thousand products and no guidance. The assistant also depends on search, so a better engine helps both.

Common questions

Do I need a separate vector database for an AI shopping assistant?

Often no. Many search engines can store vectors alongside keyword indexes, so check yours before adding infrastructure. A Diagnostic confirms what fits your stack.

Can the model just read my whole catalog?

Not at any realistic size. A catalog of tens of thousands of products does not fit in a prompt, and long inputs raise the cost of every turn. Retrieval narrows the catalog to a few relevant products first.

How do I know if the problem is search or the model?

Test retrieval separately. If the right products are not in the top results for a question, the model never had a chance. If they are there and the answer is still wrong, look at the prompt, the policies and the answer checks.

What is a good top-five hit rate?

There is no universal number. Measure your current search on your own real queries, then require that any change, including adding an assistant, does not make that baseline worse.