Agentic Retrieval vs Single-Shot RAG for Ecommerce Search

A shopper types "waterproof trail running shoes under $120 in size 11 wide" into a search box. Four constraints, three of them hard requirements. Most retrieval pipelines built on single-shot RAG will fumble at least one of them: the price cap gets treated as a vague semantic hint, the width vanishes entirely, and the shopper lands on a $185 shoe in size 10. That failure is structural, not a tuning problem. Across 2025 and 2026, Microsoft and AWS both shipped a different answer as managed infrastructure: agentic retrieval, in which a language model plans the query - decomposing it into focused sub-queries, executing them in parallel, and semantically reranking the merged results - rather than firing one embedding lookup and hoping. This piece breaks down how query planning works, what the vendor benchmarks actually claim once the fine print is read, and why the pattern matters most for ecommerce product discovery.

Why Single-Shot RAG Breaks on Multi-Part Queries

Classic RAG runs one retrieval per user turn. The query is embedded, a vector index returns its nearest neighbors, perhaps a keyword score gets blended in, and the top results go to a generator or a results page. This works when the question maps to a single intent that can be expressed as one point in embedding space.

Multi-constraint product queries do not map to one point. "Waterproof trail running shoes under $120 in size 11 wide" contains a semantic concept - waterproof trail running shoe - and three hard constraints that are numeric and categorical, not semantic. A single cosine similarity score blends all of them into one number. The embedding of "under $120" sits uncomfortably close to the embedding of "$185, marked down from $220," and no reranker can fix that, because the information needed to enforce the constraint was never extracted in the first place. Hard constraints have to become structured filters before ranking happens. Extraction is a planning step, and single-shot pipelines have no planner.

The academic literature says the same thing in drier language. A January 2025 survey of agentic RAG systems (Singh et al., arXiv 2501.09136) describes traditional RAG as constrained by static workflows and lacking the adaptability required for multi-step reasoning and complex task management.

That rigidity is expensive commercially, because shoppers rarely give a failed search a second chance. Algolia reports that only 20 percent of shoppers re-attempt a more refined search after a failed query. The other 80 percent leave, and the analytics dashboard logs a zero-results session on what was actually a high-intent, wallet-out visit.

How Agentic Retrieval Plans a Complex Query

Agentic retrieval replaces the single lookup with a plan. In Azure AI Search's implementation, documented in the Azure AI Search documentation, a large language model receives the user's prompt along with the conversation history and decomposes it into smaller, focused sub-queries. The canonical example in Microsoft's docs is commerce-shaped: "find me a hotel near the beach, with airport transportation, and that's within walking distance of vegetarian restaurants." One prompt, three retrieval problems. The planner splits them apart, and because chat history feeds the planner, a follow-up like "under $200 a night" refines the existing plan instead of starting a new search from zero.

Microsoft announced the capability in public preview at Build 2025. As of the 2026-04-01 REST API, parts of it are generally available programmatically, while portal access remains in preview.

AWS followed in November 2025. Amazon OpenSearch Service introduced agentic search in OpenSearch 3.3: a query clause that accepts plain natural language such as "Find red cars under $30,000," hands it to an agent that interprets intent, generates OpenSearch DSL through a built-in QueryPlanningTool, and returns a reasoning summary explaining its choices. Look at what the planner produced there. Not an embedding - a structured query with a numeric filter on price. That is precisely the extraction step single-shot pipelines skip.

Underneath both offerings sits the same research pattern: ReAct (Yao et al., ICLR 2023), which interleaves chain-of-thought reasoning with tool actions and observation of the results. OpenSearch names ReAct explicitly as the foundation of its conversational agents, which iterate until they reach a final answer or hit an iteration limit, and ships plan-execute-reflect agents for longer multi-step workflows on top.

Parallel Execution and Semantic Reranking

Decomposition alone would just make search slower. The second half of the pattern is parallel execution: in Azure's pipeline, all sub-queries from a plan run simultaneously, each one is semantically reranked on its own (L2 reranking, in Microsoft's terminology), and the results merge into a unified grounding payload. The response can include source references and an execution activity log recording each step the planner took, which matters operationally - the log is how teams debug bad plans and audit what each retrieval cost.

Here is how the two architectures compare, stage by stage:

	Traditional single-shot RAG	Agentic retrieval
Retrieval plan	One query, taken literally as typed	An LLM decomposes the prompt plus chat history into focused sub-queries and structured filters
Data recency	One snapshot per turn; follow-ups re-embed the raw question with no memory of context	Fresh sub-queries on every turn against live indexes, informed by conversation state
Execution loop	Retrieve once, generate, done - no feedback path	Plan, execute in parallel, rerank each sub-query, merge; ReAct-style agents iterate until finished
Complexity handling	Every constraint collapses into a single similarity score	Each constraint gets its own sub-query or filter, and merged results respect all of them

Economics change along with the architecture. Billing shifts from per-query to per-token. Microsoft's worked example assumes roughly three sub-queries per plan and up to 50 chunks semantically reranked per sub-query; at that rate, 2,000 retrievals - about 6,000 sub-queries and 150 million reranking tokens - cost roughly $4.32 in total across Azure AI Search and Azure OpenAI. Cheap in absolute terms. The catch is that the meter now runs on plan complexity rather than request count, so a chatty planner that fans out into six sub-queries where three would do quietly multiplies the bill.

What the Vendor Numbers Say - and the Fine Print

Microsoft's headline claim is up to 40 percent better relevance on complex queries versus traditional single-shot RAG, based on internal testing announced with the Build 2025 preview. Engineers should read the qualifiers as carefully as the number. It says "up to." The detailed methodology was never published. And Microsoft's own companion material notes that on classic simple queries, the agentic engine merely matched the standard search API - the gains appeared on complex queries, where they held across all tested languages.

At Ignite in November 2025, Microsoft refreshed the numbers with Foundry IQ, built on the same agentic retrieval engine: an average 20-point (36 percent) improvement in end-to-end RAG answer quality compared with brute-force searching every source at once, plus evidence recall gains of up to 46 percent for knowledge bases over single-shot RAG and up to 54 percent when a smaller agent model does the planning. Every one of those figures is vendor-measured. Independent replication does not exist yet, so buyers should treat them as a directional signal rather than a guarantee.

The latency tradeoff is stated just as plainly. Microsoft's documentation says agentic retrieval adds latency compared with a single-query pipeline, and a Microsoft engineer has noted that overall speed tracks the number of sub-queries the planner generates. Azure accordingly exposes a reasoning-effort dial - minimal, low, medium - and at the minimal setting, LLM query planning is skipped entirely and the query goes straight to the knowledge sources. Read that dial for what it is: the vendor's own admission that not every query deserves a plan.

What This Means for Multi-Constraint Product Matching

Both vendors chose commerce-shaped queries as their marquee examples - a hotel near the beach with airport transport, red cars under $30,000 - and that is no accident. Product discovery is where multi-constraint queries concentrate. A 40,000-SKU catalog does not get searched with single nouns; it gets searched with intents like "gluten-free protein powder under $40 that mixes without a blender," or the size-11-wide trail shoe from the opening paragraph. Each of those needs decomposition into semantic sub-queries plus hard filters, which is exactly the problem of matching shoppers with multi-constraint queries to the precise products that satisfy them.

The routing lesson follows directly from the cost and latency math. Head queries - "running shoes," "black dress," "iphone case" - resolve fine through a fast single-shot path, and Microsoft's own data shows the agentic engine adds nothing there. The long tail is where complexity and intent concentrate, and historically it is the worst-served slice of search traffic. A sensible production architecture routes simple queries through the cheap path and reserves query planning for the long tail, where a few hundred extra milliseconds buy a correct result instead of a bounce on a $79 order.

That routing logic, plus constraint extraction, hybrid retrieval, and semantic reranking, adds up to a lot of infrastructure to assemble by hand. Seekora builds it into its AI-powered search engine for ecommerce, delivered through an API and UI SDK, so teams get intent understanding and multi-constraint handling without operating their own planner, reranker, and filter-extraction stack.

The Bottom Line for Ecommerce Search Teams

Agentic retrieval graduated from research pattern to managed infrastructure in under a year, and both hyperscalers picked product-style queries to show it off. The evidence so far supports a specific, bounded conclusion:

• Single-shot RAG structurally cannot enforce hard constraints, because one similarity score cannot hold a price cap, a size, and a width at the same time.
• LLM query planning fixes this by decomposing intents into parallel sub-queries and structured filters, with semantic reranking stitching the results back together.
• The gains are real but qualified: Microsoft reports up to 40 percent better relevance on complex queries only, vendor-measured, with parity on simple ones.
• Cost and latency scale with plan complexity, so routing simple queries around the planner belongs in the design from day one.

For ecommerce teams, the evaluation step is concrete. Pull the top 200 zero-result and zero-click queries from last quarter's search analytics and count how many contain two or more constraints. That count approximates the revenue currently handed to a pipeline that cannot parse what shoppers are asking for. Teams that want to see query planning work against their own catalog can request a Seekora demo and watch those same 200 queries come back with products that actually match.