Context Architecture: Runtime Retrieval for the Agent Era

Why RAG Hits Its Limits with Agents

Retrieval-Augmented Generation (RAG) solved a real problem: language models hallucinate when they lack grounding data, so engineers learned to stuff relevant documents into the prompt before the model was called. That approach — chunking, embedding, and retrieving top-k results via semantic similarity — works well for the human-scale query problem. A user asks a question; the system fetches relevant context; the model answers.

The agentic workload is structurally different. Agents don't ask one question — they execute multi-step workflows involving dozens or hundreds of tool calls, each potentially requiring different slices of data. The retrieval pipeline built for a single human query cannot absorb that volume. Enterprises that have deployed agents at scale consistently report a variant of the same failure: the models are correct, but the data layer beneath them is scattered, stale, and structured for human readers rather than machine queries.

The structural mismatch is not subtle. As of Q1 2026, leading AI infrastructure analysts frame it as a capacity problem: companies will eventually run orders of magnitude more agents than human users, which means orders of magnitude more read load on backend systems — precisely the class of problem that historically required a new caching or retrieval architecture tier.

The Shift: Pull Replaces Push

Context architecture is the emerging response. Rather than pushing a pre-selected payload of documents into the agent's context before the model is called (RAG's operating model), context architecture lets the agent pull exactly the data it needs, at runtime, through tool calls addressed to a governed retrieval layer.

The inversion sounds subtle, but its implications are significant:

• In the RAG model, an operator pre-selects retrieval patterns and hard-codes them into pipelines before deployment. This works when queries are predictable.
• Agents cannot predict their own data needs in advance — they discover requirements dynamically as they execute. If the retrieval interface wasn't built for the query the agent generates at runtime, the agent stalls or hallucinates.

One way to understand the difference is through a caching analogy: classic RAG is like running to a remote store every time you need an ingredient — it works, but doesn't scale when the cooking pace (agent throughput) increases dramatically. Context architecture pre-positions the right data close to the consumer, keeps it fresh via continuous synchronization, and serves it through interfaces built for machine consumption rather than human search.

Redis Iris: A Reference Implementation

Redis Iris, launched in May 2026, is the first production platform built explicitly around context architecture principles. It consists of five components:

Redis Data Integration (RDI) — Generally available. Uses change data capture (CDC) pipelines to continuously sync data from relational databases, warehouses, and document stores (Oracle, Snowflake, Databricks, Postgres) into Redis. The result is a low-latency read layer that reflects the source of truth in near-real time without requiring full re-ingestion.

Context Retriever — In preview. Developers define a semantic model of business entities using Pydantic models; Redis auto-generates MCP tools that agents use to query the model directly. Row-level access controls are enforced server-side. The retrieval interface is tool-native from the start, not retrofitted from a text-search stack.

Agent Memory — In preview. Stores short and long-term agent state across sessions, so agents carry context between turns without re-deriving it on each invocation. This is especially valuable for long-horizon workflows — multi-hour sessions, multi-day tasks — where re-fetching from scratch on every turn would be both expensive and error-prone.

Redis Flex — A rewritten storage engine that runs 99% of data on flash (SSD) and 1% in RAM, delivering petabyte-scale retrieval at sub-millisecond latencies at approximately one-tenth the cost of a purely in-memory stack.

LangCache — A semantic caching layer that reduces redundant model calls. When a new query is semantically similar to a prior one (cosine similarity above a configurable threshold), the cached result is returned without re-invoking the model or re-traversing the data layer.

Market Context

Q1 2026 market data (VB Pulse RAG Infrastructure Market Tracker) reflects the transition in buyer intent:

• Enterprise intent to adopt hybrid retrieval tripled from 10.3% to 33.3% between January and March 2026.
• Retrieval optimization surpassed evaluation as the top enterprise AI investment priority for the first time.
• Custom in-house retrieval stacks rose from 24.1% to 35.6% as enterprises outgrew off-the-shelf options.
• Retrieval optimization investment rose from 19% to 28.9% across Q1, overtaking evaluation spending.

These numbers indicate the industry is not debating whether to upgrade retrieval infrastructure — active procurement decisions are already underway.

Governance: The Critical Challenge

Every context layer faces the same governance challenge: agents must be faster, cheaper, and safer to operate simultaneously. A context architecture that improves retrieval latency but introduces new data access risks or audit gaps will not scale in regulated industries.

Analysts frame the governing question as four axes: freshness (how current must the data be?), access control (who and what is allowed to query it?), latency (what is the per-retrieval latency budget?), and cost (what does each retrieval operation cost?). Answering these before selecting a context layer technology produces better architectural decisions than asking "should I use a vector database, long context, memory, or a context engine?"

Within Iris, these axes are addressed by different components: server-side row-level access controls address the access dimension; LangCache addresses cost; Agent Memory addresses freshness for state; CDC pipelines address freshness for source data.

Production Case: Real-Time Clinical AI

A healthcare AI platform deployed across large medical facilities demonstrates the Agent Memory use case in a high-stakes environment. The system runs multiple specialized agents in parallel — one for entity identification, one for relationship reasoning, one for case history integration — across complex sessions such as group therapy or clinical case reviews.

The memory requirement is concrete: tracking who said what during a one-hour group session, and surfacing relevant case history to the clinician in real time. The platform is built on Redis from the start, with retrieval, memory, and session state all running through a single stack. The motivation for this architecture choice was avoiding stitching together separate tools that don't share a consistency model — a problem common in early RAG deployments where vector stores, caches, and state backends operated independently.

In environments where incorrect context has direct patient-outcome consequences, the runtime properties (sub-millisecond latency, freshness guarantees, access controls) are non-negotiable.

Implications for Enterprise Teams

For teams that built their AI stack around RAG, the shift represents a meaningful architectural change:

1. The semantic model is production infrastructure. Defining business entities, their relationships, and access rules requires versioning, testing, and change management — the same discipline as a data pipeline.
2. Staffing gaps emerge. RAG pipelines are typically owned by ML engineers. Context architecture requires data engineers with CDC, schema design, and access-control expertise.
3. The cost model shifts. RAG's cost is dominated by embedding and retrieval at query time. Context architecture moves cost toward synchronization (CDC pipelines) and away from per-query embedding, which may reduce cost at high query volumes.
4. Context architecture and RAG are not mutually exclusive. Reference implementations are explicitly designed to sit alongside existing databases rather than replace them — reflecting and caching from systems of record rather than becoming one.

Teams that delay context layer investment are likely to revisit the architecture when agent workloads scale, under more time pressure. The Q1 2026 market data suggests this window is closing.

Changelog

2026-05-20 — Page created from VentureBeat May 18, 2026 (Type B, confidence 70)