Skip to content

Agentic OS — context, memory, orchestration as a stack

A strategic argument that standalone CLI agents like Claude Code are an "evolutionary dead end" unless they sit inside a broader Agentic Operating System that provides shared context, persistent memory, and multi-agent orchestration. The label is community shorthand; the underlying three-layer pattern is real and converging across major labs.

Video

Source: https://www.youtube.com/watch?v=Bgxsx8slDEA

Transcript notes

Key beats from the talk:

  • The "CLI island" problem — a CLI-only agent lacks environmental awareness: no calendar, no chat history, no persistent memory of past architectural decisions. Re-establishing context every session burns thousands of tokens.
  • The three-layer model of an Agentic OS:
  • Context layer — pulling in calendar, email, tickets, documentation (typically via MCP).
  • Memory layer — persistent state across sessions; if an agent failed a task on Monday, it should not repeat the mistake on Tuesday.
  • Orchestration layer — task decomposition and routing; instead of the user manually invoking the CLI, an orchestrator "hires" the CLI agent as a specialised worker.
  • From tool to teammate — the shift from agents you use to agents that work for you. The user gives a high-level objective; the OS breaks it down, fetches context, and directs sub-agents.
  • The "token tax of ignorance" — without progressive disclosure, every new session re-pays the context-establishment cost. An Agentic OS feeds the agent only what it needs, when it needs it.
  • Architect/worker patterns — a powerful model (e.g. Opus) acts as architect while a faster, cheaper model (e.g. Sonnet) handles the grunt work of writing unit tests or running scripts.
  • MCP as the connective tissue — the protocol that lets agents reach external systems is the load-bearing piece, not the model itself.
  • Practical implication for senior IT — don't build in a vacuum; orchestration is the new coding skill; state management is king.

Synthesis

The term "Agentic OS" has no single coiner. The closest verified ancestor is Andrej Karpathy's "LLM OS" framing, introduced in his May 2023 Microsoft Build keynote and elaborated in his November 2023 "Intro to Large Language Models" talk, where the LLM is cast as a kernel with the context window as RAM, tool use as system calls, and external storage as a file system. Academic formalisation followed quickly: Ge et al. proposed an "AIOS-Agent" ecosystem in December 2023 with the LLM as OS, agents as apps, and tools as peripherals; Mei et al. specified a concrete three-layer architecture in March 2024 (application / kernel / hardware) with discrete modules for scheduling, context, memory, storage, tools and access control, accepted to COLM 2025. The shorter "Agentic OS" phrasing is most visible in 2026 popularizer writing — a synthesis of Karpathy's LLM-as-kernel framing with the AIOS academic line and Anthropic's multi-agent research engineering. No major lab uses "Agentic OS" as an official product name, which is why the label is best read as a community umbrella over a converging pattern rather than a defined product category.

In current practice the pattern resolves into three layers. The context layer is dominated by the Model Context Protocol (MCP), open-sourced by Anthropic in November 2024 and adopted by OpenAI in March 2025, Google DeepMind shortly after, and donated to a Linux Foundation directed fund in December 2025. MCP standardises how agents read data, call tools, and fetch prompt templates from external systems, and its widespread adoption (10,000+ active public servers, first-class clients in ChatGPT, Cursor, Gemini, Microsoft Copilot, VS Code, Sourcegraph Cody, Zed, and Claude) makes it the de facto context-layer protocol. The memory layer remains the least standardised piece: vendor-specific stores (Claude Memory, OpenAI Memory, Google's Project Astra memory work) are mutually incompatible, and Anthropic's May 2026 update to Claude Managed Agents collapsed memory ("Dreaming"), evaluation ("Outcomes"), and orchestration into a single bundled runtime — a move VentureBeat characterised as Anthropic attempting to "own your agents' memory, evals, and orchestration." The orchestration layer is implemented as supervisor/worker graphs in production: LangGraph's langgraph_supervisor for hierarchical multi-agent systems, Anthropic's research system that spawns parallel sub-agents, the Claude Agent SDK's programmatic/filesystem/built-in sub-agent primitives with context isolation and tool restrictions, and OpenAI's Agents SDK with agents-as-tools handoffs, guardrails, sessions, and MCP tool calling.

The "CLI island" critique that anchors the video has empirical weight behind it. Repo-review benchmarks show CLI-only sessions burning startling amounts of context: one published test of a 52-file TypeScript library consumed 285K tokens for a single review prompt, with auto-compaction firing at 187K and costing an additional 100–200K tokens per run, sometimes up to three times per turn. That is the practical face of what the talk calls the "token tax of ignorance" — every new session re-pays the cost of establishing what the project is, who the team is, what the recent decisions were, and what's already been tried. The Agentic OS argument is that this is a solvable architectural problem: externalise context (MCP), memory (persistent store), and orchestration (supervisor/worker) into a shared layer that every agent draws from, and the per-session re-establishment cost drops sharply. Andrew Ng's parallel framing in DeepLearning.AI's agentic-AI course pushes the same direction from a different angle: agentic workflows benefit from explicit design patterns — reflection, tool use, planning, multi-agent collaboration — rather than single-shot prompts.

Implementation candidates in the wild span the lab spectrum. Anthropic's multi-agent research system is publicly documented as an orchestrator agent that spawns parallel subagents — the practitioner shorthand "Opus orchestrates, Sonnet executes" maps directly to this pattern. The Claude Agent SDK exposes the same primitives. OpenAI's Agents SDK, the production successor to the experimental Swarm framework, ships agents-as-tools handoffs, guardrails, sessions, and MCP tool calling. Microsoft Copilot Cowork, announced GA in Frontier on 30 March 2026, brings the Cowork agentic-workspace platform — multi-step long-running agent work with skills, scheduled prompts, and multi-app workflows — into Microsoft 365. The lineage runs back through AutoGPT and Yohei Nakajima's BabyAGI in spring 2023, which established the task-decomposition loop that all of the modern frameworks refined.

The "from tool to teammate" framing in the video resolves into two patterns now widely adopted. The first is the architect/worker split: a high-capability model (Claude Opus, GPT-5.5) decomposes the objective, plans the work, and reviews outputs; a faster, cheaper model (Claude Sonnet, GPT-5 mini) handles the routine implementation. This pattern shows up in Anthropic's research system, in the LangGraph supervisor template, and in any number of practitioner write-ups; the empirical claim is roughly 5–10× cost reduction at comparable quality for well-decomposed tasks. The second pattern is specialist sub-agents with isolated contexts — a code agent, a documentation agent, an evaluator — each operating on its own slice of the project so that one agent's context bloat doesn't poison another's reasoning. Both patterns require an orchestrator that can route work, merge results, and resolve conflicts; both depend on a memory layer that survives the individual agent's session.

The honest current view as of mid-2026 is that the pattern is production-ready in narrow verticals — research synthesis, coding assistants, customer triage — and unsettled outside them. Industry trackers report only 11–14% of agent pilots reach production scale (FifthRow's April 2026 enterprise playbook is one of several sources reporting numbers in this range), with most pilots stalling on infrastructure, compliance, or operational readiness rather than model capability. The bundling question — should the context, memory, and orchestration layers come from one vendor or remain composable — is unsettled too: Anthropic is consolidating with its May 2026 bundled runtime, LangGraph and the OpenAI Agents SDK keep components composable, and Microsoft wraps the whole pattern inside Microsoft 365 with Cowork. There is no obvious winner yet, and the choice is increasingly governance-driven rather than performance-driven.

The takeaway the video closes on — "stop chatting with your code, start governing your agents" — translates cleanly into a checklist for anyone building in this space in 2026: pick a context protocol (MCP is the safe bet); pick a memory strategy (vendor-bundled vs composable; if composable, plan for the integration tax); pick an orchestration pattern (supervisor/worker for most cases, specialist sub-agents for high-isolation needs); and treat the agent runtime as infrastructure rather than as a chat assistant. The Agentic OS label may not survive the next round of branding, but the architectural pattern it points at is the working model for production agents going forward.

Changelog

  • 2026-05-11 — Scaffold created from video title and transcript summary (https://videohighlight.com/v/Bgxsx8slDEA); awaiting full transcript for synthesis
  • 2026-05-11 — Synthesised from transcript + deep research; confidence pending → 87 (Type C transcript + 5 Type A primary sources + 3 Type B and 1 Type C corroborating sources); related_pages enriched with youtube-cowork-watch link