Building RAG with Claude

 

Part 1: Define Your Data Structure

1.1 How to Organize Documents

Documents need to be broken into small, retrievable chunks. Each chunk should be standalone and focused.

Document + Chunk Format

One Document = a file (policy, API doc, contract)
One Chunk = a single paragraph or section (256-512 tokens)
Metadata = source, type, date, author

Example: HR Policy Document

{
  "document_id": "hr-policy-2024",
  "title": "HR Policy Manual",
  "source": "hr-policies/vacation.md",
  "type": "policy",
  "chunks": [
    {
      "chunk_id": "hr-policy-2024-chunk-001",
      "text": "Employees receive 20 days of paid vacation annually. Unused days can carry over to the next year, up to a maximum of 5 days.",
      "section": "Vacation Policy",
      "metadata": {
        "page": 5,
        "updated": "2024-01-15"
      }
    },
    {
      "chunk_id": "hr-policy-2024-chunk-002",
      "text": "Vacation requests must be submitted 2 weeks in advance...",
      "section": "Vacation Policy",
      "metadata": {
        "page": 5,
        "updated": "2024-01-15"
      }
    }
  ]
}

Tip: Keep chunks between 200-500 characters. Too small = lost context. Too big = harder to retrieve exactly what's needed.

Part 2: Index and Embed Your Data

2.1 What Embedding Does

Embedding converts text into numbers (vectors) so the system can find similar text mathematically.

"Vacation policy allows 20 days" ↓ [0.12, -0.45, 0.78, 0.23, -0.91, ...] (384-dimensional vector)

Similar text has similar vectors. Your vector store finds chunks close to the user's question vector.

2.2 Simple Python Setup

from openai import OpenAI

# Initialize client
client = OpenAI(api_key="sk-...")

# Embed a chunk of text
def embed_text(text):
    response = client.embeddings.create(
        input=text,
        model="text-embedding-3-small"
    )
    return response.data[0].embedding

# Example
chunk = "Vacation policy allows 20 days annually"
vector = embed_text(chunk)
print(f"Vector (first 5 values): {vector[:5]}")
# Output: Vector (first 5 values): [0.012, -0.045, ...]

Pro tip: Use text-embedding-3-small for cost-efficiency, or text-embedding-3-large for higher accuracy.

Part 3: Store and Retrieve

3.1 Simple In-Memory Vector Store

For prototyping, store vectors in Python. For production, use Pinecone, Weaviate, or Milvus.

import json
from math import sqrt

class SimpleVectorStore:
    def __init__(self):
        self.chunks = []

    def add_chunk(self, chunk_id, text, vector, metadata=None):
        """Store a chunk with its vector"""
        self.chunks.append({
            "chunk_id": chunk_id,
            "text": text,
            "vector": vector,
            "metadata": metadata or {}
        })

    def cosine_similarity(self, vec1, vec2):
        """Calculate similarity between two vectors"""
        dot_product = sum(a * b for a, b in zip(vec1, vec2))
        norm1 = sqrt(sum(a ** 2 for a in vec1))
        norm2 = sqrt(sum(b ** 2 for b in vec2))
        return dot_product / (norm1 * norm2) if norm1 * norm2 != 0 else 0

    def search(self, query_vector, top_k=3):
        """Find top K most similar chunks"""
        scores = []
        for chunk in self.chunks:
            similarity = self.cosine_similarity(query_vector, chunk["vector"])
            scores.append((chunk, similarity))

        # Sort by similarity, return top K
        scores.sort(key=lambda x: x[1], reverse=True)
        return [chunk for chunk, _ in scores[:top_k]]

# Usage
store = SimpleVectorStore()

# Add chunks
store.add_chunk(
    "hr-001",
    "Vacation policy allows 20 days annually",
    [0.12, -0.45, 0.78, ...],  # actual embedding vector
    {"source": "hr-policy", "section": "vacation"}
)

# Search for similar chunks
query_vector = embed_text("How much vacation do I get?")
results = store.search(query_vector, top_k=3)

for chunk in results:
    print(chunk["text"])

Part 4: Retrieve and Augment

4.1 The Retrieval Pipeline

When a user asks a question:

1. Embed the question into a vector
2. Search the vector store for top 3-5 relevant chunks
3. Assemble context from those chunks
4. Send to Claude along with the question

4.2 Retrieval Function

def retrieve_context(user_question, vector_store, top_k=3):
    """
    Retrieve relevant chunks for a question
    """
    # 1. Embed the question
    question_vector = embed_text(user_question)

    # 2. Search vector store
    relevant_chunks = vector_store.search(question_vector, top_k=top_k)

    # 3. Assemble context string
    context = ""
    for i, chunk in enumerate(relevant_chunks, 1):
        source = chunk["metadata"].get("source", "unknown")
        context += f"[{i}] (Source: {source})\n{chunk['text']}\n\n"

    return context, relevant_chunks

# Example
question = "How much vacation do I get?"
context, chunks = retrieve_context(question, store)

print("Retrieved context:")
print(context)
# Output:
# [1] (Source: hr-policy)
# Vacation policy allows 20 days annually...
#
# [2] (Source: hr-policy)
# Unused days can carry over...

Part 5: Connect to Claude

5.1 Send Context to Claude

Combine the retrieved context with the user question and send to Claude's API.

from openai import OpenAI

client = OpenAI(api_key="sk-...")

def rag_query(user_question, vector_store):
    """
    Main RAG function: retrieve context and query Claude
    """
    # Step 1: Retrieve relevant context
    context, chunks = retrieve_context(user_question, vector_store)

    # Step 2: Build the prompt
    system_message = """You are a helpful assistant.
Answer questions using ONLY the provided context.
If the answer isn't in the context, say "I don't have this information."
Always cite which source [1], [2], etc. you used."""

    user_message = f"""Context from our documents:

{context}

Question: {user_question}"""

    # Step 3: Call Claude
    response = client.messages.create(
        model="claude-3-5-sonnet-20241022",
        max_tokens=1024,
        system=system_message,
        messages=[
            {"role": "user", "content": user_message}
        ]
    )

    return response.content[0].text

# Test it
answer = rag_query("How much vacation do I get?", store)
print(answer)
# Output:
# Based on the HR policy [1], you receive 20 days of paid vacation annually.
# Additionally [2], up to 5 unused days can carry over into the next year.

Why This Works

Claude sees the actual policy text [1] [2], so it can answer accurately. It cites sources, making the answer traceable. If your policy changes, you just update the document, no retraining needed.

Part 6: Complete RAG Flow

6.1 End-to-End Example

# SETUP: Load all documents and embed them once
documents = load_documents("policies/")  # Load your docs
store = SimpleVectorStore()

for doc in documents:
    for chunk in doc["chunks"]:
        vector = embed_text(chunk["text"])
        store.add_chunk(
            chunk["chunk_id"],
            chunk["text"],
            vector,
            chunk["metadata"]
        )

print(f"Indexed {len(store.chunks)} chunks")

# RUNTIME: Answer questions using RAG
questions = [
    "How much vacation do I get?",
    "When do I need to request vacation?",
    "Can I carry over unused vacation days?"
]

for q in questions:
    print(f"\nQ: {q}")
    answer = rag_query(q, store)
    print(f"A: {answer}")

Documents ↓ (split into chunks) Chunks ↓ (embed each) Vectors + Chunks → Vector Store ↑ User Question ↓ (embed) Query Vector ↓ (search top-k) Retrieved Chunks ↓ (assemble context) Context + Question → Claude API → Answer

Part 7: Best Practices

1

Chunk Size Matters

Too small: Loses context. Too big: Retrieval noise. Sweet spot: 200-500 characters (1-2 paragraphs).

2

Always Add Metadata

Include source, section, date, author. Helps with traceability and filtering.

3

Validate Retrieval

Test that your top-3 results are actually relevant. If not, adjust chunk size or embedding model.

4

Version Your Documents

Track which version of a policy was used. Policies change—you need to know what was current when the answer was given.

5

Use a Real Vector Store for Production

Simple in-memory stores don't scale. Use Pinecone, Weaviate, Qdrant, or Milvus for production.

Quick Start Checklist

Step	Action	Tool/Tech
1. Prepare	Gather documents (policies, docs, manuals)	Any format (PDF, MD, TXT)
2. Split	Break into chunks (200-500 chars each)	LangChain, llama-index, or manual
3. Embed	Convert chunks to vectors	OpenAI Embeddings API
4. Store	Save vectors + metadata	Pinecone, Weaviate, or JSON
5. Retrieve	Find top-k similar chunks for queries	Vector similarity search
6. Augment	Add context to user question	String assembly
7. Query	Send to Claude API	Claude 3.5 Sonnet

Claude Skills: A Progressive Disclosure Architecture for Extending AI Agent Capabilities

Abstract

Anthropic's Agent Skills format lets a Claude based agent store reusable, specialized procedures as a folder of instructions, scripts, and reference material, and load them only when a task actually calls for them. This paper reviews Agent Skills as a system, drawing entirely on Anthropic's own published documentation, blog posts, and repository materials, to describe what a Skill contains, how its progressive disclosure loading mechanism works, and how it relates to three adjacent mechanisms already available to a Claude based agent: the Model Context Protocol, subagents, and plugins. The review finds a three level loading architecture, always loaded name and description metadata, full instructions loaded only once a Skill is matched to a request, and supporting scripts or reference files loaded only as those instructions call for them, documented to scale to hundreds of installed Skills without a proportional context cost, and a division of labor across the four mechanisms in which each addresses a distinct concern (procedural knowledge, external connectivity, execution isolation, and packaged distribution) rather than competing to solve the same problem. The review also surfaces two documented inconsistencies in Anthropic's own token cost estimates for the loading levels, and one exception to the format's otherwise cross surface portability claim tied to Claude Code specific frontmatter fields. The paper concludes that Agent Skills is best understood as a narrow, complementary addition to an existing toolkit rather than a replacement for any part of it, and notes that its documented exclusion from Zero Data Retention coverage warrants direct verification by any organization operating under federal or similarly regulated data handling requirements before deployment against regulated data.

Introduction

Large language model agents increasingly serve as the operational layer for real world software tasks, from writing and reviewing code to filling regulatory forms and orchestrating multi step workflows. Anthropic's own engineering writing observes that when Claude operates as an agent across long running or complex tasks, its core reasoning ability alone is not sufficient; it also needs procedural knowledge, the specific conventions of a codebase, the correct sequence of operations for a workflow, or the completed template for a recurring output. Without a mechanism to carry that expertise across sessions, an agent effectively starts every session from a blank slate, repeating the same explanations and reconstructing the same context each time a task recurs.

This shortfall carries a direct operational cost. Two levers govern how much specialized capability a model based agent can bring to a task: retraining or fine tuning the model itself, an expensive and slow process, and augmenting the model's context at run time. Context augmentation is the practical lever available to most teams, but every token added to a system prompt or a tool definition is a token unavailable for the actual task, and every additional capability loaded up front increases the chance that irrelevant instructions crowd out what a specific request actually needs. As the number of distinct tasks an agent is expected to handle grows into the hundreds, loading full instructions for every one of them at the start of a conversation becomes impractical.

Anthropic's response to this problem, introduced in October 2025 and extended through the end of that year, is Agent Skills: folders of instructions, scripts, and other resources, packaged behind a single required file named SKILL.md, that Claude loads only when a task calls for them, and that are portable across Claude's consumer app, its command line tool Claude Code, and its developer facing API. This paper examines Agent Skills as a system: what a Skill contains, the mechanism by which Claude decides when to load one and how much of it to load, and how the design compares with the adjacent mechanisms, namely the Model Context Protocol, subagents, plugins, and conventional system prompts, that a builder might otherwise reach for. The aim is to give a reader who has not yet used Skills a concrete and verifiable account of how the architecture works and why it was built this way, grounded throughout in Anthropic's own published documentation rather than in secondary description.

Background

Before Agent Skills, three mechanisms already existed for extending what a Claude based agent could do, and each addresses a different piece of the same underlying problem.

The most direct mechanism is a static system prompt, or in Claude Code specifically, a checked in CLAUDE.md file read at the start of every session. This approach is simple and requires no additional infrastructure, but it does not scale well as the number of task specific conventions grows. Anthropic's own guidance on context engineering describes the context window as a finite and shared resource, noting that agents perform best when they discover relevant context incrementally rather than holding everything in view at once. A single always loaded instruction file that tries to cover many specialized tasks pushes against this constraint directly: every token spent on a rule for a task the current request does not touch is a token unavailable for the request that is actually in front of the model, and it is repeated in full on every single session regardless of relevance.

The Model Context Protocol (MCP), released as an open standard, addresses a different gap: it lets an agent connect to external tools and live data sources, for example a database, a ticketing system, or an internal API, through a standardized server interface. Anthropic's own comparison of the two mechanisms is direct about the boundary: MCP provides connectivity, while the question of how to use that connectivity well, in what order, under what conditions, with what fallback, is left largely unaddressed by the protocol itself. An agent connected to a dozen MCP servers still has to be told, somehow, when a given tool applies and how its outputs should be composed with everything else the agent is doing, and that procedural layer has not had a standard home.

Subagents, a Claude Code specific mechanism, address workload isolation rather than knowledge packaging: a subagent runs a task in a separate context window, which keeps a noisy or exploratory operation from polluting the main conversation, but the reusable expertise a subagent might apply to that task still has to be defined somewhere else. Plugins, meanwhile, are Anthropic's packaging and distribution layer; a plugin can bundle commands, hooks, MCP servers, and skills together for installation as a single unit, but a plugin is not itself a format for expressing procedural knowledge. Anthropic's own reference documentation states plainly that a plugin contributes context through its skills, agents, and hooks, not through a CLAUDE.md style file of its own.

None of these mechanisms, static instructions, external tool connectivity, isolated execution, or packaged distribution, was designed to answer the specific question this paper takes up: how an agent should store and progressively load the accumulated, reusable expertise for a specific, recurring task, in a form that survives across sessions and across the different surfaces on which Claude runs. Agent Skills is Anthropic's answer to that specific question, built to sit alongside, not replace, each of the mechanisms above.

Agent Skills Framework

Skills package procedural knowledge as a self contained folder whose only required member is a single file, SKILL.md, written with YAML frontmatter followed by a Markdown body. Anthropic's own template repository ships a minimal example consisting of nothing more than a name field, a description field, and a body placeholder reading "Insert instructions below," which is deliberately the smallest possible Skill: everything else described in this section is optional structure a Skill can add as its instructions grow.

The SKILL.md File and Frontmatter

Two frontmatter fields are required everywhere Skills run: name, limited to 64 characters of lowercase letters, numbers, and hyphens, and description, limited to 1024 characters, which must state both what the Skill does and when it should be used. Claude Code recognizes a substantially larger set of optional fields on top of these two, among them allowed-tools and disallowed-tools to scope which tools a Skill may invoke, disable-model-invocation and user-invocable to control whether a Skill can be triggered automatically, by the user directly through a slash command, or both, model and effort to override the reasoning configuration for a Skill's own execution, and context: fork to run a Skill in an isolated context rather than the main conversation. Outside Claude Code, in claude.ai uploads and in the Skills API, only six fields are valid at all: name, description, license, compatibility, metadata, and allowed-tools; any Claude Code specific field included in a Skill deployed through those surfaces produces a hard validation error rather than being silently ignored. A Skill of any complexity is expected to extend beyond the single SKILL.md file into a small bundle, typically scripts in a scripts/ folder, longer form documentation in a references/ folder, and static assets or templates in an assets/ folder, all referenced from the SKILL.md body rather than loaded automatically. Anthropic's authoring guidance is explicit that SKILL.md itself should stay under roughly 500 lines, with anything longer moved into one of those supporting files.

The Progressive Disclosure Loading Model

The mechanism that makes a large library of Skills practical is what Anthropic calls progressive disclosure, described through the analogy of a manual that opens with a table of contents, proceeds to specific chapters, and only then reaches a detailed appendix. Concretely, Anthropic's documentation describes three loading levels. At level one, the name and description of every installed Skill are loaded into the system prompt at startup, at a documented cost of roughly 100 tokens per Skill, giving Claude just enough information to recognize when a Skill might apply without paying the cost of its full content. At level two, once a Skill is judged relevant to the request in front of it, Claude reads the SKILL.md body from the file system using its own command line tool, bringing the instructions into the context window at a stated cost of under 5,000 tokens. At level three and beyond, any scripts or reference files the SKILL.md body points to are read or executed only as the instructions direct, and when a script is executed, only its output enters the context window, never the script's own source code.

It is worth noting plainly, for a reader checking this account against Anthropic's own materials, that the documentation is not perfectly internally consistent on the exact token figures. A separate Claude blog post describes the same three levels with different estimates, approximately 50 tokens for metadata and approximately 500 tokens for the SKILL.md body, against the 100 and under 5,000 token figures given on the primary architecture overview page. Both sources agree on the shape of the mechanism; they disagree on the specific numbers, and this paper reports that disagreement rather than silently choosing one figure as authoritative.

Discovery and Invocation

Discovery is primarily automatic: Claude compares an incoming request against the loaded description metadata for every installed Skill and decides for itself which, if any, are relevant, the same matching process that governs whether zero, one, or several Skills load for a single request. Claude Code adds an explicit invocation path on top of this automatic one, through the disable-model-invocation and user-invocable frontmatter fields described above, so that a Skill author can restrict a given Skill to model only use, user only use through an explicit slash command, or leave both paths open. The Skills API adds a third variant: a developer must explicitly attach a Skill to a request through a container.skills parameter before Claude can use it at all, after which Claude still decides autonomously, within that request, whether the attached Skill is actually relevant, a hybrid of explicit attachment and autonomous use that differs from both the fully automatic discovery in Claude apps and the flag based control available in Claude Code.

Portability Across Surfaces

Anthropic states directly that Skills use the same format across Claude's consumer apps, Claude Code, and the developer API, summarized as building a Skill once and using it everywhere. This claim holds at the level of the SKILL.md format itself, but it comes with a documented exception worth stating precisely rather than glossing over: the Claude Code specific frontmatter fields described above, hooks, disable-model-invocation, and several others, are not part of the six field set recognized outside Claude Code, so a Skill written to take advantage of Claude Code specific behavior will not carry that behavior, and in some cases will not even validate, if deployed unmodified through the API or claude.ai. Portability, in other words, applies cleanly to the baseline Skill format and only partially to the richer feature set available in any one surface.

Design Rationale

The central design bet behind Agent Skills is that procedural knowledge should be organized around when it is needed rather than loaded in full at every session, and the case for that bet rests on three claims Anthropic makes explicitly: efficiency, composability, and portability.

The efficiency claim follows directly from the token costs reported in the previous section. Loading only name and description metadata for every installed Skill, at a cost on the order of a hundred tokens each, means an agent can carry a library described as running into the hundreds of Skills without spending meaningful context budget on any Skill it never actually uses in a given session. This is a direct application of a broader context engineering principle Anthropic has articulated separately, that an agent's context window functions as a shared, finite resource best treated as a public good rather than a place to accumulate every instruction that might someday be relevant. Compared against a single, ever growing system prompt or CLAUDE.md file, the practical difference is that the cost of a Skill's full instructions is paid only by the sessions that actually trigger it, not by every session regardless of relevance.

The composability claim is that Claude can stack multiple Skills within a single task and coordinate their use, rather than a builder having to anticipate every combination of capabilities a request might need and write a single monolithic instruction set to cover it. This mirrors the same modular design pressure that produced the small, single purpose tools philosophy in a much older systems context: many narrow, well described units are easier to combine correctly than one broad unit is to keep correct as it grows.

The portability claim, that a Skill written once runs unmodified across Claude's consumer app, Claude Code, and the API, is the least architecturally novel of the three but arguably the most operationally significant for an organization standardizing on Claude across more than one surface. As already noted, this claim is accurate for the baseline six field format and only partially accurate once a Skill uses Claude Code specific fields, so an organization building Skills for use across surfaces should treat the baseline format, not the full Claude Code feature set, as its actual portability contract.

Table 1 summarizes how Skills relate to the three adjacent mechanisms discussed in Background, drawing directly on Anthropic's own comparison of Skills against MCP together with its documentation of subagents and plugins.

Mechanism	What it provides	When it loads	Best suited for
Agent Skills	Procedural knowledge, packaged as instructions plus optional scripts and references	On demand, matched against task description	Recurring, specialized tasks with a defined procedure
Model Context Protocol	Connectivity to external tools and live data	Tool definitions loaded upfront, per connected server	Reaching a system or dataset the model cannot otherwise access
Subagents	An isolated context window for a delegated task	Invoked explicitly for that task	Keeping exploratory or noisy work out of the main conversation
Plugins	Packaging and distribution for commands, hooks, MCP servers, and Skills together	At install time, per component	Shipping a bundle of the above as a single installable unit

The comparison suggests a fairly clean division of labor rather than genuine competition between mechanisms. Anthropic's own framing of the decision between Skills and MCP, that a procedure a builder would explain to a person is a Skill while an actual system the agent must reach is MCP, generalizes reasonably well to the other two rows in the table: a subagent is chosen for isolation, a plugin for distribution, and a Skill for the specific knowledge that either of the other two might need in order to act well once it is invoked.

One design consequence deserves a compliance oriented note given how the format is likely to be adopted inside regulated environments. Anthropic's documentation states plainly that Agent Skills, unlike some other parts of the platform, is not covered by Zero Data Retention arrangements, and that skill definitions and execution data are instead retained under Anthropic's standard data retention policy. For an organization operating under federal contracting or similarly regulated data handling obligations, this is a materially different retention posture from a Zero Data Retention covered feature, and it should be verified directly against current Anthropic policy, rather than assumed, before any Skill handling regulated or sensitive data is deployed in such an environment. This paper flags the exception; it does not resolve it, since the applicable compliance determination depends on the specific regulatory regime and contract in question, and is outside this paper's own scope.

Experimental Setup

This paper is a documentation grounded technical review rather than an experiment on running code, so reproducibility here means a reader independently locating and confirming the same primary sources this paper cites, rather than rerunning a measurement. The material below states plainly which facts are drawn from Anthropic's own primary documentation and which, if any, are not, following the same evidentiary standard a clinical methods section applies to a named instrument.

All claims in this paper about SKILL.md structure, frontmatter fields, the progressive disclosure loading levels, discovery and invocation behavior, and the relationship between Skills, MCP, subagents, and plugins are drawn from domains Anthropic itself owns and operates: anthropic.com, claude.com, docs.claude.com, support.claude.com, and the anthropics organization on GitHub. One exception is noted directly: Anthropic's own materials repeatedly point to agentskills.io/specification as the canonical Agent Skills specification, but this review could not independently confirm that agentskills.io is an Anthropic owned domain from the pages available to it, so any figure or claim traceable only to that domain is marked secondary below and was not used as the sole support for any claim in this paper.

Three further limits of this review are stated directly rather than left implicit. First, two of Anthropic's own pages report different token cost estimates for the same progressive disclosure levels, as already noted in the Agent Skills Framework section; this paper reports both figures rather than resolving the discrepancy, since resolving it would require access to Anthropic's internal measurement methodology, which is not publicly documented. Second, the publish date of one cited blog post could not be confirmed with certainty during this review and should be verified against the live page before being cited elsewhere. Third, numeric claims that appeared only in summaries not published by Anthropic itself, for example a specific active Skill count limit, were deliberately excluded from this paper rather than reported as fact, consistent with this review's rule of favoring primary sources over secondary ones wherever the two disagree or where a claim could not be confirmed on an Anthropic owned page at all.

Sources consulted

1. Anthropic. "Introducing Agent Skills." https://www.anthropic.com/news/skills (October 16, 2025; updated December 18, 2025). Primary. Supports the definition of Skills, the efficiency, composability, and portability claims, and cross surface availability.

2. Anthropic Engineering. "Equipping agents for the real world with Agent Skills." https://www.anthropic.com/engineering/equipping-agents-for-the-real-world-with-agent-skills. Primary. Supports the progressive disclosure analogy, the three level loading description, and the discovery mechanism.

3. Claude Blog. "Building agents with Skills: Equipping agents for specialized work." https://claude.com/blog/building-agents-with-skills-equipping-agents-for-specialized-work. Primary, though this review could not confirm its publish date with certainty. Supports the problem statement and an alternate set of token cost estimates.

4. Anthropic Docs. "Agent Skills overview." https://docs.claude.com/en/docs/agents-and-tools/agent-skills/overview. Primary. Supports frontmatter field limits, the three level token cost table, the Zero Data Retention exception, and the SKILL.md length guidance.

5. Anthropic Docs. "Agent Skills best practices." https://docs.claude.com/en/docs/agents-and-tools/agent-skills/best-practices. Primary. Supports the context window as a shared resource framing and the 500 line guidance.

6. Anthropic Docs. "Skills guide (Build with Claude)." https://docs.claude.com/en/docs/build-with-claude/skills-guide. Primary. Supports the six field portability limit, the API attachment model, and API level size and count limits.

7. Anthropic Docs. "Claude Code Skills." https://docs.claude.com/en/docs/claude-code/skills. Primary. Supports the full Claude Code frontmatter field set and the six field validation error outside Claude Code.

8. Anthropic Docs. "Agent SDK Skills." https://docs.claude.com/en/docs/agent-sdk/skills. Primary. Supports the statement that Claude Code specific fields do not carry over to other surfaces.

9. Anthropic GitHub. "anthropics/skills README." https://github.com/anthropics/skills/blob/main/README.md. Primary. Supports the base definition of a Skill as a folder.

10. Anthropic GitHub. "anthropics/skills template SKILL.md." https://raw.githubusercontent.com/anthropics/skills/main/template/SKILL.md. Primary. Supports the minimal Skill example.

11. Claude Blog. "Extending Claude's capabilities with skills and MCP servers." https://claude.com/blog/extending-claude-capabilities-with-skills-mcp-servers (December 19, 2025). Primary. Supports the Skills versus MCP comparison and decision rule.

12. Anthropic Docs. "Claude Code subagents." https://docs.claude.com/en/docs/claude-code/sub-agents. Primary. Supports the Skills versus subagents distinction.

13. Anthropic Docs. "Claude Code plugins reference." https://docs.claude.com/en/docs/claude-code/plugins-reference. Primary. Supports the Skills versus plugins distinction.

14. Anthropic Engineering. "Effective context engineering for AI agents." https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents (September 29, 2025). Primary. Supports the context window scarcity framing that predates and motivates the Skills specific application of progressive disclosure.

15. Claude Support. "Use Skills in Claude." https://support.claude.com/en/articles/12512180-use-skills-in-claude. Primary. Supports the automatic discovery description and enterprise level administration.

16. Agent Skills specification. https://agentskills.io/specification. Secondary, provenance unresolved. Referenced only because Anthropic's own materials point to it as a canonical specification; not used as the sole support for any claim in this paper.

Results

Two structural results follow directly from the architecture described above: one about the shape of a single Skill invocation, and one about how Skills divide labor against the three adjacent mechanisms already discussed.

Figure 1 traces what happens inside the context window over the lifetime of a single request that triggers one Skill, based directly on the sequence Anthropic's own architecture documentation describes.

No diagram type detected matching given configuration for text: No diagram type detected matching given configuration for text:

The diagram shows why the architecture scales the way Anthropic claims it does. Only the top step, name and description for every installed Skill, is paid on every single request regardless of relevance. The middle step is paid once per request that actually matches a Skill. The bottom step is paid only when the matched Skill's own instructions call for it, and even then only the output of a script, never the script itself, reaches the context window. An agent library measured in the hundreds of Skills is therefore only as expensive, in context terms, as the number of Skills actually triggered in a given request, not the number installed.

The second result is the division of labor already summarized in Table 1 of the Design Rationale section. Read across the four mechanisms, no single row subsumes another: a Skill supplies a procedure, MCP supplies a connection, a subagent supplies isolation, and a plugin supplies a way to ship several of these components together. Anthropic's own documentation describes builders combining these rather than choosing exactly one, for example a Skill that itself calls tools exposed by an MCP server without duplicating that server's own connectivity. The practical implication for a builder deciding how to extend a Claude based agent is that the four questions, what to know, what to reach, what to isolate, and what to ship, are separable, and the architecture reviewed in this paper answers only the first of them.

Conclusion

Agent Skills answers a narrow but previously unaddressed question in the Claude ecosystem: how an agent should store and progressively load the specific, recurring procedural knowledge a task needs, in a form that survives across sessions and travels unmodified across Claude's consumer app, Claude Code, and its API. The architecture's three level progressive disclosure model, metadata always in context, full instructions loaded only on a matched trigger, and supporting scripts or references loaded only as instructions call for them, is the mechanism that makes this practical at the scale of hundreds of installed Skills, and it sits alongside, rather than in competition with, the Model Context Protocol, subagents, and plugins already available to a builder. The clearest remaining gap this review can name is the two conflicting token cost figures Anthropic's own documentation reports for the same loading levels; resolving that gap would require Anthropic's internal measurement methodology, which is not currently public, and until it is, a builder estimating context budget for a Skill heavy deployment should treat the published figures as directionally, not precisely, reliable. Given that Agent Skills is documented as falling outside Zero Data Retention coverage, any organization bound by federal or comparable regulatory data handling requirements should confirm current retention terms directly with Anthropic before deploying Skills against regulated data, a compliance determination this paper flags but does not itself make.

Acknowledgements

This review was prepared entirely from publicly available Anthropic documentation, blog posts, and repository materials, with no external funding. AI assisted search tooling was used to locate and cross check the primary sources listed above; every source cited was independently verified against the URL given rather than accepted from the search tooling's summary alone.

The Model Context Protocol Standardizes AI Agent Access to External Tools and Data

 

Abstract

The Model Context Protocol (MCP) is an open standard, originally released by Anthropic in November 2024 and since donated to an independent, Linux Foundation hosted body, that governs how an AI application connects to external tools and data sources. This paper reviews MCP as a system, drawing entirely on the protocol's own specification, its official documentation site, and Anthropic's own announcements, to describe the problem MCP addresses, its host, client, and server architecture, the primitives a server exposes, its transport layer, and how its governance and version history have evolved since release.

The review finds a client server architecture in which one host application manages a dedicated client for every server it connects to, three server side primitives distinguished by who controls each one (tools controlled by the model, resources controlled by the application, prompts controlled by the user), and a wire protocol built on JSON-RPC 2.0 that has moved, across five dated specification revisions in under two years, from a stateful connection handshake to a stateless, per request version declaration. The review also finds that MCP's governance moved from a single vendor to a multi party foundation structure within roughly thirteen months of its release, and flags one widely circulated design analogy whose attribution to Anthropic could not be confirmed on any official page checked during this review.

The paper concludes that MCP's contribution is standardizing connectivity, a distinct concern from the procedural knowledge layer that a mechanism such as Agent Skills provides, and notes that any organization bound by federal or comparable regulatory data handling requirements should review, before deployment, exactly which third party MCP servers a given integration would connect to and what data those servers would receive.

Introduction

An AI application that needs to read a file, query a database, or act on a ticketing system has historically had to be built against that exact system, one integration at a time. Anthropic's own announcement of the Model Context Protocol describes the resulting cost plainly: every new data source requires its own custom implementation, which makes a genuinely connected system difficult to scale, since the number of integrations a team must build and maintain grows with every new tool or data source it wants an agent to reach, not with any shared, reusable layer underneath them.

This is a distinct problem from the one procedural knowledge formats such as Agent Skills address. A Skill tells an agent how to carry out a specific, recurring task once the agent already has a way to reach whatever the task touches. The problem this paper takes up is prior to that one: how does an agent reach an external system in the first place, in a way that does not require a bespoke integration for every combination of AI application and external tool.

The Model Context Protocol is Anthropic's answer to that specific question, open sourced in November 2024 as a universal, open standard for connecting AI systems to data sources, intended to replace fragmented, one off integrations with a single shared protocol. This paper examines MCP as a system: the roles it defines for the applications and servers that use it, the primitives a server can expose, the wire protocol those primitives ride on, and how the protocol's own version history and governance have changed since its release. The aim is to give a reader unfamiliar with MCP a concrete, verifiable account of how the protocol works and why it was built this way, grounded in the protocol's own specification and in Anthropic's own published materials rather than in secondary description.

Background

Before MCP, an AI application that needed to reach an external tool or data source generally did so in one of two ways, and both carry a cost that scales poorly as the number of tools or applications grows.

The first is a fully custom, point to point integration: code written specifically to connect one particular AI application to one particular external system, with no expectation that the same code could serve a second application or a second system. Anthropic's own framing of this state of affairs is direct: every new data source requires its own custom implementation, and the resulting fragmentation is exactly what a shared protocol is positioned to replace.

The second is vendor specific function calling, in which a model provider defines a schema format for describing callable functions and lets a developer register a fixed list of them for a given conversation. This approach standardizes the shape of a single request within one vendor's API, but it does not standardize discovery or connectivity across vendors or across tools: the list of available functions is typically static for the life of a request, defined by the calling application rather than discovered from the tool itself, and a function schema written for one model provider's calling convention is not portable to another's without translation. MCP's own architecture documentation frames its primitive methods, list, get, and call, as enabling dynamic discovery specifically in contrast to this kind of fixed, hard coded function list, though the protocol's own materials stop short of naming any specific vendor's function calling format directly.

Neither of these two prior approaches was designed to answer the question MCP takes up: how a tool or data source should describe itself once, in a form that any compliant AI application can discover and use, without either a bespoke integration or a hard coded, provider specific function list. MCP's own documentation frames this explicitly as a scaling problem, one where the number of integrations needed grows with every new pairing of application and data source under the two prior approaches, and grows only with the number of new systems, not new pairings, once a shared protocol is in place.

MCP Architecture

MCP defines a small set of roles and a small set of primitives, and nearly everything the protocol specifies is a rule about how those roles exchange those primitives over a shared wire format.

Host, Client, and Server Roles

The protocol's own architecture documentation describes three participants. The host is the AI application itself, for example Claude Code, Claude's desktop application, or an integrated development environment, and it is the host that a user directly interacts with and that coordinates one or more clients. A client is a protocol level component, instantiated by the host, that maintains a connection to exactly one server; the documentation states this as a strict one to one relationship, one client per server, even when a single host is coordinating several clients at once. A server is the program that actually provides context, tools, or data, and it can run locally, for example as a local process the host starts directly, or remotely, over a network connection. A local server communicating over standard input and output typically serves exactly one client, while a remote server reachable over a network transport is built to serve many clients from many different hosts at once. This host to client to server structure is the same shape regardless of what a given server exposes, whether that is a filesystem, a database, or a business application, which is what allows a single client implementation inside a host to work against any compliant server without host specific code for that server.

The Three Core Primitives and Who Controls Each

A server exposes its capabilities through three primitives, and MCP's own documentation distinguishes them by naming who or what controls each one rather than only by what each one contains.

Primitive	What it is	Who controls it
Tools	Functions the model can actively call, deciding when to use them based on the request in front of it	The model
Resources	Read only data sources that provide context without being actively invoked	The application
Prompts	Pre built instruction templates that direct the model to use specific tools and resources together	The user

This three way division matters for reasoning about what a given MCP integration actually does: a server that only exposes resources is a read only context source the model consults but cannot act through, while a server exposing tools is one the model can actively drive, and a server exposing prompts is offering the user, not the model, a way to invoke a particular combination of the other two. Alongside these three server side primitives, the protocol also defines client side primitives that let a server ask something of the host rather than the reverse; elicitation, for example, lets a server request a specific piece of information from the user mid interaction, such as asking a seat preference while booking travel. Two earlier client side primitives, sampling, which let a server ask the client's own model for a completion, and roots, which let a client tell a server which filesystem paths to focus on, are both marked as deprecated in the protocol's current specification revision, a detail worth noting for any implementation built against an earlier version of the spec.

Transport and Wire Protocol

Every message exchanged between an MCP client and an MCP server is a JSON-RPC 2.0 message; this is a hard requirement in the specification, not an implementation choice left to individual servers. Two transport mechanisms carry those messages. Standard input and output transport is used for local processes running on the same machine as the host, and it is this transport that typically serves exactly one client per server. Streamable HTTP is the transport used for remote servers reachable over a network, and it is built to serve many clients at once. Streamable HTTP itself replaced an earlier transport, described in the specification as HTTP combined with server sent events, and that earlier transport has since been formally reclassified as deprecated under the protocol's own feature lifecycle policy, with existing implementations directed to migrate to Streamable HTTP.

Versioning and the Move to a Stateless Handshake

MCP versions its specification by date rather than by a conventional major or minor number, and five such dated revisions are traceable in the protocol's own published history: an original release, a revision that introduced Streamable HTTP as a replacement for the earlier transport, a widely referenced stable revision, a further revision, and the specification's current revision. Under the version scheme used through the widely referenced stable revision, a client and server negotiated compatibility through an explicit initialize handshake at the start of a connection, with the negotiated version subsequently carried on later requests through a dedicated protocol version header on network transports. The current specification revision removes that handshake entirely: every request now carries its own protocol version and client capabilities directly, and a server must implement a discovery method to advertise which protocol versions, capabilities, and identity it supports, with a mismatched version returning a specific, named error rather than failing the connection during an initial negotiation step. This is a substantial architectural change to how compatibility is established, moving from a connection level negotiation to a per request declaration, and it is recent enough, relative to when this paper was prepared, that its practical migration impact across existing MCP servers had not yet been broadly documented at the time of this review.

Design Rationale

MCP's central design bet is that connectivity between an AI application and an external tool or data source should be a shared, standardized layer rather than something every application and every tool builds separately, and three specific design choices follow from that bet.

The first is discoverability. Because a server's tools, resources, and prompts are each retrievable through a listing method rather than hard coded into the calling application ahead of time, a host can connect to a server it has never seen before and learn what that server offers at connection time, rather than requiring the application's own code to already know the server's capabilities in advance. This is the specific advantage MCP's own architecture documentation claims over a static, hard coded list of callable functions defined entirely on the calling side.

The second is the strict separation of roles between host, client, and server. Because a client's only job is to maintain one connection to one server, and a host's job is to coordinate however many clients a given task needs, a host implementation does not need special case logic for any particular server; every server the host reaches looks the same at the client interface, whether it is a filesystem, a ticketing system, or a database. This separation is also what makes the three primitive types meaningful as a design choice rather than an arbitrary categorization: naming the model, the application, and the user as the three distinct controllers of tools, resources, and prompts respectively gives an implementer a direct answer to the question of who is responsible for deciding when each kind of capability gets used.

The third is standardization through open governance rather than single vendor control. Anthropic open sourced MCP at release in November 2024, and roughly thirteen months later transferred its governance to the Agentic AI Foundation, a directed fund under the Linux Foundation co founded by Anthropic together with other model providers and cloud infrastructure companies, with the protocol now formally structured as an independent project under that foundation's policies. Anthropic's own announcement of that transfer reported more than ten thousand active public MCP servers and adoption across multiple AI applications beyond Claude itself at the time of the transfer, offered as evidence that the protocol had reached a scale where independent governance, rather than continued single vendor stewardship, was the appropriate structure.

Table 1 places MCP against Agent Skills, the complementary Anthropic mechanism examined in a companion review, using Anthropic's own published comparison rather than an inferred one.

Dimension	Agent Skills	Model Context Protocol
What it is	Procedural knowledge	Tool connectivity
What it does	Teaches Claude how to do something	Gives Claude access to something
When it loads	On demand, when relevant to the request	Available continuously once connected

Anthropic's own comparison is direct on the relationship between the two: Skills do not replace MCP, since the two solve different problems, MCP providing connectivity and Skills providing the procedural knowledge for using that connectivity well, and Anthropic's own materials describe the most capable workflows as using both together rather than treating either as a substitute for the other.

Compliance note

One design consequence warrants a compliance oriented note given how frequently MCP connects an agent to systems outside the AI application's own boundary. Because an MCP server can be operated by any party, run locally or remotely, and receive whatever data the connected client sends it, each MCP server a host connects to is, from a data governance standpoint, a distinct third party recipient of the request and context passed to it, regardless of who wrote the server. For an organization operating under federal contracting or comparable regulatory data handling obligations, this means an MCP integration's actual compliance posture depends entirely on which specific servers a deployment connects to and what data flows to each one, a determination that must be made per integration rather than assumed from the protocol's own openness or from Anthropic's role in originating it. This paper flags the exception; it does not resolve it, since the applicable compliance determination is specific to each deployment and is outside this paper's own scope.

Experimental Setup

This paper is a specification and documentation grounded technical review rather than an experiment on running code, so reproducibility here means a reader independently locating and confirming the same primary sources this paper cites against the live specification, rather than rerunning a measurement. Because MCP's own specification is dated and has changed materially across revisions, every claim in this paper about roles, primitives, transports, and version negotiation is tied, where the underlying source states it, to a specific dated specification revision rather than treated as a single unchanging fact about the protocol.

Claims in this paper are drawn from three kinds of primary sources: the official specification and documentation hosted at modelcontextprotocol.io, the official modelcontextprotocol GitHub organization and its repositories, and Anthropic's own announcements on anthropic.com and claude.com.

Verification note

Two specific claims that circulate widely in secondary discussion of MCP could not be confirmed on any of these primary sources during this review and are flagged rather than asserted as fact. First, a shorthand description of the integration problem MCP solves, sometimes phrased as an “M by N” integration problem, does not appear on any official page checked for this review; Anthropic's own wording describes the same underlying cost without using that specific shorthand, and this paper reports Anthropic's own wording rather than the shorthand. Second, a widely repeated analogy comparing MCP to a universal hardware connector standard could not be confirmed as Anthropic's own phrase on any official page reachable during this review, one specification page that reportedly contains it having failed to load during this review's research pass; this paper does not use the analogy as an attributed Anthropic quote as a result, and a reader who wants to cite it should verify it directly against the live specification introduction page first.

Sources consulted

• PrimaryAnthropic. “Introducing the Model Context Protocol.” (November 25, 2024)

• anthropic.com/news/model-context-protocol

• Supports the definition of MCP, the fragmentation problem statement, and the open sourcing of the protocol.

• PrimaryAnthropic. “Donating the Model Context Protocol and establishing the Agentic AI Foundation.” (December 9, 2025)

• anthropic.com/news/donating-the-model-context-protocol-and-establishing-of-the-agentic-ai-foundation

• Supports the governance transfer, the foundation structure, and adoption figures at the time of transfer.

• PrimaryAnthropic Engineering. “Equipping agents for the real world with Agent Skills.”

• anthropic.com/engineering/equipping-agents-for-the-real-world-with-agent-skills

• Supports the framing of Skills and MCP as complementary rather than competing.

• PrimaryClaude Blog. “Extending Claude's capabilities with skills and MCP servers.” (December 19, 2025)

• claude.com/blog/extending-claude-capabilities-with-skills-mcp-servers

• Supports the Skills versus MCP comparison table and the “does not replace” statement.

• PrimaryModel Context Protocol. “Governance.”

• modelcontextprotocol.io/community/governance

• Supports the Lead Maintainer, Core Maintainer, and Maintainer structure and the Linux Foundation policy relationship.

• PrimaryModel Context Protocol. “Architecture” (2026-07-28 documentation)

• modelcontextprotocol.io/docs/2026-07-28/learn/architecture

• Supports the host, client, and server definitions, transport descriptions, and the list, get, call discovery framing.

• PrimaryModel Context Protocol. “Server concepts” (2026-07-28 documentation)

• modelcontextprotocol.io/docs/2026-07-28/learn/server-concepts

• Supports the tools, resources, and prompts definitions and their respective controllers.

• PrimaryModel Context Protocol. “Client concepts” (2026-07-28 documentation)

• modelcontextprotocol.io/docs/2026-07-28/learn/client-concepts

• Supports the elicitation, sampling, and roots definitions and the deprecation of the latter two.

• PrimaryModel Context Protocol. “Changelog” (specification revision 2026-07-28)

• modelcontextprotocol.io/specification/2026-07-28/changelog

• Supports the stateless handshake change, the discovery method, and the deprecation of the earlier transport.

• PrimaryModel Context Protocol. “Streamable HTTP transport” (specification revision 2026-07-28)

• modelcontextprotocol.io/specification/2026-07-28/basic/transports/streamable-http

• Supports the transport replacement history.

• PrimaryModel Context Protocol. Specification, revision 2025-06-18

• modelcontextprotocol.io/specification/2025-06-18

• Supports the earlier initialize handshake and protocol version header behavior.

• PrimaryModel Context Protocol. Specification base protocol overview, revision 2025-06-18

• modelcontextprotocol.io/specification/2025-06-18/basic/index

• Supports the JSON-RPC 2.0 requirement and the server feature and client feature grouping.

• PrimaryModel Context Protocol GitHub organization

• github.com/modelcontextprotocol

• Supports the protocol's own description of itself as an open, seamless integration standard.

• PrimaryModel Context Protocol specification repository README

• raw.githubusercontent.com/modelcontextprotocol/modelcontextprotocol/main/README.md

• Supports the repository level description of the protocol.

• SecondaryPress coverage referencing a hardware connector analogy for MCP, published following the December 2025 governance transferCited only to note that the analogy circulates publicly; its attribution to Anthropic directly could not be confirmed on an official page during this review.

Results

Two structural results follow from the architecture described above: the shape of the host, client, and server topology itself, and the way the protocol's own version history shows a shift in how compatibility is established between client and server.

Figure 1 shows the topology a single host establishes when it connects to several MCP servers at once, following directly from the one to one client to server relationship described in the official architecture documentation.

Figure 1. One host, three clients, three servers. Each client is fixed to exactly one server.

The diagram makes explicit a structural constraint that is easy to miss when MCP is described only in prose: the host, not the client, is the component that scales to handle many servers, since every client is fixed to exactly one server for the life of that connection. A host adding a new server does not modify any existing client; it instantiates one additional client dedicated to the new server, leaving every other client and server pair untouched.

The second result concerns how the protocol establishes compatibility between a client and a server, which has changed materially across the specification's own dated revisions. Under the widely referenced stable revision, compatibility was established once, at the start of a connection, through an explicit handshake, and the negotiated version then applied to every later request on that connection. Under the specification's current revision, that connection level handshake has been removed entirely in favor of a declaration carried on every individual request, with a dedicated discovery method letting a server advertise its supported versions ahead of time and a specific, named error returned on a mismatch rather than a failed negotiation. Read as a single trend rather than two isolated facts, this is a move away from treating a client and server pair as maintaining shared connection state at all, toward treating every single request as self describing and independently checkable. Whether this reduces or increases the practical burden on server implementers maintaining compatibility across client versions is not yet settled in the protocol's own published materials at the time of this review, since the change is recent relative to when this review was prepared.

Conclusion

The Model Context Protocol answers a specific, previously unstandardized question: how an AI application should discover and connect to external tools and data sources without a bespoke integration for every pairing of application and system. Its host, client, and server architecture, with a strict one to one relationship between each client and the single server it connects to, and its three server side primitives distinguished by whether the model, the application, or the user controls each one, are the concrete mechanism that answers that question, riding on a JSON-RPC 2.0 wire protocol that has itself moved, across the protocol's dated revisions, from a connection level handshake to a per request version declaration. The protocol's governance has moved in a parallel direction, from a single originating vendor to an independent, multi party foundation within roughly thirteen months of release, which this review reads as evidence of the scale the protocol reached rather than as evidence about its technical design. The clearest remaining gap this review can name is that the practical consequences of the recent shift to a stateless, per request handshake are not yet documented at the scale the earlier, stable revision was, and a builder relying on version negotiation behavior should verify it directly against the specification revision a given server actually implements rather than assume the behavior described for any one revision applies universally. Because MCP by design connects a host application to servers operated by any party, an organization bound by federal or comparable regulatory data handling requirements should treat each MCP server a deployment would connect to as its own distinct data governance question, and confirm what data that specific server receives, before connecting an agent handling regulated data to it.

Acknowledgements

This review was prepared entirely from the Model Context Protocol's own published specification and documentation, Anthropic's own announcements, and the protocol's official GitHub repositories, with no external funding. AI assisted search tooling was used to locate and cross check the primary sources listed above; every source cited was independently verified against the URL given rather than accepted from the search tooling's summary alone, and two widely circulated claims that could not be confirmed on an official page are reported as unconfirmed rather than as fact.