Dual-LLM architecture for maximum prompt injection defense
The strongest architectural form of guardrailing is the dual-LLM pattern: a privileged LLM holds the tools but never reads untrusted content directly, and a quarantined LLM reads untrusted content but cannot take action. The privileged model receives only structured summaries or labels from the quarantined one, which breaks the path that injected instructions need to reach the actor. Described by Simon Willison at simonwillison.net/2023/Apr/25/dual-llm-pattern/.
MCP architecture: Host, Client, Server, Tools flow
The MCP architecture consists of a User connecting to an MCP Host (the AI application like Claude Desktop or Cursor), which runs an MCP Client that connects to one or more MCP Servers. Each MCP Server exposes tools, resources, and prompts and connects to external Tools, Data, and APIs. The MCP Client passes tool definitions to the LLM, and the LLM sees all tool descriptions from all connected servers in its context.
MCP transports: stdio and HTTP/SSE
MCP supports two transport mechanisms: stdio for local communication between the MCP Host and local MCP Servers, and HTTP/SSE for remote MCP Server connections.
Implementation Priority: Immediate Controls
Foundational RAG security controls to implement immediately: document hashing and integrity verification at ingestion; context window protection with delimiters and chunk limits; access control metadata on every vector chunk; tenant and classification isolation in vector stores; query normalization and abuse pattern detection; output validation and policy enforcement; full pipeline observability and logging; fail-closed behavior across the RAG pipeline.
Implementation Priority: Next Controls
Compliance and audit controls to implement after foundational controls: signed source attribution on every RAG response; vector index integrity monitoring and access controls; tool invocation controls and agent safety; cache isolation and invalidation; supply chain vetting for ingestion connectors; data deletion and retention controls for regulatory compliance.
Implementation Priority: Advanced Controls
Advanced controls for high-security and regulated environments: embedding distribution monitoring and cross-model validation; embedding privacy controls and differential privacy.
Restrict Vector Index Write Access
Restrict write access to the vector index to authorized ingestion pipelines only. No application code or agent endpoint should have direct write access.
Vector Index Snapshots for Rollback
Implement index snapshots for rollback in case of detected tampering.
Vector Database Authentication Configuration
Deploy vector databases with authentication enabled and strong credentials. Some vector databases or deployment modes may ship with authentication disabled or optional by default. Authentication, network isolation, and strong credentials must be explicitly configured before production use.
Pin Embedding Model Versions
Pin versions of embedding models and ingestion libraries. An uncontrolled update to the embedding model can change retrieval behavior across the entire corpus.
Ingestion Source and Connector Inventory
Maintain an inventory of all ingestion sources and connectors with their access credentials, update schedules, and responsible owners.
Least Privilege for Ingestion Connectors
Apply least privilege to ingestion connectors. The connector should have read access to specific folders, not admin access to the entire drive.
Fail-Closed Design Principle
When any component of the RAG pipeline fails, the system must deny the request rather than fall back to potentially unsafe behavior. This principle applies at every stage.
Retrieval Failure Handling
When retrieval fails, do not answer from model memory alone. Return an error indicating the knowledge base is unavailable.
Access Control Failure Handling
When access control check fails, return nothing, not a filtered subset. A failed access control check may indicate a system error, not a clean result.
Source Attribution Failure Handling
When source attribution cannot be generated, block the response. An unattributed response in a regulated environment is a compliance violation.
Document Hash Verification Failure Handling
When document hash verification fails, exclude the document from retrieval and alert. A hash mismatch means the document has been modified since ingestion.
Cache Lookup Failure Handling
When cache lookup fails, generate a fresh response. Do not serve a stale or potentially compromised cached response as a fallback.
Clear Error Messages for Diagnostic Purposes
Return clear error messages that indicate which stage failed, so operators can diagnose the issue.
AI coding agents operate across multiple trust boundaries
AI coding agents interact with developers, model providers, repositories, MCP servers, and CI/CD systems. Repository content (issues, PRs, comments, READMEs, changelogs) becomes instructions when read by agents. MCP servers act as tool providers and can be compromised or malicious. Rules files like .cursorrules, CLAUDE.md, AGENTS.md, and .windsurfrules silently steer behavior across all future generations. Agents running with auto-accept and full developer permissions can execute commands, install packages, write files, and push branches with the same blast radius as a compromised developer workstation. CI/CD agents with access to org secrets present confused-deputy risks.
Rules files enable persistent steering of agent behavior
Files including .cursorrules, .cursor/rules/, CLAUDE.md, .claude/, AGENTS.md, .github/copilot-instructions.md, .windsurfrules, .aider.conf.yml, and custom system prompt files silently steer agent behavior across all future interactions. An attacker who modifies them controls every subsequent generation. The agent itself can modify rules files to embed persistent instructions.
Do: Sandbox AI coding agent execution
Run AI coding agents in sandboxed environments: dev containers, restricted shells, virtual machines, or ephemeral cloud workspaces. Use tool allowlists that restrict which commands the agent can execute. Block access to credential stores, SSH keys, cloud CLI configurations, and sensitive directories. Apply egress controls on the agent's runtime—if the agent does not need outbound network access for the current task, block it. Use ephemeral credentials scoped to the current task rather than long-lived developer credentials. Understand and evaluate the risk of flags like --dangerously-skip-permissions or auto-accept modes that bypass confirmation prompts. Set resource limits (CPU, memory, disk, process count) on agent execution environments.
Don't: Run agents with unrestricted developer access
Do not run AI coding agents with full developer credentials, SSH keys, and cloud access tokens without sandboxing. Do not enable auto-accept mode on untrusted or unfamiliar codebases. Do not allow agents to access production credentials, deployment keys, or org-level secrets from the development environment. Do not assume the agent will only touch files relevant to the current task.
Do: Enforce strict rules file review and protection
Treat rules files as security-critical configuration and review changes with the same scrutiny as CI/CD pipeline changes. Add rules files to code review requirements and require explicit approval for any modification. Monitor for unexpected rules file creation or modification, including by the agent itself. Use git hooks or CI checks that flag changes to known rules files in every PR. Audit existing rules files for instructions that weaken security controls, disable safety features, or direct the agent to ignore certain file types or patterns.
Don't: Allow rules file modification without review
Do not allow PRs from external contributors to add or modify rules files without security review. Do not allow the AI agent itself to modify its own rules files without explicit developer approval. Do not assume rules files are benign because they are plain text—they are instruction injection surfaces with session-level persistence.