Agent monitoring: structured event logging
Log all agent decisions, tool calls, and outcomes with AgentSecurityEvent structure including: event_type (e.g., 'tool_call'), severity (INFO, WARNING, CRITICAL), agent_id, session_id, user_id, timestamp, details dict, optional tool_name. Before logging, redact sensitive data (password, api_key, token, secret, credential fields). Log structured decision metadata for high-risk actions: action classification, risk score, authorization outcome, approval identifier, execution result, policy version. Monitor for drift in approval behavior, repeated approval bypass attempts, elevated privilege usage, abnormal tool invocation frequency, and sudden increases in high-risk actions.
Agent monitoring: anomaly detection thresholds
Define ANOMALY_THRESHOLDS as monitoring limits: tool_calls_per_minute = 30, failed_tool_calls = 5, injection_attempts = 1, sensitive_data_access = 3, cost_per_session_usd = 10.0. Track session metrics including tool_calls (timestamped list), failed_calls count, total_cost. When recent tool calls within 60-second window exceed threshold, log anomaly event with event_type 'anomaly_detected', severity WARNING, and reason 'excessive_tool_calls'.
CodeQL scanning for GitHub Actions workflows
Enable CodeQL actions scanning with 'language: actions' included in the workflow (if enabling via workflow file) or ensure 'GitHub Actions' appears under the Languages section (if enabling via Settings → Advanced Security → Code scanning → CodeQL analysis). CodeQL is freely available for open-source repositories. Configure CodeQL to run on every relevant pull request and mark as required status checks before merging, blocking merges when high or critical severity issues are detected. Run comprehensive workflow scans on a scheduled basis (e.g., daily) and track findings over time.
Zizmor static analysis for GitHub Actions
Use Zizmor for defense in depth scanning of GitHub Actions workflows. Periodically upgrade Zizmor as new releases contain updated detection rules. Configure Zizmor to run on every relevant pull request and mark as required status checks. The Zizmor 'impostor-commit' rule can automate checks that used commits belong to the specified organization/repository, preventing dependency confusion attacks.
Restrict egress traffic from GitHub-hosted runners
Use solutions such as Harden-Runner to monitor and restrict egress traffic from GitHub-hosted runners to prevent secret exfiltration.
Security monitoring for legacy applications
Legacy applications should be subject to an especially high degree of security monitoring with rapid response efforts made to investigate potential incidents. Potential challenges include intra-operability issues where logs produced by the application are in a format that cannot be readily ingested by security monitoring tools. Workarounds might include developing custom APIs to modify security-applicable information from the legacy application and its logs into a format ingestible by security monitoring solutions, or using automation scripts to generate reports that assess for indicators of compromise.
Comprehensive monitoring for prompt injection attacks
Implement these monitoring controls: implement request rate limiting per user/IP, log all LLM interactions for security analysis, set up alerting for suspicious patterns, monitor for encoding attempts and HTML injection, track agent reasoning patterns and tool usage.
MCP resource controls for DoS and abuse prevention
Apply resource controls including rate limits, quotas, and timeouts per session or tenant to resist DoS attacks and limit impact of abuse. Combine with sandboxing to contain local escape impact.
MCP Monitoring, Logging & Auditing requirements
Log all MCP tool invocations with full parameters, user context, and timestamps. Feed MCP logs into a SIEM for anomaly detection. Alert on unusual patterns: new tools being called, admin-level queries, abnormal call frequency. Redact secrets and PII from logs. Conduct regular security audits and simulated attacks against MCP setups.
Embedding Model Version Logging
Log the embedding model version used for each document. When models are updated, flag documents whose relative positions change significantly.
Retrieval Logging for Compliance
Log every retrieval with the querying agent or user's identity and the access control metadata of the retrieved chunks. This log is essential for compliance audits.
Deletion Log for Regulatory Compliance
Maintain a deletion log for regulatory compliance tracking GDPR right to erasure and data retention policies.
Vector Store Orphan Chunk Audit
Periodically audit the vector store for orphaned chunks whose source documents no longer exist.
Vector Index Integrity Monitoring
Monitor vector index integrity using periodic checksum verification.
Index Modification Logging
Log all index modifications (inserts, updates, deletes) with timestamps and the identity of the modifier.
Alert on Unexpected Index Size Changes
Alert on unexpected index size changes. Sudden growth may indicate bulk poisoning. Sudden shrinkage may indicate deletion attacks.
Query Pattern Monitoring for Reconnaissance
Monitor query patterns for reconnaissance behavior such as an agent systematically varying query terms to map the contents of the vector store.
Query Logging for Audit
Log all queries with the querying entity's identity for audit purposes.
Tool Invocation Traceability Logging
Log all tool invocations with full traceability: which query triggered which retrieval, which retrieval influenced which model output, and which model output triggered which tool call.
Cache Hit Logging
Log cache hits with the same detail as fresh retrievals for audit purposes.
Full RAG Pipeline Logging
Log the full pipeline for every request: query received, chunks retrieved (with document IDs and access control metadata), model input assembled, model output generated, and any tool calls triggered.
Replayable Traces for Incident Investigation
Store replayable traces that allow security teams to reconstruct exactly what happened during an incident, including which query retrieved which chunks and which chunks influenced which output.
Anomaly Detection Alerts for RAG
Alert on anomalous patterns: unusual retrieval patterns (a user suddenly retrieving from document collections they have never accessed), repeated prompt injection attempts, access control violations (attempts to retrieve restricted chunks), and sudden changes in retrieval distribution (may indicate index tampering).
Alert on Repeated Pipeline Failures
Alert on repeated failures, which may indicate an active attack such as an attacker deliberately causing retrieval failures to force the model into answering from memory.
Lack of Monitoring and Drift Detection
Absence of systems to detect shifts in model behavior, data distribution, or performance creates operational risks.
Monitoring and Logging controls
Monitoring and logging controls include: monitor input distribution, output entropy, and latency; detect drift via statistical analysis or shadow models; log requests and access with traceability while avoiding logging sensitive data; alert on unusual usage patterns such as scraping or injection attempts.