FedRAMP was built for traditional cloud. AI systems — LLMs, RAG pipelines, autonomous agents, vector databases, MCP integrations — introduce attack surfaces that standard cloud assessments don't cover. This guide explains exactly what federal AI compliance requires, what gaps to close before authorization, and how to answer the questions every agency will ask.
FedRAMP was designed to assess cloud infrastructure — servers, networks, storage, and web applications — that behave predictably and follow defined logic. A web server either accepts a valid HTTP request or it doesn't. An AI system interprets instructions, makes probabilistic decisions, generates dynamic outputs, retrieves documents based on semantic similarity, and executes actions through connected tools. The attack surface is fundamentally different.
The most important misconception: FedRAMP authorization for a cloud platform does not automatically cover AI workloads running on that platform. If AI components were not explicitly included in the System Security Plan (SSP) and assessed by a 3PAO as part of the authorization, they are outside the authorization boundary. Adding LLMs, RAG pipelines, or autonomous agents to an existing FedRAMP-authorized system requires a significant change review at minimum — and potentially a new assessment.
The most common compliance gap: An organisation achieves FedRAMP authorization for their SaaS platform, then adds an AI assistant feature powered by an external LLM API. The AI feature is not in the SSP, the LLM vendor is not FedRAMP-authorized, and federal customer data flows through an unauthorized system. This creates a reportable compliance gap — often discovered during agency security review, not before.
Federal AI deployments process some of the most sensitive information that exists — CUI, law enforcement data, defence information, citizen records. Each component of an AI system introduces vulnerabilities not present in traditional cloud applications:
Adversarial instructions override AI safeguards — potentially exposing CUI, bypassing access controls, or causing the agent to perform unauthorized actions in connected federal systems. Indirect injection through retrieved documents is particularly dangerous in government RAG deployments.
The highest-risk scenario for federal AI. CUI appearing in AI outputs to unauthorized users, CUI stored in embeddings without appropriate controls, or CUI transmitted to third-party AI APIs without proper agreements creates immediate reportable compliance events.
Agents with excessive permissions across federal systems can be manipulated into accessing records, executing workflows, or triggering actions beyond their defined scope. The blast radius scales with the breadth of connected federal systems.
Third-party model providers, open-source model repositories, AI plugins, and MCP connectors all introduce supply chain risk. A compromised model or dependency could affect every federal deployment that uses it.
RAG systems without fine-grained access controls can return documents above a user's clearance or authorisation level in response to crafted queries. Cross-authorization-level retrieval is a critical finding in federal security assessments.
Function calling and MCP integrations into federal systems can be manipulated to escalate privileges, trigger unauthorised workflows, or exfiltrate data through legitimate-looking API calls that bypass traditional monitoring.
| Control Family | AI Relevance | Key AI-Specific Requirements |
|---|---|---|
| AC — Access Control | Critical | Who can access model, prompts, outputs, retrieval layer? Least privilege for AI agents. CUI access control in RAG systems. |
| AU — Audit & Accountability | Critical | Log prompts, outputs, retrieval events, tool calls, agent decisions. Retention for investigation. AI events differ from traditional audit events. |
| CM — Config Management | Critical | Track model versions, prompt templates, agent configurations, retrieval settings. AI config drift is a real risk. |
| IR — Incident Response | Critical | AI-specific playbooks for prompt injection, data leakage, agent compromise. Traditional IR procedures are insufficient. |
| RA — Risk Assessment | Critical | AI threat modelling, supply chain evaluation, ongoing model risk assessment, emerging threat monitoring. |
| SC — System & Comms Protection | Critical | Encrypt model APIs, vector databases, agent communications. Segment AI infrastructure from other federal systems. |
| SA — System Acquisition | High | Supply chain risk for AI models, plugins, dependencies. Vendor assessment requirements for AI providers. |
| SI — System Integrity | High | AI output integrity monitoring, model behaviour validation, prompt injection detection. |
| CA — Assessment & Authorisation | High | AI components must be in the SSP and assessed. Significant change process for AI additions. |
The single most operationally important question for federal AI deployments is: does CUI enter the AI system? If yes, every component that touches that data must be within an appropriate authorization boundary with corresponding controls.
No restrictions. Can use any AI deployment model. No special controls required for the data itself.
Can use FedRAMP Low or Moderate authorized systems. Standard access controls apply.
Requires FedRAMP Moderate minimum. Strict access controls, logging, and handling requirements. Third-party AI APIs must have appropriate agreements.
Requires FedRAMP High or equivalent. Enhanced controls. Many organizations choose private RAG deployments to maintain full control.
Cannot use commercial FedRAMP cloud AI. Requires classified-accredited systems (IL4/IL5/IL6). AISA-authorized solutions only.
The vector database CUI problem: Many organisations index CUI into RAG vector databases without recognising that the database now contains CUI. Embeddings can be used to reconstruct source content. A vector database containing CUI embeddings must be protected with the same controls as the source documents — encryption at rest, access controls, audit logging, and authorisation boundary inclusion.
FedRAMP CA-8 requires penetration testing. While specific AI testing requirements are not yet codified in FedRAMP, agencies and 3PAOs are increasingly requesting AI-specific security testing evidence as part of authorization packages. Proactively including this evidence accelerates authorization and reduces the risk of assessment findings.
Supply chain risk has become a top federal security priority following multiple high-profile software supply chain incidents. AI introduces new dimensions of supply chain risk that differ from traditional software dependencies.
Organisations that prepare documented, evidence-backed answers to these questions before the procurement review significantly accelerate federal sales cycles. Those that encounter these questions for the first time during agency security review typically experience months of delays.
The most common gap. AI features added to an existing authorized system must go through the significant change process before processing federal data.
OpenAI, Anthropic, and Google APIs are not FedRAMP-authorized. Federal data sent to these APIs leaves the authorization boundary. Azure OpenAI Service is the primary FedRAMP-authorized path for GPT models.
AU control family requires logging of relevant events. Prompts and AI outputs are the primary evidence source for AI security incidents. Without them, IR investigations cannot be completed.
Vector databases containing CUI embeddings are CUI systems. Many teams treat them as infrastructure rather than data stores, missing encryption, access control, and audit requirements.
RA control family requires risk assessment. A threat model that doesn't include prompt injection, RAG poisoning, agent abuse, and AI supply chain risks is incomplete for federal AI systems.
3PAOs and agency reviewers increasingly require prompt injection testing results and AI red team documentation. Organisations that have not performed AI-specific testing have no evidence to provide.
The HexTyx AI Security Assessment evaluates your prompt injection resilience, RAG security, agent controls, and compliance readiness. Get documented findings to include in your security assessment package.