Leadership wants AI. Your compliance officer wants to keep the CMMC boundary intact. Your IT team wants to know what they're actually supposed to build. And your AI vendor is telling you their tool is "FedRAMP authorized" and that should be enough.

It isn't enough. "FedRAMP authorized" tells you a service has been through a compliance evaluation. It tells you nothing about whether your specific deployment — in your specific Azure tenant, processing your specific data — is within your CMMC authorization boundary. Those are two completely different questions, and conflating them is how defense contractors build AI systems that become findings.

I designed and deployed a production enterprise AI assistant inside a CMMC-scoped GCC High environment: Azure OpenAI with a RAG architecture, CosmosDB for conversation history, Azure AI Search for document retrieval, Bot Framework for the interface layer. It runs in production and it's in the SSP. Here is what I learned about getting the compliance posture right before writing a line of code.

The Three Decisions That Happen Before Any Code

Most organizations want to start with the technology — which model, which interface, which vector database. Those are implementation questions. They come second. The compliance questions come first, and they are architectural decisions that constrain every implementation choice that follows.

Decision 1: Does the AI system touch, process, transmit, or store CUI?

If yes, the system is in scope for CMMC. Every Azure service it uses must be within your CMMC boundary. Every control in NIST SP 800-171 applies to it. It needs SSP entries. It needs audit logging. It needs access controls tied to your identity management infrastructure.

If no — meaning the AI system will never see CUI — it can potentially live outside the CMMC boundary. But "no" is harder to sustain than it sounds. If users are asking the AI questions about programs, contracts, technical specifications, or anything adjacent to what your company does for DoD, there is a strong likelihood that CUI will eventually enter the system through user queries, uploaded documents, or integrated data sources. Many organizations decide "no CUI" in the design phase and find themselves with a CUI-adjacent system eight months later that was never scoped or documented. Build the boundary on the conservative assumption: if there's a plausible path to CUI contact, treat it as in scope.

Decision 2: Is your Azure tenant GCC High — not commercial?

This is non-negotiable for CMMC Level 2 when CUI is involved. GCC High is a separate Azure environment with different endpoints, different data residency guarantees, different authorization boundaries, and different availability of specific services. Commercial Azure, even with FedRAMP-authorized services, is not an acceptable environment for CUI under CMMC.

FedRAMP authorization of a service means that service has been evaluated and found to meet specific security requirements. It does not mean that service, as deployed in your commercial Azure tenant, is within your CMMC authorization boundary. Your authorization boundary is defined by your SSP — by the documented system you describe, the controls you implement, and the services you include. A commercial Azure deployment has no place in a CMMC Level 2 SSP that covers CUI.

Decision 3: Have you amended your SSP?

The AI system — including every Azure service it uses — needs to appear in your System Security Plan. Not as a footnote. Not as a comment in the body of a control implementation statement. As a documented system component with boundary documentation, data flow diagrams, service descriptions, and explicit control mapping.

"We added an AI assistant" is not an SSP entry. "Azure OpenAI Service (GCC High), deployed in subscription [ID], accessible via private endpoint, with conversation history stored in CosmosDB, handling employee queries that may include CUI per the data handling SOP at [reference]" is closer to an SSP entry. The detail is what survives an audit.

Why "FedRAMP Authorized" Doesn't Mean What You Think

The confusion here is understandable, and vendors actively contribute to it. When an AI vendor tells you their product is "FedRAMP authorized," they are telling you something about the product's compliance pedigree. They are not telling you anything about your deployment.

Here is the distinction that matters: a FedRAMP authorization covers the service as operated by the cloud service provider in their infrastructure. It covers their controls, their configurations, their data center operations. Your authorization boundary covers what you build on top of that service, how you configure it, who has access to it, what data flows through it, and how it connects to your other systems.

An assessor evaluating your CMMC posture is not going to look at Azure OpenAI's FedRAMP package and conclude your deployment is compliant. They are going to look at your SSP, your access controls, your audit logs, your network architecture, and your policy documentation. The FedRAMP authorization is a prerequisite — it means you can use the service in a compliant deployment. The compliance work is everything you do around it.

Azure OpenAI in GCC High — What's Actually Available

GCC High availability for Azure AI services differs from commercial Azure, and it changes over time. As of the publication date of this article, the following services are available in GCC High and appropriate for a production RAG deployment:

Some Azure AI services have delayed or limited availability in GCC High. DALL-E and certain newer cognitive services may not be available. AI Studio features available in commercial may not have full parity in GCC High. Design your system around confirmed GCC High availability — do not build a commercial prototype and assume you can lift-and-shift to GCC High later.

The RAG Architecture Pattern That Survives an Audit

A Retrieval-Augmented Generation architecture in GCC High for a CMMC-scoped deployment has specific characteristics that separate compliant from non-compliant implementations. Here is the pattern that works:

Azure OpenAI (GCC High) as the model endpoint. Deployed in your GCC High subscription. Accessed via private endpoint — not the public Azure OpenAI endpoint. Model access restricted to service principals with specific roles; no direct user access to the model API.

Azure AI Search as the vector store and retrieval layer. Documents are chunked, embedded, and indexed here. The index contains CUI-adjacent content, so it is fully within the CMMC boundary. Access restricted to the application service principal. No public endpoint.

Azure Blob Storage for the document corpus. Documents uploaded for indexing live here before processing. Storage account with no public access. Private endpoint. Lifecycle management policies documented in the SSP.

Azure CosmosDB for conversation history. This is critical and frequently overlooked: conversation history contains the user's questions, which may reference CUI, and the model's answers, which may contain CUI extracted from indexed documents. CosmosDB storing this data is in scope. Private endpoint. Encryption at rest. Retention policy documented.

Azure Entra ID for authentication end-to-end. Users authenticate to the AI application with their Entra ID credentials. The application uses a managed identity or service principal to authenticate to each downstream service. No shared credentials, no API keys stored in application configuration.

Private endpoints throughout. No Azure service in this architecture should have a public endpoint enabled. Network Security Groups with explicit allow rules for required traffic paths. Virtual network integration for the application layer.

Azure Monitor and Log Analytics for audit logging. Every user interaction, every model invocation, every document retrieval event should be logged. This covers the AU (Audit and Accountability) control family under NIST SP 800-171. Retention period should be documented in your SSP as an organization-defined parameter under Rev 3.

What the Assessor Will Ask

A C3PAO assessor who encounters an AI system during your CMMC assessment will work through a predictable set of questions. If you can't answer these with documentation, your AI system becomes a finding — and findings tied to AI systems tend to be significant because they implicate multiple control families simultaneously.

"Show me the SSP entry for the AI system." They want to see the system description, the component inventory, the boundary definition, and the data flow documentation. Not a paragraph. A proper SSP component entry.

"Show me the system boundary diagram." Draw the boundary. Show every Azure service, every data flow, every external interface, every user access path. If you can't draw the boundary, you can't defend it.

"How does CUI flow through the system?" From user query to model to retrieval to response — trace it. Where does it get stored? Who can access it? How long is it retained? What happens when a user's session ends?

"Show me your access controls." Who has access to the AI system? How is that access granted and reviewed? Who has access to the underlying Azure services? How are privileged accounts managed?

"Show me your audit logs." What is logged? How long are logs retained? Who has access to the logs? What is your process for reviewing them?

"Show me your acceptable use policy." Do users know what they can and cannot ask the AI? Is there documented guidance on CUI handling through the AI interface?

The SSP Amendment

When you deploy an AI system into a CMMC-scoped environment, your SSP changes. This is not optional — it is a requirement of the NIST SP 800-171 Rev 3 configuration management controls (CM family) and a basic condition of maintaining an accurate boundary description.

The SSP amendment needs to include: a system description (what the AI assistant does, what data it processes, what business purpose it serves), a component inventory (every Azure service, with subscription IDs, service names, and resource groups), an updated system boundary diagram, data flow documentation, control mapping for the new components, and references to any new policies (acceptable use, data handling SOP).

Organization-defined parameters under Rev 3 that specifically apply to AI systems include audit log retention periods, the definition of privileged access roles for AI infrastructure, and the approval process for new AI capabilities or model changes (a configuration management control). Document these parameters explicitly — assessors will ask.

Ready to deploy AI inside your CMMC boundary?

Fulcrum Advisory has designed and deployed enterprise AI in a production GCC High environment. We can do the same for your organization — and document it for your auditors.

Schedule a Call