What “HIPAA-compliant generative AI” means
HIPAA does not name any technology. It regulates covered entities and their business associates, and it defines administrative, physical and technical safeguards for protected health information. A generative AI deployment inherits those obligations the moment PHI enters it.
That distinction matters commercially. A vendor cannot truthfully say a model is HIPAA-compliant. What a vendor can say is that its platform implements the required safeguards, will sign a business associate agreement, and can evidence both. When you evaluate claims, the useful question is never "is your AI HIPAA-compliant?" but "which safeguards do you implement, and how would you evidence them in an audit?"
| Term | What it means in an AI deployment |
|---|---|
| Protected health information (PHI) | Individually identifiable health information. In an AI system this includes prompts, retrieved context, generated output and logs — not just the source record. |
| Business associate agreement (BAA) | The contract that makes a vendor accountable for PHI. Every party that can access PHI needs one, including any model provider reached through the platform. |
| Minimum necessary | Only the data required for the task may be disclosed. In practice this shapes what retrieval returns and what is sent to a model. |
| Audit controls | Hardware, software and procedural mechanisms that record and examine activity in systems containing PHI. For AI this means logging prompts, retrievals, model calls and outputs. |
Where PHI travels, and where it must stop
The single most important architectural decision is the location of the compliance boundary. In a well-built healthcare AI platform, PHI enters the boundary from the source systems, is processed inside it, and leaves it only under contract and only as much as the task requires.
- Ingress. PHI arrives from the EHR, a data warehouse or a document store through an authenticated integration, carrying the requesting user's identity.
- Retrieval. The platform selects the minimum necessary context for the task. This is where enterprise RAG design directly affects compliance posture, not just answer quality.
- Model call. The assembled prompt goes to the selected model over an encrypted API connection, under a BAA, with provider-side training on the data contractually disabled.
- Generation and return. Output returns into the boundary, is logged, and is shown only to users authorised for that patient or that workflow.
- Retention. Prompts, context, outputs and logs are retained on a defined schedule and destroyed on it.
The most common compliance gap is not the model — it is the surrounding tooling. Copy-pasted PHI in a general-purpose chat assistant, an unlogged evaluation harness, or a prototype notebook that pulled real records all put PHI outside the boundary while the production system looks compliant.
The control set a health system should require
| Control | What good looks like |
|---|---|
| Encryption | TLS 1.2 or above in transit; AES-256 at rest, including vector stores, caches and logs. Vector embeddings derived from PHI are treated as PHI. |
| Identity and access | SSO against the health system's identity provider, role-based access, and per-agent scoping so an agent can only reach the data its function requires. |
| Model boundaries | Providers reached only at the API level under a BAA, with zero-retention or contractually disabled training on submitted data. |
| Audit logging | Immutable, queryable logs of who asked what, which context was retrieved, which model answered, and what was returned — retained for the period compliance requires. |
| Retention and deletion | Defined retention for prompts, outputs, embeddings and logs, with enforced deletion and a documented process for patient-driven requests. |
| Human oversight | A named review step for any output that informs care, with the reviewing clinician recorded in the log. See our AI governance position. |
| Subprocessors | A current, published list of every subprocessor that may touch PHI, with notice before it changes. |
Inference Analytics implements this control set once, in the platform core, rather than per application. Every agent built in Dev Studio inherits it. Our full position, including certifications and data residency, is on the security, compliance and trust page.
Retrieval and agents change the compliance surface
A plain question-and-answer assistant has one PHI pathway. A retrieval-augmented system has several, and an agentic system has more still, because the agent chooses which tools to call and which data to fetch. The compliance surface grows with autonomy.
- Retrieval creates derived copies of PHI in embeddings and caches. Both must sit inside the boundary and inherit its encryption, access control and retention.
- Tool use means an agent can reach systems the original prompt never named. Scope each tool to the minimum necessary and log every invocation.
- Memory persists PHI across sessions. Treat any memory store as a PHI repository with its own retention schedule.
- Multi-step autonomy compounds error. Bound what an agent may do without a human in the loop, and see agentic AI for healthcare for where that line reasonably sits today.
How to evaluate a vendor's claim
Most procurement cycles stall because the questions are too general to disprove. These are specific enough to be answered — or conspicuously not answered.
- Will you sign a BAA, and does it cover every subprocessor that can access PHI, including the model providers?
- Is provider-side training on our submitted data contractually disabled, and can you show the clause?
- Are embeddings and caches derived from PHI encrypted and retained under the same schedule as the source data?
- Can you produce, for a named user and date range, every prompt, retrieval, model call and output?
- What exactly can an agent do without human review, and where is that boundary configured?
- Which certifications do you hold, what is their scope, and when were they last assessed?
The full version of this checklist, covering governance, model flexibility, citation accuracy, integration and scalability as well as security, is in our vendor-neutral healthcare AI platform evaluation guide.
Frequently asked questions
Is ChatGPT HIPAA compliant?
The consumer version of ChatGPT is not appropriate for protected health information: there is no business associate agreement covering consumer use, and PHI entered there leaves the health system's compliance boundary. Enterprise arrangements with model providers can be covered by a BAA, but compliance still depends on the surrounding deployment — access control, logging, retention and oversight — not on the model itself.
Can generative AI be used with PHI at all?
Yes. HIPAA permits the use of PHI for treatment, payment and healthcare operations, including through business associates under contract. Generative AI is not excluded. What is required is that the deployment implements the administrative, physical and technical safeguards HIPAA specifies, and that every party with access to PHI is covered by a business associate agreement.
Do we need a BAA with the model provider, or just the platform vendor?
With both, if the model provider can access PHI. A platform that sends prompts containing PHI to a third-party model is disclosing PHI to that provider, which makes the provider a business associate. Ask your platform vendor to show the chain of agreements rather than assuming their BAA covers it.
Are vector embeddings of clinical notes considered PHI?
Treat them as PHI. Embeddings are derived from identifiable health information and, in practice, information can be recovered or inferred from them. The defensible position is to hold vector stores inside the same compliance boundary as the source data, with the same encryption, access control and retention.
How is InferAgents HIPAA-compliant?
InferAgents implements the HIPAA safeguards once in its platform core, and every agent built on it inherits them: PHI stays inside the compliance boundary, model providers are reached only at the API level under a business associate agreement with training on submitted data disabled, access is governed per user, role and agent, and every prompt, retrieval, model call and output is written to an auditable log with a defined retention schedule.