Start With the Four Questions
Every compliance framework — HIPAA, SOC 2, GDPR, and the EU AI Act on top — is asking variations of the same four things:
- Where does the data live? Your servers, a US cloud, an EU cloud? The answer determines which laws even apply.
- Who can access it? Role-based access, unique user identification, least privilege — the frameworks all want named, controlled access.
- What gets logged? Audit trails of who did what, when — the evidence that answers every auditor's first question.
- When does it get deleted? Retention rules differ — HIPAA wants six years, SOC 2 typically one, GDPR varies by lawful basis — but all require a documented policy.
Answer those four well, and you've covered most of what all three frameworks demand. The frameworks differ in vocabulary, not in substance.
The Three Frameworks, Plainly
🩺 HIPAA
Protects health information. Any vendor touching PHI needs a business associate agreement; access needs controls and audit trails; records need six-year retention. AI that processes PHI is inside this regime — no exceptions.
📋 SOC 2
An audit framework for service organizations: security, availability, and confidentiality controls, documented and tested. The product is the evidence. For AI, that means access management, logging, and incident response that an auditor can verify.
🇪🇺 GDPR
Governs personal data: a lawful basis for processing, data minimization, storage limitation, and defensible cross-border transfers. The EDPB's Opinion 28/2024 adds that AI models trained on personal data can't automatically be called anonymous — and downstream controllers stay responsible.
And since August 2, 2026, the EU AI Act layers on top: risk classification, transparency, and documentation duties for AI systems. It doesn't replace the GDPR — it sits beside it, pointing at the same workloads.
Where Cloud AI Trips Over These Frameworks
| Framework demand | Cloud AI problem |
|---|---|
| Where does data live? | Prompts go to a third party — possibly across borders, possibly US providers subject to the CLOUD Act |
| Who can access it? | The provider's engineers, contractors, and regulators can access your data under their terms |
| What gets logged? | You get a usage bill, not an audit trail of your own data's handling |
| When does it get deleted? | The provider's retention policy governs — often longer than your regulation allows |
Why On-Premise AI Answers All Four at Once
Run the pipeline inside your boundary and the four questions answer themselves:
- Where does data live? On your servers. No transfer, no CLOUD Act reach, no third-party processing.
- Who can access it? Your IAM system, your roles, your rules — integrated with your existing identity stack.
- What gets logged? Your logging pipeline, feeding your SIEM, retained per your policy.
- When does it get deleted? Your retention policy, enforced by your pipeline, on your schedule.
That's why regulated sectors lead the on-premise shift. The compliance work doesn't disappear — it becomes work you already know how to do, applied to a system you fully control.
💡 The deep dives: Offline AI for Regulated Industries covers the sector-by-sector case; GDPR-Compliant AI in 2026 covers the GDPR side; Private RAG for Regulated Industries is the deployment playbook.
Frequently Asked Questions (FAQ)
What is HIPAA's requirement for AI?
HIPAA governs protected health information: business associate agreements with any vendor touching PHI, access controls, audit trails, and six-year retention. AI that processes PHI must sit inside that regime — or avoid processing PHI entirely.
What does SOC 2 require of AI?
SOC 2 is an audit framework for service organizations covering security, availability, and confidentiality controls. For AI it means documented controls: access management, logging, incident response, and typically one-year retention. Evidence of control is the product.
How does GDPR apply to AI?
GDPR governs personal data processing: a lawful basis, data minimization, storage limitation, and defensible cross-border transfers. The EDPB's Opinion 28/2024 adds that AI models trained on personal data can't automatically be considered anonymous, and downstream controllers remain responsible.
How do these frameworks overlap for AI?
They all reduce to the same core demands: know where data lives, control who accesses it, log what happens, and delete it on schedule. On-premise AI answers all three at once because the entire pipeline stays inside your boundary.
Does the EU AI Act add another layer?
Yes. The AI Act became applicable on 2 August 2026 for most obligations, adding risk classification, transparency, and documentation duties on top of the GDPR. It doesn't replace HIPAA, SOC 2, or GDPR — it layers on.
🏛️ Need a compliance-ready AI deployment?
I design and deploy on-premise AI for regulated industries — private RAG, sovereign infrastructure, and compliance-first architecture through Haal Lab. Contact me for a scoping conversation.