What Air-Gapped AI Is (and Isn't)
An air gap is a physical separation: the machine has no route to the internet — no NIC to the outside, no WiFi, no Bluetooth bridging, no USB devices with radios. Data cannot leave because there is no path to leave by.
- It is not "private mode" in a cloud app, encrypted API calls, or a VPN. Those still transit someone else's infrastructure.
- It is not a compliance exemption. It's the strongest technical posture — GDPR-style accountability still applies (see the GDPR guide).
- It is the standard for the environments where data loss is not a risk category but a national-security event: defense, intelligence, critical infrastructure, and hard-confidential legal work.
The Transfer Problem: Getting Models In
The central practical question: how do models get onto a machine that never touches the internet? Answer: a controlled, reviewed import pipeline.
- Download on a connected machine. Pull GGUF files from Hugging Face or the model's official repo. Prefer the RAM-ranked picks sized to the target hardware.
- Verify integrity. Record SHA-256 hashes; check them again on the air-gapped side before loading. This is the anti-tamper step — never skip it.
- Copy via approved media. A dedicated, encrypted, scanned USB drive (or burned disc for stricter environments). One-way transfer is the ideal: the import machine has no read-back path.
- Load locally. llama.cpp reads GGUF directly from disk. Ollama imports: create a Modelfile pointing at the local file, then
ollama create mymodel -f Modelfile.
# On the connected machine: download + hash
wget https://huggingface.co/Qwen/Qwen3-8B-GGUF/resolve/main/qwen3-8b-q4_k_m.gguf
sha256sum qwen3-8b-q4_k_m.gguf > qwen3-8b.sha256
# On the air-gapped machine: verify then import
sha256sum -c qwen3-8b.sha256
# Ollama import from local GGUF
printf 'FROM ./qwen3-8b-q4_k_m.gguf\n' > Modelfile
ollama create qwen3-8b -f ModelfileDependencies travel the same way: Python wheels, Docker images (docker save → transfer → docker load), and the embedding models your RAG pipeline needs.
The Full Offline Stack
| Component | Air-gapped choice | Notes |
|---|---|---|
| Inference | llama.cpp (or Ollama) | Zero phone-home in llama.cpp; block Ollama's update checks |
| Chat UI | Open WebUI / LobeChat (self-hosted) | Runs entirely inside the enclave |
| Embeddings + vector store | BGE-M3 + ChromaDB/FAISS | Fully local, no telemetry |
| Voice | Whisper + Piper/Kokoro TTS | Local STT/TTS, no cloud |
| Code assistant | Continue → local llama.cpp endpoint | Offline autocomplete/chat |
| OCR / documents | Tesseract / local document parsers | Pre-installed, no API OCR |
Every component has an offline path in 2026 — the ecosystem finally treats no-connectivity as a first-class deployment mode rather than an edge case.
Which Models Belong in an Air Gap
- License purity first. Permissive or fully open licenses (Apache-2.0, MIT) avoid license-compliance friction: Qwen3, Gemma 4, GLM are the usual candidates.
- Size to hardware. An air-gapped server with one 24GB GPU runs 30B-class; a laptop runs 7–14B. Use the RAM table to choose the exact quant file.
- Frozen is fine. A model snapshot doesn't rot — it's reproducible, which compliance teams love. Freshness comes through scheduled, reviewed update cycles (quarterly is typical).
- Embeddings too. Your embedding model is part of the same import pipeline — and note that changing it later means re-embedding the whole corpus.
The Security Reality Check
⚠️ The air gap is not a shield. The classic attack vectors survive: malicious USB media (the transfer pipeline is the prime injection point), insider access, and unpatched software accumulating risk over time. Treat the gap as one strong control inside a layered defense — scanning, hashing, access control, and monitoring still apply.
- Hash-verify everything imported — the file you scan is the file you run.
- One-way transfer design where policy allows — no read-back from the enclave.
- Patch cycles still matter — route security updates through the same reviewed pipeline.
- Log locally, export on schedule — audit logs that can't leave can't be audited; plan their extraction.
Frequently Asked Questions (FAQ)
What is an air-gapped AI system?
An air-gapped AI system runs models on hardware physically isolated from the internet — no network egress at all. Data never leaves the machine, which is the strongest possible privacy posture and a requirement for classified, defense, and some legal/medical environments.
How do you get models onto an air-gapped machine?
Download GGUF (or HF safetensors) files on a connected machine, verify hashes, copy to an approved USB drive, and load them directly from disk — llama.cpp reads local files, and Ollama imports GGUF via a Modelfile. Package dependencies and Docker images the same way, ahead of time.
Can air-gapped AI be useful without updates?
Yes — models are frozen snapshots, which is a feature for compliance: reproducibility is guaranteed. The trade-off is that model improvements arrive only through deliberate, reviewed transfer cycles. Plan a quarterly or event-driven update pipeline through the same approved channel.
What models should I run in an air-gapped environment?
Prefer permissively licensed, fully open weights so the license itself doesn't create compliance issues: Qwen3, Gemma 4, GLM, and similar (see the open-source ranking in this blog). Size the model to the hardware using the RAM-ranked GGUF guide, and verify licenses before transfer.
Is air-gapped AI slower or worse?
Not inherently — inference speed depends on hardware, not connectivity. The ceiling is model choice: you're limited to open weights, so frontier cloud models are out. For document work, RAG, and domain analysis, modern open models are often entirely sufficient.
Do I still need security controls in an air-gapped environment?
Absolutely — arguably more. Air gaps create a false sense of safety: USB transfers are the classic attack vector, insider threats remain, and unpatched systems accumulate risk. Treat the air gap as one control in a layered defense, not a substitute for the rest.