Air-Gapped AI: Running LLMs Fully Offline with Ollama

Why air-gapped is a deployment shape, not a feature flag

Most "self-hosted" AI products still leak. A model API call here, an embedding request there, a telemetry ping, a license check that phones home - each one individually defensible, and together they mean your internal documents, prompts, and answers cross an internet boundary you do not control. For defense contractors, hospitals, banks, and any organization with a data-residency mandate, that is disqualifying. The requirement is not "mostly offline" - it is an AI system that functions with the network cable pulled.

Simmetric Chat is built around that requirement. Air-gapped operation is not a mode you unlock or a plugin you buy: the community platform's default configuration is already the offline one - a local LLM, local embeddings, a file-based local vector store, and a document pipeline that never needs the internet. This article walks through each layer that makes zero-outbound connectivity work, and what you give up - very little - by closing the perimeter.

The local stack: every layer has an offline default

An AI chat has four external dependencies in a typical SaaS architecture: the language model, the embedding model, the vector store, and the document pipeline. Simmetric Chat's defaults replace all four with local components:

• The LLM runs in Ollama. The platform's default provider is Ollama on the local network, and the default model (gemma4:latest) is pulled at setup time so the chat works before any model download happens at runtime. Ollama speaks OpenAI-compatible HTTP on your own network, and the provider configuration points at it with a URL - not a cloud endpoint.
• Embeddings run locally. The default embedding provider is a local transformers model (384 dimensions, running in-process in the collector), not a hosted embeddings API. Local embeddings are air-gap compatible by construction, and the embedding-mismatch guard means switching between local and remote providers later is safe - retrieval degrades to keyword search until re-indexing completes rather than serving scores from mixed spaces.
• Vectors live in LanceDB. The default vector store is file-based, stored in the deployment's own storage directory, with a collection per workspace. No vector database server, no network hop. Qdrant, pgvector, and Chroma remain available for teams whose scale calls for a networked index, but nothing requires them.
• Document parsing happens in your network. The collector - the service that parses, chunks, and embeds uploads - is part of your deployment. PDFs, Word and Excel files, presentations, and YouTube transcripts are processed inside the perimeter, including scanned documents routed through the server-side vision-model OCR pipeline, which uses a local vision model.

Nothing in that list is special-cased or bolted on. The strategy pattern at each boundary means the local option is just the default provider, selected by configuration like any other.

DLP: redaction before anything leaves - or before it moves at all

Closing outbound connectivity does not close every exfiltration path, because content moves inside your network too: prompts enter the platform, answers stream out to users, and both pass through chat interfaces that people copy from. Simmetric Chat's data-loss-prevention filter scans both directions - the inbound prompt and the outbound answer, including token-by-token streaming flushes - and redacts matches before content is stored or displayed.

The scanner recognizes the PII classes that matter in a chat context: email addresses, credit card numbers, API keys and private keys, and - added in v0.22 - structured European identifiers: Italian Partita IVA, codice fiscale, and IBAN, seeded as built-in patterns that are idempotent and label-anchored so ordinary numbers never trigger them. Pattern configuration is admin-managed and database-backed: enable or disable any pattern, add up to fifty custom patterns with live compile validation and a test preview, and inspect a match history panel filtered by source.

The design intent is that DLP is the last line, not the first. The first line is architecture: with local models, local embeddings, and local vectors, there is no third-party endpoint for sensitive content to reach in the first place. DLP then covers the paths that remain - content that leaves the platform toward your own users, or that a well-meaning employee pastes into a prompt. It also covers the widget surface, so visitor-facing conversations on your website are redacted by the same filter chain, with the audit trail distinguishing chat from widget traffic.

The single-container deployment: everything on one host

Closing the perimeter is one thing; shipping the software into a perimeter is another. Air-gapped environments typically cannot run a five-service compose stack comfortably - they run one host with one image, and the fewer the moving parts, the fewer the artifacts that must be vetted and transferred.

Simmetric Chat ships a single-container all-in-one image for exactly this case. One container, supervised by a process manager, runs everything: PostgreSQL 16 (with the pgvector extension compiled from source at image build time, so it is installed offline), the API server, the document collector, and the widget service. The container exposes three ports - 3000 for the API, 3210 for the collector, 3211 for the widget - and healthchecks its own /api/health endpoint, which reports database, collector, and disk status.

Getting the image in is the same discipline as getting any artifact in: transfer the image tarball (or the registry mirror you vet), load it, and run it. The environment defaults in the image are scaffolding placeholders - production overrides JWT_SECRET, COLLECTOR_SECRET, and the encryption key through environment configuration, and the server fails loud at boot if production secrets are missing, so a half-configured air-gap deployment never silently starts. After boot, software updates are new image versions, not package installs - no internet access required at any point of the lifecycle.

# Transfer into the perimeter, then:
docker load -i simmetric-chat-all-in-one.tar
docker run -d \
-p 3000:3000 -p 3210:3210 -p 3211:3211 \
-v simmetric-storage:/app/storage \
-e JWT_SECRET=$(openssl rand -hex 32) \
-e COLLECTOR_SECRET=$(openssl rand -hex 32) \
simmetric-chat:all-in-one

Model weights follow the same path. Ollama models are pulled on a connected machine, saved as blobs, and imported inside the perimeter - the platform then treats them like any other local model. Nothing in the stack phones home to verify a model, a license, or itself: telemetry is disabled by default, there is no external error-reporting dependency, and license validation is a local RS256 signature check against an embedded public key.

What still works offline - and what is genuinely absent

An air-gapped deployment should be honest about its edges. What works with zero outbound connectivity:

• Chat with local LLMs through Ollama, streamed token-by-token over SSE
• Document upload and ingestion - parsing, chunking, local embeddings, LanceDB vectors
• Hybrid retrieval with citations, metadata filtering, and synthesis pipelines
• RBAC, workspaces, projects, audit trails, and the full admin surface
• Webhooks (delivered to your own receivers), Web Push (VAPID from your own server), and API keys
• The embeddable widget, running against your own widget service

What is genuinely absent: cloud LLM providers, hosted embeddings, external web search, and anything else that requires an outbound call. The graceful-degradation design means every one of those integrations is a configuration add, not a code change - but the offline default ships with them simply not present. The platform boots, serves requests, and processes documents identically whether the perimeter is open or closed.

The result is an AI workspace whose security posture does not depend on trust in a vendor's data handling, because there is no vendor in the data path. Everything that touches your documents - the model, the embeddings, the vector index, the parser - runs on hardware you own, inside a network you control, in a container image you can audit.

Read the full guide in the docs →