Server-Side Vision-Model OCR for Scanned Documents

Why scanned documents break text extraction

Ask a text extractor to read a scanned PDF and you get back almost nothing. A PDF produced by a word processor carries a text layer - the characters are stored as text, and extraction is trivial. A PDF produced by a scanner is an envelope of page images: there is no text layer at all, just pixels. The collector's parser can only extract what is actually there, so image-only pages come back empty or nearly empty, and a document that looks perfectly fine in a viewer quietly contributes nothing to your knowledge base.

Simmetric Chat handles this on the server, before a document ever reaches chunking and embedding. When ingestion detects a text-poor PDF, the platform routes it through a dedicated OCR pipeline whose engine is not a classical pattern-matching OCR library but a vision-capable large language model served locally through Ollama. The result is extracted Markdown that preserves headings, lists, and table structure far better than character-level OCR - and it runs entirely inside your own infrastructure. No document ever leaves your network to be read.

This article walks through how the pipeline works, which knobs it exposes, and the human approval step that keeps a misreading model from quietly poisoning a knowledge base.

From pixels to Markdown: the pipeline

OCR in Simmetric Chat is a job-based, server-side pipeline. An upload whose extracted text fails the precheck - by default, fewer than about 200 extractable characters marks a document as a scan - becomes an OCR job, and a polling scheduler picks jobs up every 10 seconds with a global concurrency limit, so a batch of uploads cannot saturate the machine. One document then flows through five stages:

1. Load. pdfjs-dist opens the PDF server-side and enumerates its pages, capped at 50 pages per job as a safety limit against runaway processing.
2. Render. Each page is rendered to a PNG image at an enlarged scale - 2x by default - so small print in a fax-quality scan still arrives at the model with readable resolution.
3. Recognize. Every page image is sent to the configured Ollama vision model as a base64 image, and the model transcribes the page as Markdown text.
4. Guard. The raw transcription passes through a hallucination guard and a grounding cleanup step. Vision models sometimes annotate their output with grounding markers, or invent plausible content when a page is genuinely illegible - both are caught here instead of being stored.
5. Score and assemble. Each page receives a quality score, the pages are concatenated into one document, and the job completes with a truthful summary: pages that failed are counted and reported as failed pages, never silently absorbed into a clean "completed" result.

Every step in that list happens on the server. The browser never runs OCR, and no third-party OCR API is called - the only network hop is from your server to your own Ollama instance.

Choosing the model

The vision model is configurable, and the pipeline ships with a small model registry that knows how to talk to each supported family:

• glm-ocr - the default (OCR_MODEL=glm-ocr:latest), an OCR-specialized vision model. The registry routes it through Ollama's chat endpoint, which detects end-of-transcription more reliably for this model family.
• deepseek-ocr - a second OCR-tuned family that uses grounding tokens; the registry sends it single-page images and strips its grounding markers during cleanup.
• Any generic vision model - models the registry does not recognize fall back to a generic prompt through Ollama's generate endpoint, so a multimodal model you already run locally works too.

The model-related knobs are environment variables, documented alongside the rest of the configuration surface:

# Server-side vision OCR
OCR_MODEL=glm-ocr:latest   # vision model (supports model:version)
OCR_TIMEOUT=600000         # 10 minutes - vision models are slow
OCR_NUM_PREDICT=8192       # max output tokens per page

A prewarm endpoint lets an administrator load the model into memory ahead of the first job, so the first scanned upload of the day does not pay the model's cold-start cost.

The human in the loop: approve or reject

Transcription quality varies with scan quality, and a knowledge base is the wrong place to discover that a model misread a page. The pipeline therefore treats OCR output as a draft, not a fact. Every job produces a reviewable artifact: the preview keeps the extracted text bound to the same page slice as the page-image pane, so a reviewer can spot-check any page against the scan it came from.

From the OCR settings panel an administrator sees the model catalog, the job list with per-page quality and credibility scores, and the workflow itself:

• Approve - the extracted pages are accepted and flow onward: an approved archive job materializes its pages into the knowledge base, and a synthesis run can be triggered from the approved result.
• Reject - the output is discarded and the decision is recorded on the job.

Automation is available without giving up the gate. An event-driven auto-approve hook can finalize jobs whose quality signals are clean, and it deliberately preserves the manual approve flow wherever a human decision is the contract - an import explicitly routed to a knowledge base is never silently auto-approved. Either way, job summaries stay truthful about page health: a job that finished with three failed pages says so in its result and in the logs.

The point of the ceremony is one invariant: text enters your RAG index only after either a clean quality pass or a human looked at it.

Where the extracted text goes

Approved OCR output rejoins the normal ingestion flow. For uploaded documents, the extracted Markdown replaces the text layer the scan was missing, and the document continues through chunking, embedding, and hybrid search indexing exactly like a native PDF. For archives, approved pages become part of the workspace's curated knowledge base, where wikilinks, backlinks, and the synthesis pipeline can operate on them like any other page.

Because the pipeline is server-side, it composes with the rest of the platform's guarantees: DLP filters still scan content before it leaves the network, RBAC still gates who can trigger jobs and approve results, and nothing in the OCR path requires an outbound connection to a cloud OCR service.

Practical settings

Most deployments need only three decisions:

• Enable it. OCR_ENABLED and OCR_PRECHECK_CHARS - the character threshold that routes a PDF into the pipeline - are managed in the admin settings.
• Pick the model. glm-ocr:latest is a sensible default; the model catalog lists what the registry recognizes, per-job model choice is available from the settings panel, and custom instructions can tune transcription behavior for your document domain.
• Budget the time. Vision OCR trades speed for fidelity. The 10-minute default timeout and the 50-page cap exist so a pathological scan degrades into a visible failure rather than a hung worker.

If your corpus is mostly native PDFs you will rarely notice the pipeline: healthy text layers bypass OCR entirely. The moment a scanned document arrives, it goes through rendering, the vision model, and your approval - and ends up searchable instead of invisible.

Read the full guide in the docs →