Why Local Translation Now
- Privacy: contracts, medical records, and internal documents are exactly what must not enter a cloud translator — the risk analysis applies word for word.
- Cost: after hardware, translation is free. Document-heavy workloads that would cost thousands per month in API fees cost nothing.
- Glossary control: cloud MT ignores your terminology; a local pipeline enforces it.
- Offline: the pipeline works in field offices, basements, and air-gapped environments.
The Stack
| Layer | Choice |
|---|---|
| Model | Qwen3.5-9B via Ollama (see Dari/Pashto coverage); NLLB for dedicated MT |
| Document extraction | PyMuPDF / python-docx / plain text — same parsing as multimodal RAG |
| Glossary | CSV or JSON: source term → target term, per domain |
| QA | COMET/BLEU for regressions + native review for release |
The Pipeline
# glossary.csv: source, target
# "termination clause","بند ختم قرارداد"
# "force majeure","قوه قهریه"
import json, requests
def load_glossary(path="glossary.csv"):
return {src: tgt for src, tgt in
(line.strip().split(",") for line in open(path))}
GLOSSARY = load_glossary()
def translate(text, src="en", tgt="fa", context=""):
entries = "\n".join(f"{k} → {v}" for k, v in GLOSSARY.items())
prompt = f"""Translate from {src} to {tgt}.
TERMINOLOGY — must use these translations exactly:
{entries}
Previous section (for consistency): {context}
Text: {text}"""
r = requests.post("http://localhost:11434/api/generate", json={
"model": "qwen3.5:9b", "prompt": prompt,
"options": {"temperature": 0.2}, "stream": False})
return r.json()["response"].strip()
# Document-aware loop: translate in order, pass prior context
sections = extract_sections("contract.pdf") # your parser
out, prev = [], ""
for sec in sections:
t = translate(sec, context=prev)
out.append(t); prev = t
save("contract-fa.txt", "\n\n".join(out))Three design decisions carry the quality: low temperature (0.2 — translation is not creative writing), glossary injected into every prompt (terms never drift), and previous-section context (voice and terminology stay consistent across the document).
Evaluating Quality Honestly
| Stage | Method | Use |
|---|---|---|
| Regression | COMET/BLEU on a fixed test set | Fast signal when you change models or prompts |
| Release | Native-speaker review (fluency, fidelity, terminology) | The only metric that matters for shipping |
| Terminology audit | Script: does every glossary term appear correctly? | Catch glossary violations automatically |
For Pashto and Dari, native review is non-negotiable — machine metrics understate the dialectal issues. For major languages, local LLM translation with a glossary is close enough to professional for most internal work.
Scaling the Pipeline
- Batch through vLLM when volume grows — the Ollama vs vLLM guide covers the throughput upgrade.
- Parallelize by document, not by sentence — document context matters; keep each document's sections in order.
- Cache translations — translated sections keyed by source hash make re-runs instant.
- Pair with TTS for voice output in the target language — the full multilingual assistant.
🚀 The pattern in production
This is the exact engine behind the Lawyer Assistant language story: glossary-locked legal terminology, local model, zero document egress. When translation carries legal meaning, consistency and privacy aren't nice-to-haves — they're the product.
Frequently Asked Questions (FAQ)
Can local LLMs translate as well as cloud services?
For major languages, close — a 9B Qwen model translates formal and technical text well, and beats generic MT when you add glossaries and context. For low-resource languages like Pashto, it's usable but needs human review. The wins over cloud: privacy (documents never leave the machine), no per-word cost, and glossary control.
What is a translation glossary and why does it matter?
A glossary locks domain terminology — legal, medical, or product terms must always translate to the approved target term. Without it, LLMs translate terms inconsistently across a document. With it, you get enterprise-grade consistency: the single biggest quality win for professional translation.
Which model should I use for local translation?
Qwen3.5-9B (or Qwen3-8B) is the 2026 default — strong multilingual coverage including Dari/Pashto. For dedicated translation tasks, NLLB remains a solid specialized option. Larger Qwen models (27B+) improve quality further if your hardware allows.
How do I evaluate translation quality?
The honest way: a test set scored by native speakers (fluency + fidelity + terminology consistency). Machine metrics (BLEU, COMET) give quick signals but miss idiomatic quality — use them for regression testing, native review for release decisions. The pipeline in this guide separates both stages.
Can the pipeline translate documents, not just sentences?
Yes — the best pattern is document-aware: extract text (or read the PDF/Word directly), translate section by section with context from the previous section (consistency), then reassemble. Long documents translate better with chunk context than sentence-by-sentence.
Is translation privacy-sensitive?
Often extremely — contracts, medical records, and internal documents are exactly what you should not paste into a cloud translator. A local pipeline keeps everything on your machine, which is the whole point of the privacy-cluster guides on this site.