Why Legal Research Can't Trust the Cloud
Legal work has a confidentiality problem that most AI tools ignore. The moment a document leaves your machine and hits a cloud API, you've handed a third party the very material that may be protected by attorney–client privilege. For law firms, in-house counsel, and solo practitioners, that single fact rules out most "AI legal research" products on the market.
There's a second, quieter problem: verification. Generic chatbots answer from memory. They will confidently summarize a contract that isn't in front of them, invent a statute, or blend two clients' facts into one answer. An opinion is not a sourced answer.
Lawyer Assistant was built to solve both problems at once: keep every document on the user's machine, and never answer without showing the exact source it relied on.
What Lawyer Assistant Does
The app is free, open source, and runs on Windows, macOS, and Linux. Point it at a folder of PDFs, scans, contracts, and notes, and it gives you three things:
📎 Answers with Citations
Every claim is grounded in retrieved passages. Inline citations link straight to the source file, so you can verify before you rely on it.
📋 Compliance Playbook Scan
Define your own rules — jurisdictional, formatting, disclosure — and scan any document against them. Violations are flagged, rated, and explained.
🔎 Hybrid Search Engine
Semantic understanding and exact keyword matching, fused and re-ranked. Legal shorthand, typos, and paraphrases all find their target.
Three Phases: Ingest → Ask → Verify
- Phase 1 — Ingest. Point the app at a project folder. Your PDFs, scans, contracts, and notes are read, OCR'd if needed, chunked, and indexed — locally.
- Phase 2 — Ask. Type a question in plain English. The planner routes it — search your documents, scan for compliance, or both — and streams its reasoning live.
- Phase 3 — Verify. Read the answer alongside inline citations. Open any source to inspect the exact passage, then star it, pin it, or move on.
Inside the RAG Pipeline
At its core, Lawyer Assistant is a reference-grade retrieval-augmented generation system. The whole flow — a full ingest → index → retrieve → generate loop — runs on-device:
1. Ingest & Indexing
Documents are parsed with OCR when they're scanned images, then split into chunks sized for retrieval. Each chunk is embedded with BGE-M3, a multilingual embedding model, and stored in ChromaDB, the local vector store. Because indexing happens on your machine, there is no moment where your documents exist anywhere but your disk.
2. Hybrid Retrieval: BM25 + Vectors
Legal language is unforgiving. A contract clause may use exact boilerplate phrasing that a pure semantic search would paraphrase away, while a casual question may use words that never appear in the document. Lawyer Assistant therefore runs two retrievers in parallel:
- BM25 keyword search — catches exact terms, clause citations, and legal shorthand
- Semantic vector search — catches paraphrase and meaning, so "what happens if the other party breaches" finds a clause about "default by counterparty"
The two result sets are fused and re-ranked into a single ranked list. This is the difference between searching a library by index card and searching it by meaning — and the app does both at once.
3. Grounded Generation with Citations
The retrieved passages are passed to the LLM as grounding context. The model is instructed to answer only from that context and to attach inline citations that link back to the exact page and paragraph numbers. That's what makes every answer verifiable: the citation isn't decoration, it's the contract between the user and the answer.
4. Agent Loop & Streaming
Queries are routed by a lightweight planner: search your documents, scan for compliance, or both. The loop supports iterative retrieval — if the first pass doesn't find enough evidence, the agent refines and retries. Responses stream out over REST + SSE (Server-Sent Events), so you see the reasoning as it's built rather than waiting for a wall of text.
🧩 Multi-model orchestration. Embeddings, rerankers, and the LLM run as separate services. That separation is deliberate — you can swap the reranker, change embedding models, or plug in a different generator without touching the retrieval layer.
Tech Stack at a Glance
| Layer | Technology | Role |
|---|---|---|
| Embeddings | BGE-M3 | Multilingual embeddings for semantic retrieval |
| Vector Store | ChromaDB | Local vector index of document chunks |
| Keyword Search | BM25 | Exact-term retrieval for legal shorthand and citations |
| Document Parsing | OCR | Reads scanned PDFs and images into searchable text |
| Local LLM | Ollama (Llama · Mistral · Gemma) | On-device generation — fully offline mode |
| Cloud Option | ChatGPT · Claude | Optional providers when an internet connection is acceptable |
| API Layer | REST + SSE | Streaming responses and agent tool calls |
Privacy by Design, Not by Policy
Most products promise privacy in a terms-of-service page. Lawyer Assistant earns it in the architecture:
- No cloud path required. With a local Ollama model, the app is fully functional offline — embeddings, search, chat history, and compliance scans all run locally.
- No account, no telemetry. There is nothing to sign up for and nothing phoning home. No question, document, or answer ever leaves your machine in local mode.
- The internet is opt-in. You only need a connection if you deliberately choose a cloud API provider instead of a local model.
🔒 Bottom line: attorney–client privilege is a design goal, not a checkbox.
Beyond Chat: The Compliance Playbook Scan
A search-based assistant answers questions you ask. A compliance scan asks questions for you. Define a playbook — your firm's jurisdictional requirements, formatting rules, or disclosure obligations — and scan any document against it. Each violation is flagged, rated, and explained, which turns a 200-page contract review from a weekend of reading into a structured list of risks with their locations.
This is where the hybrid retrieval really earns its keep: a playbook rule about "confidentiality obligations" must match boilerplate that may say "non-disclosure covenant" — which is exactly the semantic-keyword fusion the pipeline was built for.
Open Source & a Learning Resource
Lawyer Assistant is free and MIT-licensed, and it doubles as a reference implementation for anyone building RAG systems on local models. The repo also powers a featured example on Local AI Zone, where you can study the wiring: the full ingest → index → retrieve → generate flow, multi-model orchestration, the agent loop, hybrid retrieval, grounding with citations, and streaming responses — all running on-device.
Getting Started
⬇️ Free & open source — no account, no sign-up
- Windows: setup installer for Windows 10 & 11
- macOS: universal archive (extract and run
launch.sh) - Linux: AppImage — no install needed
Frequently Asked Questions (FAQ)
What is Lawyer Assistant?
Lawyer Assistant is a free, open-source, local-first AI legal research app. You point it at a folder of your own legal documents, ask questions in plain English, and it answers with inline citations grounded in the exact passages it retrieved — 100% on your computer, with no account, no cloud, and no telemetry.
What is a RAG pipeline?
Retrieval-Augmented Generation (RAG) is an architecture that retrieves relevant passages from a document store first, then feeds them to an LLM so answers are grounded in real sources instead of the model's memory. The flow is ingest, index, retrieve, then generate.
How does Lawyer Assistant keep legal data private?
Everything — ingesting, indexing, searching, and answering — runs on your own computer. With a local Ollama model the app is fully functional offline, so no question, document, or answer ever leaves your machine. Attorney-client privilege is a design goal, not a checkbox.
What tech stack does Lawyer Assistant use?
BGE-M3 for embeddings, ChromaDB as the vector store, BM25 for keyword search, OCR for scanned documents, and Ollama for local LLM inference (Llama, Mistral, Gemma). Retrieval is hybrid: BM25 keyword matching fused and re-ranked with semantic vector search. It also supports optional cloud APIs like ChatGPT or Claude.
Does Lawyer Assistant run on my operating system?
Yes. It ships as a Windows installer (.exe), a macOS universal archive, and a Linux AppImage — free and open source, with no sign-up required.
Can I scan documents for risky clauses?
Yes. The compliance playbook scan lets you define your own rules — jurisdictional, formatting, disclosure — and scan any document against them. Violations are flagged, rated, and explained.