Voice Assistants You Can Build with Local AI (2026 Guide)

Published: August 8, 2026 — The voice-AI stack went fully local, and it's better than the hype suggests. Whisper hears you, a local LLM thinks, Piper speaks — all on your hardware, no cloud. A 3–5 second end-to-end response on a laptop, ~10s on a Pi, with zero audio ever leaving the room. This guide builds the complete assistant in ~80 lines of Python.

⚡ Quick Takeaways

The Stack

Stage Tool Notes
Speech-to-text faster-whisper (Whisper-small/base) Faster than real-time on laptop; base model for Pi
Reasoning Qwen3-8B / Qwen3.5-4B via Ollama 4B keeps latency low; 8B for quality
Text-to-speech Piper or Kokoro Fast, natural, multi-voice; <0.5s
Wake word openWakeWord Local, near-zero CPU when idle

The ~80-Line Assistant

import sounddevice as sd
import numpy as np
from faster_whisper import WhisperModel
import requests

stt = WhisperModel("small", device="auto", compute_type="int8")

def hear():
    audio = sd.rec(int(5 * 16000), samplerate=16000, channels=1)
    sd.wait()
    text, _ = stt.transcribe(np.squeeze(audio).astype("float32"))
    return text.strip()

def think(question):
    r = requests.post("http://localhost:11434/api/generate", json={
        "model": "qwen3:8b",
        "prompt": f"Answer briefly, as a voice assistant: {question}",
        "stream": False,
    })
    return r.json()["response"].strip()

def speak(text):
    import subprocess
    subprocess.run(["piper", "--model", "en_US-lessac-medium",
                    "--output_file", "out.wav"],
                   input=text.encode(), check=True)
    sd.play(*sd.read("out.wav")); sd.wait()

while True:
    q = hear()
    if not q: continue
    print("You:", q)
    speak(think(q))

Add a wake-word gate around hear(), stream the LLM tokens into TTS for faster first-word response, and you've built the core of every commercial voice assistant — locally. For tools and actions, route think() through the tool-calling loop.

Latency Targets (2026, Realistic)

Hardware STT LLM (short answer) TTS End-to-end
Laptop with GPU ~1s 1–3s <0.5s 3–5s
Apple Silicon 16GB ~1s 1–4s <0.5s 3–6s
Pi 5 8GB 2–3s 2–6s (4B model) ~1s 6–10s

Streaming changes the feel: emit the LLM's first sentence while it generates the rest. For big speedups per stage, the speed guide applies throughout.

Why Local Voice Matters

A cloud voice assistant records your audio, transcribes it, and processes it on someone else's servers — by definition. A local one never produces a network packet. For conversations that must not leave the room — client calls, medical notes, sensitive brainstorming — this is the difference between a tool and a liability. It's also the natural input layer for a fully offline workspace.

🚀 Beyond chat: voice + tools

Give the assistant tools (via tool calling or MCP) and the voice assistant becomes an operator: "file that summary under Q3 reports", "what's in the knowledge base about clause 4?" — grounded by local RAG. That's the full local voice agent, and every piece runs on your hardware.

Frequently Asked Questions (FAQ)

Can I build a voice assistant that works offline?

Yes — the full stack is local in 2026: Whisper-family models for speech-to-text, any local LLM (Qwen3, Gemma) for reasoning, and Piper/Kokoro for text-to-speech. A laptop or even a Raspberry Pi 5 runs the whole pipeline with no network calls.

What is the best local speech-to-text model?

The Whisper family (faster-whisper for speed) remains the default — Whisper-small runs faster than real-time on a laptop, Whisper-base on a Pi. For multilingual including Dari/Pashto, larger Whisper variants and newer multilingual STT models are stronger; test on your languages.

How much latency can I expect?

On a laptop with a GPU: ~1s for STT, 1–3s for LLM generation (short answers), <0.5s for TTS — roughly 3–5s end-to-end, comparable to many cloud assistants. On a Pi 5, add a couple seconds per stage; keep answers short.

What do I need for a local voice assistant?

A microphone, speakers, and compute: 16GB RAM laptop or Pi 5 8GB is the minimum useful; a GPU makes it feel native. Software: faster-whisper (or whisper.cpp), a local LLM via Ollama, Piper/Kokoro TTS, and a small Python glue script. The code in this guide is the whole app.

Can a voice assistant have a wake word locally?

Yes — lightweight wake-word engines (like openWakeWord and Porcupine's offline modes) run locally at near-zero cost. They listen continuously on the CPU while the heavier STT+LLM pipeline only starts after the wake word, keeping idle power low.

Why build a local voice assistant instead of using Alexa or Siri?

Privacy — no audio ever leaves the device (see the data-privacy guide), full control over behavior and tools, no subscription, and it works with zero internet. The trade-offs: less polish, fewer integrations, and smaller models than the cloud giants.

Sources & Further Reading