Why offline, specifically

Most language-practice tools assume a network connection to a cloud API — for the LLM, for speech-to-text, for text-to-speech, sometimes for all three. That’s a reasonable default for a product, but it creates a dependency that doesn’t need to exist for a personal practice tool: every session becomes contingent on an API being up, a key being valid, and a per-request cost. For something used daily, running entirely on local hardware removes all three variables at once, and an M2 Mac Mini has enough unified memory to run a capable local stack without needing a GPU box.

The lab has three moving parts — a conversational model, speech recognition, and speech synthesis — plus a thin HTML interface that ties them together. Each piece is a separate open-source project; the actual engineering is in getting them to talk to each other with low enough latency that a conversation feels like a conversation.

The conversational core: Ollama

Ollama runs quantized LLMs locally and exposes them over a local HTTP API, which makes it the natural hub — everything else in the stack just calls localhost instead of a cloud endpoint.

ollama pull llama3.1:8b
ollama serve

For language practice specifically, the system prompt matters more than the model choice. A prompt that just says “help me practice French” tends to produce a model that lectures rather than converses. What works better is constraining the model’s role explicitly:

SYSTEM_PROMPT = """You are a conversation partner practicing {language} with a learner.
Respond only in {language}. Keep responses to 1-3 sentences.
If the learner makes a grammar mistake, continue the conversation naturally
without correcting them mid-flow — corrections come at the end of the session, not inline."""

The API call itself is a plain POST to the local Ollama server:

import requests

def get_response(user_text, language, history):
    payload = {
        "model": "llama3.1:8b",
        "messages": [
            {"role": "system", "content": SYSTEM_PROMPT.format(language=language)},
            *history,
            {"role": "user", "content": user_text}
        ],
        "stream": False
    }
    r = requests.post("http://localhost:11434/api/chat", json=payload)
    return r.json()["message"]["content"]

An 8B model is deliberately the practical ceiling here on an M2 — it’s fast enough that responses don’t create an awkward pause in a conversation loop, and it’s more than capable of holding a simple, constrained back-and-forth. A larger model would answer more richly but at a latency cost that breaks the conversational feel this is built for.

Turning speech into text: Whisper.cpp

Whisper.cpp is a C++ port of OpenAI’s Whisper model, optimized to run efficiently on Apple Silicon via its Metal backend. This is the piece that listens to what the learner says and turns it into text the LLM can respond to.

./main -m models/ggml-base.en.bin -f recording.wav -otxt

Two things matter for a language-learning context specifically, as opposed to general transcription:

• Model size selection is a real tradeoff. The base model transcribes fast enough for near-real-time use but stumbles on heavily accented speech from a learner still developing pronunciation. The small or medium models are meaningfully more accurate but add latency that’s noticeable in a live conversation loop. In practice, base for live conversation and medium for reviewing a recorded session afterward split the difference.
• Language pinning matters. Whisper auto-detects language by default, which sounds convenient but actively works against a learner speaking imperfect target-language audio — it will sometimes mis-detect broken French as English and transcribe it as noise. Passing -l fr explicitly forces the model to transcribe as French even when the pronunciation is rough, which is exactly the behavior wanted for practice.

Turning text into speech: Piper

Piper is a fast, local neural TTS engine, chosen here specifically because it runs comfortably on CPU without needing the Metal GPU path that Whisper.cpp uses — the two can run side-by-side without contention.

echo "Comment ça va aujourd'hui?" | ./piper \
  --model fr_FR-siwis-medium.onnx \
  --output_file response.wav

Piper’s voice models are per-language, which is a natural fit for a multi-language lab — swapping the --model flag is the entire cost of switching the practice language, with no retraining or reconfiguration needed elsewhere in the pipeline.

Tying it together: the HTML interface

The three pieces above are all command-line tools and local HTTP APIs — none of them have a conversational UI on their own. The custom HTML interface’s job is to manage the loop: record audio, send it to Whisper.cpp, send the transcript to Ollama, send the response to Piper, play the result, repeat.

async function conversationTurn(audioBlob) {
  // 1. Send recorded audio to a local Flask endpoint wrapping whisper.cpp
  const transcript = await fetch('/transcribe', {
    method: 'POST',
    body: audioBlob
  }).then(r => r.text());

  displayUserText(transcript);

  // 2. Send transcript to Ollama, get a reply
  const reply = await fetch('/chat', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ text: transcript, language: currentLanguage })
  }).then(r => r.json());

  displayAiText(reply.text);

  // 3. Send reply text to Piper, play the resulting audio
  const audioUrl = await fetch('/speak', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ text: reply.text, language: currentLanguage })
  }).then(r => r.json()).then(d => d.audio_url);

  new Audio(audioUrl).play();
}

The three /transcribe, /chat, and /speak endpoints are small Flask routes that shell out to whisper.cpp and Piper’s binaries and forward requests to Ollama’s local API — a thin translation layer rather than real application logic. Keeping the HTML interface itself framework-free (no build step, no bundler) matches the spirit of the rest of the stack: everything can be inspected and modified directly, and nothing here depends on a package registry being reachable.

What actually determines whether this feels usable

The individual pieces — Ollama, Whisper.cpp, Piper — are all well-documented and reasonably easy to get running independently. The real engineering is in the loop’s end-to-end latency: on an M2 Mac Mini, base Whisper transcription plus an 8B Ollama response plus Piper synthesis lands in roughly the range where a pause between speaking and hearing a reply feels like a beat, not a lag. That budget is the actual constraint driving every model-size decision above — it’s less “which model is most accurate” and more “which combination keeps the total round trip under about two seconds.”