QB.
GitHubLinkedInContact
All projects

Case 05 / A voice tutor that only asks

Loqui Loop

A local-first voice loop for explaining a subject aloud, designed to find the gap with one narrower question instead of supplying the answer.

Write-up only. Source is private; prototype captures and evaluation are included below.

Role
Designer and builder
Status
PrototypeApple Silicon prototype; private source; write-up only
Period
2026 - present
Key decision
Constrain the tutor to one question at a time and evaluate whether that question advances the learner rather than whether it sounds Socratic.
Evidence
A voice tutor that asks follow-up questions, with local speech processing and conversation-level evaluation.
Loqui Loop browser interface inviting a learner to explain something aloud.
The real browser interface keeps the task singular: talk, pause, and receive one question.
01 / Problem

The tutor should reveal a gap, not fill every silence.

Explaining a concept aloud reveals gaps, but a passive recorder cannot notice where the explanation stopped making progress.

A conversational tutor can help, yet common assistants quickly take over: they answer, stack questions, repeat a settled point, or mistake polished language for understanding.

02 / Architecture

Local speech, a remote questioner, one turn record.

The browser records a turn and a local MLX speech model transcribes it. Only the dialogue goes to Claude, which is prompted to return one question without an explanation. A local voice model streams the response back. Session records preserve the inputs needed for evaluation without inferring demographic traits.

System map / Loqui Loop
One turn of the Loqui LoopThe browser records speech, local Nemotron speech recognition transcribes it through MLX, and only dialogue text goes to Claude to select one next question. Fish Audio or dots dot TTS synthesizes the response locally. A turn record preserves latency and behavior evidence before the learner explains again.explain againMicrophonebrowserNemotron ASRlocal, MLXText onlyClaudeone questionLocal TTSFish or dots.ttsSpeakersbrowserTurn recordlatency + behavior evidence
  1. 01Browser microphoneSpeech and pause detection
  2. 02Nemotron ASRLocal MLX transcription
  3. 03ClaudeSelects one next question
  4. 04Fish or dots.ttsLocal speech synthesis
  5. 05Turn recordLatency and behavior evidence
Raw voice stays on the machine. Dialogue text crosses the provider boundary; recognition and synthesis remain local.
03 / Decisions

The trade-offs.

01

Make one question a product constraint

Instead of
Allow explanations, hints, and multiple follow-ups whenever the model believes they are helpful.
Choice
Ask one question, never explain the topic back, and make the next question smaller after a stall.
Why it holds
The learner has to carry the reasoning. The system listens for the next useful gap instead of becoming the lecturer.
Cost
The interaction can feel deliberately restrained, and a poor question cannot be rescued by an immediate answer.
02

Keep speech processing on the device

Instead of
Use hosted speech recognition and synthesis for easier deployment and broader hardware support.
Choice
Run Nemotron ASR and Fish Audio or dots.tts locally on Apple Silicon.
Why it holds
Raw voice does not need to leave the machine, and the audio loop remains testable independently of the dialogue model.
Cost
The prototype needs Apple Silicon, large local models, and accepts different latency and language limits by voice engine.
03

Evaluate threads, not only isolated replies

Instead of
Judge a set of single questions and assume those results carry into conversation.
Choice
Maintain separate harnesses for planted single-turn defects and multi-turn failures such as repetition, leakage, and stacking.
Why it holds
An earlier prompt scored well on individual questions while failing across threads. The product behavior exists over time.
Cost
Model variance requires repeated runs and larger samples before small changes are interpretable.
04 / Build & evidence

Evaluate the conversation, not just the last reply.

Built

  • A browser microphone loop and a terminal client backed by the same questioning behavior.
  • Local MLX speech recognition, two selectable local TTS engines, automatic language detection, and streaming audio.
  • Turn records with transcript, language, timing, latency, stop reason, and prompt hash for comparable evaluation.
  • Offline assessment, single-question, and multi-turn evaluation harnesses with saved baseline runs.

Evaluation

  • The current v5 comparison used 48 threads per arm. Good threads moved from 65% to 90%, answer leakage from 12% to 2%, and stacked questions from 19% to 0%.
  • On single questions, the same comparison stayed near 92-93% on target and good, showing that the main gains were specifically conversational.
  • The project documentation warns that the samples remain small and that identical prompts have produced materially different single-run results.

Supported result

  • Eight languages work end to end: English, Spanish, French, German, Portuguese, Russian, Korean, and Arabic.
  • The prompt revision removed observed question stacking in 144 measured turns and substantially reduced answer leakage in the comparison set.
05 / Limits

What this does not solve.

Current limitations

  • It is a local Apple Silicon prototype, not a hosted service; the personal Claude CLI route is explicitly unsuitable for public deployment.
  • Fish Audio has better voice quality but research-only licensing and slower synthesis; dots.tts is faster and Apache-2.0 but supports fewer languages well.
  • Dialogue text still reaches the configured Claude provider even though raw audio stays local.

Next improvements

  • Enforce the one-question constraint mechanically in addition to prompting for it.
  • Grow the thread suite and repeat each arm enough to distinguish improvements from model variance.
  • Add explicit conversation state for developed, closed, and abandoned threads rather than inferring all state from the transcript each turn.