The audio thread has a 20ms deadline. The model behind it answers in seconds. Most of the work is the gap between those two numbers.
live · Real-Time Voice · LLM · Speech
A voice-first astrology companion: you talk, it answers. Sub-second spoken interaction over an LLM grounded in a Vedic knowledge base, with conversational memory that carries across a session instead of restarting each turn.
The hard part is not the answer, it is the turn — deciding when to speak, and stopping cleanly when you are interrupted mid-sentence.
A Go harness that measures how well four loop designs hold a 20ms audio cadence under CPU contention and GC pressure, and refuses to report figures its host's clock cannot resolve. Under contention on Linux, time.Ticker missed 56.7% of deadlines where a hand-written absolute-deadline loop missed 1.3% — on an idle host the two look identical, so the failure is invisible until load.
idle
cpu-contention
A testbed for retrieval that is more than chunk-and-cosine: chunk boundaries chosen by a model instead of a character count, a hypothetical answer embedded in place of the question, and a reranker that has to quote the text supporting each candidate it keeps. Built on Vercel's ai-chatbot template (Apache-2.0) — the ingestion, chunking and retrieval pipeline are the additions. Its README is straight about the largest gap: retrieval quality is inspected, not measured.
The symptom was invisible from outside the SDK. Connections established, media flowed, quality was fine — and then, on the ten-minute mark, dead.
A credential-refresh timer in a TURN layer was scoped globally rather than per channel, so the second channel inherited the first one's expiry and aged out early. I found it by tracing inside the library in production and shipped a vendored patch.
Most of the transport work looks like this: SRTP, DTLS, ICE, STUN and TURN, Opus over RTP, jitter buffering. The bug is rarely where the error message is.
15 posts
since 2020
Real-time voice infrastructure: WebRTC/VoIP transport, streaming speech recognition and synthesis, and the conversation layer that decides when an agent should speak and when it should stop. Took a voice product from an empty repository to a paid live service in about six weeks across five services. Billing was per-minute against a wallet, which made correctness on call teardown a revenue problem rather than a cleanup detail.
AI systems for 10+ hospitals. An LLM evaluation framework for clinical prior authorisation, validated at 98% extraction accuracy, along with the retrieval and extraction pipeline it graded. Prior-authorisation automation across 120k+ medical procedures, and a financial reconciliation system that recovered $2.3M in pending claims.
Most of my work has been on systems where something has to happen on time. Before voice it was wallets, ledgers and idempotent settlement; before that, a freelancing platform I founded and ran.
The thread through all of it is that I would rather measure than guess.
A screen that took 7s to render went to imperceptible once I counted the work instead of theorising about it — 21,780 redundant translation lookups in a single file, reduced to 330 across 145 call sites. No architecture changed.
Python and TypeScript daily — asyncio, FastAPI, Celery, PyAV/Opus. Go, Java, C++ and Kotlin as the problem calls for them. Postgres, Redis, BullMQ and SQS, ECS and Lambda, Docker, Grafana/Loki, pgvector.