The constraint that chose the stack
Most retrieval systems begin with a model choice and work outwards. This one began with a prohibition: transcript text must never be sent to a third party. Not to a hosted model, not to an embedding API, not to a managed vector database. Everything downstream follows from that, and the architecture is largely the shape of what survives the rule.
The rule is not squeamishness. Source material here is public record, so the sensitivity is not confidentiality — it is provenance and control. If a regulator, a journalist or a member of the House asks what the system was given and what it produced, an answer that depends on a third party's logs is a weaker answer than one that depends on files you hold. And a pipeline that cannot call out is a pipeline that cannot leak.
A component you cannot run offline is a component you cannot fully account for.
What runs where
| Concern | Choice | Why this one |
|---|---|---|
| Inference | Ollama, serving Llama locally | Runs on the operator's own hardware, Metal-accelerated on Mac. Model and settings are recorded per generation, so an episode can be reproduced. |
| Embeddings | nomic-embed-text, also via Ollama | A separate model from the chat model — a distinction that bites: the chat model cannot embed and fails with HTTP 501 if you ask it to. |
| Vector search | FAISS, one index per recording | A file on disk, not a service. Indexes are per-video, so a re-run touches one directory and provenance stays per-source. |
| Orchestration | LangChain | Used for the retrieval and generation chain only. The pipeline is plain Python scripts driven by the run book, not a framework-shaped application. |
| Ingestion | yt-dlp + youtube-transcript-api | Metadata and captions. The only part of the system that talks to the network, and it fetches the source rather than sending anything to it. |
| Speech | edge-tts | Renders the reviewed script. The one place a hosted service touches the work — and it receives the edited script, never the source transcript. |
That last row is the honest exception, and worth stating plainly rather than hiding in a footnote: speech synthesis is not local. What it receives is the human-reviewed script — text the project has already decided to publish — not the transcript. The rule is about source material, and the boundary is drawn where the material stops being source and becomes output.
Fig. 03 — where source material stops being source
What it costs you
Local-first writing usually stops at the benefits. The costs are real and worth naming, because they determine whether the trade is right for a given project.
- A smaller model. A 1B-parameter local model is not a frontier model. It writes serviceable summaries of retrieved material and would struggle with anything requiring wide world knowledge. The pipeline compensates by narrowing the job: retrieve tightly, then ask the model to write only from what was retrieved.
- Retrieval quality becomes your quality. With a weaker generator, most of the achievable improvement sits in chunking and embedding rather than prompting — which is why the project scores retrieval separately.
- Operational burden moves to you. Ollama must be running, with two models pulled. There is no managed uptime, and the failure modes are yours: a missing embedding model produces a 501, not a helpful error page.
- Throughput is bounded by one machine. Ten recordings is comfortable. Ten thousand would need a different conversation about hardware, not a bigger API bill.
None of that is an argument against the choice. It is the bill that comes with it, and a project that cannot state the bill has not really made the choice.
The patterns worth reusing
Three pieces of this generalise well beyond podcasting, and are the parts worth lifting into an unrelated system.
- A resilient ingestion fallback chain. Captions are fetched in a deliberate order — manual (human) captions first, then auto-generated, then any available language — and the system records which source won. The transcript is not just text; it is text with a known provenance grade.
- Chunking as a tunable, not a constant. Text is split with a recursive character splitter at 1000 characters with 200 overlap. Those numbers are a starting position, not a truth, and the evaluation harness exists to move them on evidence.
- One index per source. Keeping FAISS indexes per recording rather than in one pooled store means retrieval can never silently mix two sources into one episode — a correctness property enforced by layout rather than by prompt.
Retrieval that cannot cross sources cannot fabricate a cross-source claim. Some correctness is cheaper to enforce in the file system than in the model.
Based on House of Lords proceedings (UK Parliament). Not affiliated with, or endorsed by, the UK Parliament.