Engineering Blog

Behind the scenes of building Savant — the challenges, the bugs, and what we learned.

July 3, 2026

HTTP 200 Doesn't Mean Success: Silent Failures in Streaming LLM APIs

The Claude API can fail inside a stream that returned HTTP 200. The parser had no handler for the error event, so users got empty replies saved as complete. Anatomy of a failure where every layer reported success.

July 2, 2026

Prompt Caching Was On. It Was Barely Working.

A realistic 7-turn agentic conversation billed 219,785 full-rate input tokens to produce 34,000 tokens of output. The cache looked enabled, the accounting said otherwise, and the fix that mattered was a sort and a loop.