Deep Dives · Every Wednesday
System
An end-to-end design for a policy and contract assistant at a professional-services firm: metadata-first retrieval, hybrid search with a reranker, per-document access control, and the numbers and threats that decide whether it ships.
System
How Eon replaced ten local MCP servers per developer with one authenticated gateway, and the three design choices that decide whether that pattern holds up: identity, policy and tool listing.
System
Why long-running agents fit workflow engines like Temporal, where the model call belongs, and the event-history ceiling that shapes how you design the loop.
System
Round-robin across vLLM replicas turns one warm prefix cache into eight cold ones; llm-d's precise prefix-cache routing is a good case study in fixing that, and in what its benchmark does and doesn't prove.