Blog
Practical thinking on AI governance, scalable systems, blockchain architecture, and the engineering discipline that makes complex systems reliable.
We benchmarked the standard vector-search stack against an OKF index — Google's new standard for LLM-built knowledge bases — on a matching problem: 50 part specs, 2,000 supplier capability statements, and one job — find every shop that qualifies. The standard stack missed most of them, and no tuning fixed it. Here's why, with numbers.
A production post-mortem on an AI scoring step that failed, fell back to a worse path, and reported success anyway — why a single overloaded return value hides real failures, and the fail-loud discipline that catches them.
A production post-mortem on an LLM job that silently stalled for 36 minutes — why inherited SDK timeouts are dangerous, why AI calls come in latency classes, and the per-class timeout and heartbeat pattern that fixes it.
How to implement distributed tracing for multi-agent AI systems — propagating trace context across async boundaries, capturing LLM-specific signals, and building the observability that makes agent debugging possible.
A systematic approach to diagnosing tool call failures in AI agent systems — from incorrect parameter construction to silent schema mismatches and the debugging patterns that catch them.
What to monitor in a production RAG system — retrieval quality metrics, embedding drift detection, index freshness, and the alerts that catch degradation before users notice.
How to architect a scalable, event-driven AI agent system on AWS Lambda with SQS — the four-tier hierarchy, countdown latches, and the patterns that make it production-ready.
AI governance for engineering teams requires enforceable constraints in the development workflow, not just policy documents that sit in a wiki.
Practical lessons on building retrieval-augmented generation pipelines that work reliably at scale, beyond the demo stage.
Why we stopped calling OpenAI for embeddings and built a Rust-based vector generation service on AWS Lambda. Architecture, deployment, and the math that makes it obvious.
Architectural patterns for scaling backend systems that process large volumes of data reliably, from partitioning strategies to backpressure mechanisms.