LLM Inference Architecture: GPUs, Model Servers, APIs, and Scaling
What actually happens between an API call and a token. A tour of the serving stack — batching, KV cache, autoscaling and the cost of every millisecond.
Marcus Feld13 min read
Tag
MLOps
Every story tagged MLOps.
AI Observability: Monitoring LLM Applications in Production
Latency and errors are not enough. Tracking quality, cost-per-request and hallucination signals for systems whose output is probabilistic.
Marcus Feld10 min read
The Infrastructure Briefing
Get the infrastructure briefing.
Practical DevOps, cloud, AI infrastructure and engineering insights — delivered weekly. Read by engineers and engineering leaders.