llm-d

llm-d 1.0: Open Source LLM Inference Scaling Hits Production Grade

llm-d 1.0: Open Source LLM Inference Scaling Hits Production Grade

AGNTCon + MCPCon North America 2026, AI, Application Development, DevOps, Events, Open Source, PyTorch Conference, Technology /

The llm-d project has reached 1.0, bringing production-grade LLM inference scaling to Kubernetes-native infrastructure under CNCF governance. With a blended serving cost of $0.11 per million tokens and support for 3,000 concurrent coding agents, the release changes the economics of enterprise AI deployment. ECI Research data shows why developer velocity and AI adoption friction make this the right infrastructure bet at the right moment.

llm-d 1.0: Open Source LLM Inference Scaling Hits Production Grade Read More »

Google TorchTPU: Native PyTorch for Cloud TPUs Goes Open Source

Google TorchTPU: Native PyTorch for Cloud TPUs Goes Open Source

AGNTCon + MCPCon North America 2026, AI, Application Development, Cloud, DevOps, Events, Infrastructure, Machine Learning, Open Source, PyTorch Conference /

Google announced TorchTPU at PyTorch Conference 2026, open sourcing native PyTorch support for Cloud TPUs and integrating it across vLLM, SGLang, llm-d, Ray, and Hugging Face. The release is less a hardware story than a developer ecosystem play, designed to make TPUs the path of least resistance for AI teams already building on PyTorch. LiteRT also joins the PyTorch Ecosystem, closing the training-to-edge deployment gap.

Google TorchTPU: Native PyTorch for Cloud TPUs Goes Open Source Read More »