Browsing: inference

For the last few years, the AI world has followed a simple rule: if you want a Large Language Model (LLM) to solve a harder problem,…

In 2025, Amazon SageMaker AI saw dramatic improvements to core infrastructure offerings along four dimensions: capacity, price performance, observability, and usability. In this series of posts,…

Cloudflare has released the Agents SDK v0.5.0 to address the limitations of stateless serverless functions in AI development. In standard serverless architectures, every LLM call requires…

Modal Labs, a startup specializing in AI inference infrastructure, is talking to VCs about a new round at a valuation of about $2.5 billion, according to…

Robots are entering their GPT-3 era. For years, researchers have tried to train robots using the same autoregressive (AR) models that…

NVIDIA has released Nemotron-Nano-3-30B-A3B-NVFP4, a production checkpoint that runs a 30B parameter reasoning model in 4 bit NVFP4 format while keeping accuracy close to its BF16…

Building AI applications with Amazon Bedrock presents throughput challenges impacting the scalability of your applications. Global cross-Region inference in the af-south-1 AWS Region changes that. You can now…

Maia 200 is Microsoft’s new in house AI accelerator designed for inference in Azure datacenters. It targets the cost of token generation for large language models…

Tencent Hunyuan has open sourced HPC-Ops, a production grade operator library for large language model inference architecture devices. HPC-Ops focuses on low level CUDA kernels for…

The adoption and implementation of generative AI inference has increased with organizations building more operational workloads that use AI capabilities in production at scale. To help…