Together AI launches production-grade inference platform with canary rollouts and autoscaling

Together AI released a major update to its Dedicated Model Inference platform, adding production controls that let teams deploy models in minutes and manage rollouts without building their own stack. The company said the platform now supports multiple deployments behind a single stable endpoint, with canary, blue-green, and rolling updates that can auto-roll-back on user-defined thresholds. A/B testing and shadow traffic let teams evaluate new weights or configurations against real production requests, and autoscaling can be driven by metrics such as inflight requests, GPU utilization, time to first token, latency, decoding speed, or throughput.

Deployment profiles and faster startups

The platform includes deployment profiles that package tested configurations for hardware, quantization, tensor parallelism, speculative decoding, and serving engine, so teams can start with a proven setup and later take more control. Together AI also rebuilt its model caching and distribution layer to share weights across the fleet and warm them before deployment. In internal testing, the company reported roughly 4 times faster warm starts across several frontier models. Average deployment times ranged from about 2 to 5 minutes for small models like Qwen 2.5 7B and Llama 3.1 8B, up to 7 to 14 minutes for frontier-scale MoE models such as Kimi K2.7 and MiniMax M3.

Custom training beta and observability

Alongside the inference update, Together AI opened a closed beta for custom training, including full-weight and LoRA reinforcement learning and advanced supervised fine-tuning. Training can be configured through the Python SDK, and checkpoints can be deployed natively to inference deployments without a separate handoff. The platform also provides an organization-level, scrapeable Prometheus endpoint for connecting to external observability tools, plus refreshed in-product analytics for deployment health, traffic, performance, and scaling behavior.

Together AI said the platform is designed for teams that want control over performance, quality, and cost without becoming experts in inference internals. The company cited its experience serving more than 400 trillion tokens per month as the basis for the update. A webinar on August 6 will cover the platform in more depth.