Together AI, IBM Cloud and NVIDIA launch dedicated B300 inference cluster

Together AI announced a collaboration with IBM and NVIDIA to scale enterprise-grade AI inference, starting with a dedicated cluster of NVIDIA B300 GPUs on IBM Cloud backed by NVIDIA Spectrum-X Ethernet networking. Together AI says it is the first customer running on this infrastructure, which it describes as the first dedicated, large-scale inference cluster of its kind on IBM Cloud.

The division of labor is explicit: Together AI operates the inference layer, IBM provides the cloud, and NVIDIA delivers the silicon and networking. The company frames the move as preparation for continued growth in token demand, which it says is climbing.

Why the cluster exists

Together AI states that it serves hundreds of trillions of tokens per month to over a million developers. The company attributes this demand to enterprises and AI-native companies choosing open models for two reasons: data sovereignty and cost. It says open models keep sovereign data with the customer and deliver frontier-level performance at a fraction of closed-model cost, and that the math improves as usage grows.

The announcement positions the collaboration as a response to that demand. NVIDIA brings the B300 GPUs and Spectrum-X Ethernet networking, which Together AI describes as engineered for high-throughput inference. IBM brings its experience running mission-critical infrastructure for large enterprises. Together AI contributes its inference platform, which it calls the fastest and most efficient way to run open models in production, hardened by demanding AI workloads.

What it means for inference teams

For engineers serving open-weight models, the practical takeaway is that a dedicated B300 cluster on IBM Cloud is now available as a deployment target, with Spectrum-X networking designed to handle high-throughput inference traffic. The cluster is purpose-built for inference rather than training, which suggests the vendor expects sustained production token loads rather than bursty experimentation.

The reliability and security framing matters for enterprise buyers. Together AI says the result is enterprise-grade inference at massive scale, with the reliability, security and guardrails enterprises expect. The company also states that open-source AI must run everywhere, at scale, as fast and reliably as anything closed, and that this collaboration is a step toward that goal.

No pricing, availability dates, or cluster size details were included in the announcement. The claims about token volume, performance, and efficiency come from Together AI and have not been independently verified.