AMD AIMs: One Container Workflow from Radeon to Instinct

AMD's AIM containers now present the same profile discovery commands, container workflow, and OpenAI-compatible API across Radeon, EPYC, and Instinct hardware. The company demonstrated this by running AIMs on a Radeon GPU, EPYC CPUs, a single Instinct GPU, and an 8-GPU Instinct node, with the same three-step process: selecting an AIM container, exploring profiles with dry-run and list-profiles, then deploying via Docker.

Profile selection under the hood

AIMs are Docker images that ship with predefined profiles, which are inference engine configurations for specific accelerators, precisions, tensor-parallel layouts, and latency or throughput targets. When a container starts, it initializes the runtime, detects the accelerator, and automatically selects a profile based on the model, precision, engine, performance metric, and detected accelerator count and model. Users can bypass automatic selection and specify a profile directly.

The dry-run command shows which profile AIM would select and the exact engine command it would run, without starting a server. In the examples, the same model on different hardware produced different profiles. On a Radeon PRO R9700, the Qwen/Qwen3.5-9B container selected bf16 precision, a tensor-parallel size of 1, and enabled the FLASH_ATTENTION_TRITON_AMD_ENABLE environment variable. On EPYC 9965 CPUs, the same model used bf16 with chunked prefill enabled and CPU thread binding set to auto. For openai/gpt-oss-120b on a single Instinct MI300X, the profile used fp4 precision with tensor-parallel size 1 and the VLLM_ROCM_USE_AITER flag; on an 8-GPU node, the same container selected tensor-parallel size 8.

The list-profiles command shows every available profile for a model, categorized by compatibility with the current configuration. Each row lists the target accelerator, precision, engine, tensor parallelism, performance metric, and profile type, such as optimized or preview. The compatibility column explains why certain profiles are excluded, for example accelerator_mismatch or metric_mismatch.

Deployment and API access

After validating the profile selection, starting the server is a single Docker run command with device access on GPU hosts. On first start, AIM detects the GPU, downloads model weights from Hugging Face, and launches the inference engine on port 8000. The container exposes an OpenAI-compatible completions endpoint, so a standard curl request with the model name, prompt, and token settings returns generated text.

AMD notes that latency is the default performance metric in profile selection. The company also supports custom environment variables and custom profile configurations beyond the built-in ones. For multi-GPU deployments with larger models, AMD points to a separate guide before proceeding.

What this means for inference engineers

The unified workflow reduces the need to manually configure engine arguments and environment variables when moving between accelerator types. The dry-run and list-profiles commands give operators a way to preview what the container will do before committing to a deployment, which is useful for validating configurations in CI or on staging hardware. The automatic profile selection also means the same container image can adapt to different node sizes, as shown by the gpt-oss-120b image switching from tensor-parallel size 1 to 8 based on detected GPU count.

The examples were validated on an AMD Instinct MI300X cluster, an AMD EPYC 9965 cluster, and an AMD Radeon PRO R9700 cluster. Version tags on the container images change with new AIM releases, so the specific tags in the post, such as 0.11.1 and 0.12.0-preview, are tied to those releases. For other models, the AIM catalog lists available images and technical specifications, including which profiles exist for a given model and hardware combination.