NVIDIA COMPASS Workflow Trains Robot Navigation Policies with AI Agents

NVIDIA has published a tutorial for training a cross-embodiment robot navigation policy using a workflow called COMPASS. The workflow adapts the pretrained NVIDIA X-Mobility policy into residual specialists using reinforcement learning. An agent-driven pipeline automates validation, scene preparation, smoke testing, residual training, checkpoint evaluation, and runtime packaging. The reference implementation uses the Boston Dynamics Spot quadruped robot, but the approach is designed to transfer to other embodiments.

Workflow and Evidence

COMPASS runs a standard residual reinforcement learning workflow that saves periodic checkpoints, monitors reward components and safety metrics, and preserves evidence for matched-condition evaluation. Each phase produces reviewable artifacts: a validation report and smoke-test log; scene registration, occupancy map, and visual inspection; training command, logs, telemetry, and checkpoints; evaluation protocol, metrics, videos, and promotion recommendation; and packaging of the approved checkpoint, config, evaluation record, and manifest. Human approval gates exist at scene acceptance, one-environment smoke test, and checkpoint promotion.

Evaluation compares the base X-Mobility policy and residual candidates under identical seeds, goals, and rollout conditions. COMPASS metrics include goal-reached rate, fall-down rate, and travel time. At runtime, the exported policy consumes RGB camera input, odometry, and a goal point, publishing velocity commands on the /cmd_vel topic. Optional cuVSLAM can supply deployment odometry.

Scene Sources and Setup

Three scene sources are supported: a built-in warehouse, generated SAGE-10K indoor scenes, and captured environments reconstructed with NVIDIA Omniverse NuRec. The built-in warehouse path uses the registered combined_multi_rack warehouse, providing the fastest reproducible baseline. The SAGE-10K dataset contains 10,000 generated indoor scenes across 50 room types; developers select a single scene rather than downloading the full set. The SAGE-10K path includes two approval gates: inspection of converted USD geometry, materials, scale, and collision meshes, then verification of the generated occupancy map before full training. Omniverse NuRec converts stereo RGB captures into an Isaac Sim-ready reconstruction with aligned visual geometry, collision meshes, and optional scene augmentation.

Setup requires Ubuntu 22.04 or 24.04 with at least 32 GB RAM, an RTX-capable GPU with 16 GB or more VRAM, and Linux driver 580.95.05, tested for Isaac Sim 6.0. The minimum GPU for Isaac Sim 6.0 is a GeForce RTX 4080. Docker Engine version 24 or later with the NVIDIA Container Toolkit is required. A Hugging Face account and read token are needed to access the gated nvidia/COMPASS and nvidia/X-Mobility repositories. The tutorial stack is pinned to NVIDIA Isaac Lab 3.0 and Isaac Sim 6.0.

Setup steps: clone the COMPASS repo, accept gated-repo terms, export HF_TOKEN, run ./docker/run.sh assets, ./docker/run.sh build, then source ./docker/activate. The assets step downloads simulation assets to ./assets/usd/ and the pretrained X-Mobility checkpoint to ./assets/x_mobility.ckpt.

For inference engineers, the workflow highlights how residual RL can specialize a general policy to specific environments while keeping the base model intact. The agent-driven pipeline reduces manual oversight, but the human approval gates ensure that scene quality and checkpoint promotion remain under control. The evidence trail supports reproducible evaluation, which is critical for deploying policies on physical robots where safety metrics matter.