NVIDIA Open Sources GPU Medical Physics Simulation for Robot Training

NVIDIA published the Medical Physics Simulation framework as an open source, GPU-accelerated capability within Isaac for Healthcare on July 28, 2026. The framework targets the data gap, generalization limits, and slow development cycles in healthcare robotics, where real-world demonstrations are scarce and benchtop, cadaver, and animal studies take months and push development cycles to 4 to 7 years. It provides classical physics solvers and generative world models for real-time device-anatomy interaction, with both simulation and learning running on the same GPU to avoid repeated CPU-GPU memory transfers.

Classical solvers for catheters and surgery

The Endoluminal Simulation Module, released for general availability, simulates long flexible instruments navigating through endoluminal cavities, with the initial release showing catheter navigation under fluoroscopic guidance. It is a standalone Python package using NVIDIA Warp and Newton Physics. Flexible instruments are modeled as Cosserat rods, and the solver uses extended position-based dynamics (XPBD) with a globally coupled approach: each rod segment contributes six constraint equations, forming a block-tridiagonal system with 6 by 6 blocks solved by the Thomas algorithm in linear time. This propagates proximal insertion and rotation along the entire instrument within each step, even for very long instruments. Catheter-vessel interaction is evaluated in parallel on the GPU using signed closest-point queries against the patient-specific triangle mesh, with rigid one-way containment; two-way coupling to deformable vessel walls is planned. Control inputs come through boundary conditions and rest state, not artificial forces. Torch-Warp interoperability keeps the reinforcement-learning interface zero-copy and GPU-resident, integrating with Isaac Lab. Reported performance includes about 1,300 Hz for single-environment physics, 60 Hz for physics across 512 parallel environments, and 63 frames per second for the full simulation and rendering loop at 256 by 256 pixels.

The Surgical Simulation Module, available through early access, models soft-tissue deformation and surgical interactions with haptic feedback. It organizes functionality as explicitly ordered simulation systems: deformation, collision, grasping, cutting, clipping, diathermy, and rendering. Soft tissue uses tetrahedral meshes with position-based dynamics and distance- and volume-preserving constraints, using a Jacobi solver to reduce constraint-ordering artifacts. Simulation and visualization stay on the GPU, with CUDA graph capture reducing kernel-launch overhead and direct GPU-to-renderer transfer avoiding CPU copies. The module runs at more than 30 frames per second with eight physics substeps and eight internal deformation constraint iterations on one consumer-grade GPU. The initial release supports an end-to-end cholecystectomy procedure, including dissection of the hepatocystic triangle, clipping and cutting the cystic duct and artery, and separating the gallbladder from the liver.

Generative world models complement physics

NVIDIA also introduced the Cosmos-H family of generative world models, which produce realistic surgical video from robot actions. Cosmos-H-Dreams enables interactive simulation at more than 30 frames per second via the FlashDreams inference engine. The company positions classical solvers and generative simulation as complementary: physics provides deterministic guarantees, while generative models add diverse visual realism. A unified simulation loop combines anatomical digital twins, device mechanics, and fluoroscopic imaging in a single GPU-resident pipeline, with the vasculature-digital-twin package generating attenuation and geometry artifacts and fluorosim compositing the solver state into the same CT-derived imaging domain. Developers can build endoluminal workflows from the catheter navigation workflow, run Cosmos-H Dreams from the GitHub repository with pretrained checkpoints, and download model checkpoints from Hugging Face. The framework is designed for high-throughput reinforcement learning, including experiments with 512 parallel environments and 1,500 training iterations, and provides a foundation for future policy training directly from fluoroscopic images.