NVIDIA's NV-Reason-CT Brings Chain-of-Thought Reasoning to 3D CT Analysis

NVIDIA released NV-Reason-CT, an open vision language model built for 3D computed tomography analysis. The model generates structured diagnostic reports and chain-of-thought reasoning that mirrors how radiologists review CT volumes, and it supports multistep follow-up conversation across chest and abdomen studies. It is a research foundation for post-training, not a cleared clinical product.

Why 3D CT needs a different architecture

A single abdominal CT study can contain 300 to 600 axial slices, and the spatial relationships between those slices define structures like masses, effusions, and infiltrates. Standard vision language models treat images as 2D token grids, so processing a CT volume as independent frames discards that through-plane context. NV-Reason-CT instead uses a full 3D vision transformer encoder that processes the volume natively, preserving anatomical continuity.

The model combines a 3D ViT encoder with a Qwen3.5-4B language model. The encoder, adapted from Primus and initialized with Colipri weights, resamples CT volumes to 192 cubed voxels at 2 mm isotropic resolution. It uses non-overlapping 8x8x8 patch tokens, producing 13,824 vision tokens that are all passed to the language model along with their 3D grid coordinates. The LLM includes 3D MRoPE to maintain spatial relationships between tokens throughout its layers.

Training and reasoning capabilities

NVIDIA trained NV-Reason-CT in two stages. First, supervised fine-tuning on a curriculum of about 550,000 structured QA examples covering chest and abdominal anatomy, laterality-specific findings, severity levels, and binary abnormality identification. Radiologists contributed chain-of-thought dictations that capture their internal review process, and the training data includes CT-RATE, NIH CT datasets, and CancerVerse, supplemented with synthetic reasoning data distilled from larger language models.

Second, reinforcement learning with Group Relative Policy Optimization refines reasoning quality. The reward function is anatomy-aware, reinforcing accuracy within each anatomical region rather than using a single global signal, which improves calibration across the full chest-abdomen findings distribution.

The model generates step-by-step reasoning that examines anatomical regions systematically, surfaces relevant findings, considers differential diagnoses, and articulates uncertainty. It also produces structured reports based on a curated ontology of 30 chest and 29 abdominal abnormalities, and it can answer follow-up questions about specific findings or probe its own reasoning.

Benchmark results and clinical review

On the CT-RATE benchmark, NV-Reason-CT achieved a Macro-F1 of 0.614 and a Macro-AUROC of 0.871, outperforming published 3D contrastive models like VoxelFM, Pillar-0, CT-CLIP, and Merlin, as well as fused 2D/3D models like ClinFusion-8B and slice-based frontier models like MedGemma 1.5. NVIDIA states this is the first time a single open model has achieved competitive CT classification and report generation simultaneously.

NIH radiologists reviewed the model's outputs and validated the clinical plausibility of both the structured reports and the reasoning traces. Baris Turkbey, Senior Clinician at the National Institutes of Health, said the step-by-step reasoning reflects how radiologists actually think through a CT study, and that reviewing the thought process rather than just conclusions is what enables trust and action on findings.

For inference engineers, the model's 13,824 vision token context and full 3D processing mean serving it requires handling a large token sequence with 3D positional encoding, which has implications for memory and attention computation. The open foundation supports post-training for specialized CT applications and integrates with NVIDIA's Medical AI models for segmentation and synthetic data generation.