Find reporting and practical references by subject.
NVIDIAInferenceNewsNVIDIA's AVO agent architecture, using Claude Opus 5, scored 100.00 RHAE on the ARC-AGI-3 public set, completing all 183 levels with about 12% fewer actions than VISTA. The result highlights system design over model capability alone.
Read the article ↗NVIDIAInferenceNewsNVIDIA's State of AI in Telecommunications report finds 89% of operators say open source models matter to their AI strategy. The company also released a 30-billion-parameter Nemotron 3 Large Telco Model fine-tuned for telecom tasks.
Read the article ↗NVIDIAInferenceNewsNVIDIA and Microsoft unveiled RTX Spark, a new Windows PC platform for local AI agents, with pre-orders starting October 7, 2026. The compact desktop and laptop variants offer up to 128 GB unified memory and one petaFLOP FP4 performance.
Read the article ↗PyTorch and NVIDIA Dynamo introduce session-level identifiers for agentic workloads, enabling session-aware routing, admission control, and KV cache reuse across multi-turn reasoning chains.
Read the article ↗NVIDIAInferenceNewsNVIDIA contributed Vera Rubin rack and compute tray designs to OCP via MGX, with MGX 1.2 adding blind-mate liquid cooling, 5,000 A busbar, and 19-inch support for racks up to 250 kW.
Read the article ↗NVIDIAInferenceNewsNVIDIA details RTX Spark laptop software ecosystem at launch, including agent apps, Nemotron 3.5 Bolt 74B, PAIR release, Adobe optimizations, Broadcast 2.3, and Windows on Arm developer tools.
Read the article ↗NVIDIAInferenceNewsNVIDIA released the full Run:ai platform as open source on October 13, 2026, as part of the DSX AI Factory Platform. The release includes four independently adoptable components, with KAI Scheduler contributed to CNCF as a Sandbox project.
Read the article ↗NVIDIAInferenceNewsNVIDIA reports Nordic AI cloud capacity will reach one gigawatt by end of 2027, with a Swedish consortium launching a shared supercomputer and multiple enterprises adopting NVIDIA-based AI tools.
Read the article ↗NVIDIAInferenceNewsNVIDIA's AI Cluster Runtime v1.0 introduces version-locked recipes, signed validation evidence, and a compatibility contract for GPU Kubernetes clusters, with integrations from Pulumi and Mirantis.
Read the article ↗NVIDIAInferenceNewsNVIDIA introduces NV-Reason-CT, an open 3D vision language model for CT scans that generates structured reports and radiologist-style reasoning, achieving state-of-the-art results on the CT-RATE benchmark.
Read the article ↗NVIDIAInferenceNewsNVIDIA introduces NodeWright, an open-source Kubernetes-native package manager that declaratively configures and updates host OSes across GPU clusters, with progressive rollout strategies and workload-aware disruption handling.
Read the article ↗NVIDIAInferenceNewsNVIDIA's Cluster Readiness Engine (NVCRE) runs distributed workloads across topology-aware node groups to catch slow GPUs, degraded links, and routing issues before production AI jobs land. It automates fault isolation, cutting debugging from days to minutes.
Read the article ↗NVIDIAInferenceNewsNVIDIA reports that confidential computing on eight B200 GPUs retained 96.1% to 98.2% of baseline throughput for DeepSeek-R1 inference, with per-token latency overhead between 1.2% and 4.3% across concurrency levels 1 through 16.
Read the article ↗NVIDIAInferenceAnalysisNVIDIA's Isaac ROS 5.0 uses the CUDA buffer backend for rosidl::Buffer to move GPU-resident payloads between nodes without host copies, with CPU fallback for incompatible peers. An AI agent skill automates the migration audit and refactor.
Read the article ↗NVIDIAInferenceNewsNVIDIA's AIPerf benchmarking tool uses a multiprocess architecture to avoid client-side bottlenecks, supports 15+ endpoint types, and offers configurable arrival patterns for realistic load testing.
Read the article ↗NVIDIAInferenceNewsNVIDIA's Vera Rubin NVL72 preview results in MLPerf Inference v6.1 show up to 3.7x higher Qwen3-VL throughput and 2.5x higher DeepSeek-R1 throughput versus GB300 NVL72, with software gains from v6.0.
Read the article ↗NVIDIAInferenceNewsNVIDIA's TileGym repository now includes an AI agent skill that ports cuTile Python and Triton-TileIR kernels to cuTile Rust, achieving 99.5% of Python performance on average across 24 operators.
Read the article ↗NVIDIAInferenceNewsThe AI Energy Management Alliance (AEMA), founded by Emerald AI, Google, and NVIDIA, aims to make data centers dynamically adjust power use based on grid conditions, enabling faster AI expansion and lower environmental impact per watt.
Read the article ↗NVIDIAInferenceAnalysisNVIDIA says Groq 3 LPX uses cycle-exact scheduling to predict current demand, enabling PEP and CPS to cut voltage droop by over 60% and reduce guardband, improving performance per watt for high-interactivity inference on Vera Rubin.
Read the article ↗NVIDIAInferenceAnalysisNVIDIA details how NVLink 6 combines lightweight FEC, physical layer retry, credit-based flow control, and software recovery to keep AI factories running through signal errors and link failures.
Read the article ↗NVIDIAInferenceNewsNVIDIA's BioNeMo Inference Runtime accelerates biomolecular structure prediction on GPUs, folding 58.5K residues per GPU-hour with Boltz-2, a 2.9x improvement over torch-compiled open-source code. The tool also processed 31 million complexes for the AlphaFold Database.
Read the article ↗NVIDIAInferenceNewsNVIDIA reports that its NIM 2.0.12 serving stack delivers 1,997 tokens per second on a 4xB200 system at a 50 TPS per user target, up from 718 tokens per second without optimizations, enabling more concurrent users at the same interactivity.
Read the article ↗
Together AIInferenceNewsTogether AI's kernels team adapted ThunderKittens for NVIDIA Vera Rubin NVL72, using wider tensor core K steps, larger tensor memory and shared memory, plus a B-side collector, to push NVFP4 GEMMs past 22 PFLOPS.
Read the article ↗NVIDIAInferenceNewsNVIDIA post-trained Nemotron 3.5 Lightning on captured allocation decisions to reach 86.7% accuracy, beating larger models. The workflow uses Palantir Foundry and cuOpt to codify human planner expertise.
Read the article ↗NVIDIAInferenceAnalysisNVIDIA Dynamo's encode-prefill-decode disaggregation separates vision encoding from LLM stages, delivering up to 5x faster time to first token and 7x faster end-to-end response for image-heavy workloads. Gains depend on media load, output length, model size, and traffic mix.
Read the article ↗NVIDIAInferenceNewsNVIDIA's CUDA Toolkit 13.4 brings Windows on Arm support, a preview of the Rubin architecture, MPS V3 for shared GPU control, and new CUDA Python features for texture and managed memory.
Read the article ↗NVIDIAInferenceAnalysisNVIDIA's NemoClaw recipe for a memory-driven Chief of Staff agent stores knowledge in Markdown and judgments in SQLite, using an intent gate and audit trail. The self model lifts overall accuracy to 90.9% from 82.8% on the Agent Memory Benchmark versus an agentic RAG baseline.
Read the article ↗NVIDIAInferenceAnalysisNVIDIA shows how Nemotron 3.5 Lightning and Qwen3.8-27B run on Jetson AGX Thor and Orin, with NVFP4 quantization and speculative decoding delivering up to 6.28x decode throughput speedup over BF16.
Read the article ↗NVIDIAInferenceNewsNVIDIA describes a central identity gateway pattern for federated Kubernetes that separates session ownership from request enforcement, cutting repeated login events by 55% across internal platforms in AWS and OCI.
Read the article ↗NVIDIAInferenceNewsAt IFA 2026, NVIDIA announced the Personal AI Router (PAIR) for distributing inference across local PCs, up to 1.9x faster llama.cpp throughput, and October availability of RTX Spark Windows PCs from Lenovo and Acer.
Read the article ↗NVIDIAInferenceNewsNVIDIA's PAIR beta proxies Ollama and LM Studio to schedule independent inference requests across paired local systems, cutting a five-subagent demo from 18 minutes to 8 minutes 48 seconds on a three-device cluster.
Read the article ↗NVIDIAInferenceNewsNVIDIA's cloud gaming service adds 28 titles in September, led by NBA 2K27 with DLSS 5 3D-Guided Neural Rendering, available to Ultimate members on RTX 5080 rigs. The Blood of Dawnwalker and Onimusha: Way of the Sword also launch on the platform.
Read the article ↗NVIDIAInferenceNewsNVIDIA's TensorRT Model Connect builds a model bundle with a Python CLI, then loads it in C++ without Python at runtime. The open-source project targets faster inference than torch.compile for supported workloads.
Read the article ↗NVIDIAInferenceNewsNVIDIA's Vera CPU, its first custom processor for agentic AI, is now shipping to AWS, Oracle Cloud Infrastructure, Anthropic, OpenAI and SpaceXAI. The 88-core chip targets CPU-heavy agent workloads with 1.2TB/s memory bandwidth.
Read the article ↗NVIDIAInferenceNewsAlibaba's 125B-parameter MoE model with recurrent and sparse attention layers reaches over 16,000 tokens per second per GPU on NVIDIA GB300 NVL72, with day-0 support in SGLang, vLLM, and TensorRT LLM.
Read the article ↗NVIDIAEdge AIAnalysisNVIDIA's COMPASS workflow uses AI agents to automate validation, scene preparation, residual training, and evaluation for robot navigation policies, with human approval gates and evidence tracking.
Read the article ↗NVIDIAInferenceNewsNVIDIA's NVLink Fusion and NVHBM memory technology aim to let hyperscalers deploy custom XPUs with up to 30% more bandwidth per stack and 15% lower HBM power use compared with standard HBM4e.
Read the article ↗NVIDIAInferenceNewsNVIDIA's shadow engine recovery in Dynamo restores LLM serving in 7.3 seconds versus 283 for a cold restart, using a parked engine and shared GPU memory. It's a preview feature for Kubernetes 1.34+ with vLLM support.
Read the article ↗NVIDIAInferenceNewsNVIDIA's CUDA Python 1.0 milestone brings semantic versioning guarantees to independently released components, with cuda.core 1.0.0 as the shared foundation for devices, streams, and buffers across the Python GPU ecosystem.
Read the article ↗NVIDIAInferenceAnalysisNVIDIA details how Spectrum-X Ethernet uses hardware-accelerated control loops and a multiplane topology to handle giga-scale AI workloads, claiming stable training times under multi-tenant congestion and fast failover versus traditional Ethernet.
Read the article ↗NVIDIAInferenceNewsNVIDIA previews Vera Rubin NVL72 results on SemiAnalysis AgentX, claiming up to 30x higher AI-factory throughput per megawatt than GB300 NVL72 at 160 tokens per second per user. GB300 NVL72 itself shows up to 15x over H200 NVL8 on DeepSeek V4 Pro and 80x on Kimi K3 2.8T.
Read the article ↗NVIDIAInferenceNewsNVIDIA's AdaptGrow solver reduces SymNMF peak storage from 20n² to 4n² bytes, fitting 100,000 instruments on one GB200 GPU and scaling to one million across 64 GPUs, with auto-configured optimization and structural-break detection.
Read the article ↗NVIDIAInferenceNewsNVIDIA details production-ready training and inference for generative recommender systems, with HSTU and Semantic ID models showing up to 2.38x speedups and 2.14x lower latency on H100 GPUs.
Read the article ↗NVIDIAInferenceAnalysisNVIDIA's open-source SkillEvaluator benchmarks verified AI agent skills with static checks and live runs, reporting an average 31-point Skill Lift across 300+ skills, with correctness and effectiveness gains of 41 and 39 points.
Read the article ↗NVIDIAEdge AINewsNVIDIA's Cosmos 3 Edge, a 4B omni-model, runs natively on Jetson Thor for robot manipulation. Post-trained on the Cosmos3-DROID dataset, it generates action chunks in about 1.53 seconds on Jetson AGX Thor T5000, achieving 22.9% success in closed-loop RoboLab evaluation.
Read the article ↗NVIDIAEdge AIAnalysisNVIDIA engineers used a coding agent with the Holoscan CLI and HoloHub skills to build an endoscopic tool segmentation dashboard in three iterations, cutting latency by 33.6% and raising throughput by 50.5%.
Read the article ↗NVIDIAInferenceNewsNVIDIA partners with SB Energy to secure 4.25 GW of capacity at PORTS-Pike in Ohio, with OpenAI as tenant. The site supports multiple GPU generations over 20 years, with potential expansion to 8 GW.
Read the article ↗NVIDIAInferenceNewsUniversitas Gadjah Mada, Indosat, and NVIDIA launched Indonesia's first university-based AI technology center in Yogyakarta, focused on healthcare, agriculture, and disaster response.
Read the article ↗NVIDIAInferenceNewsAlibaba's 2.4T-parameter MoE model with 95B active parameters runs over 4K tokens per second per GPU on NVIDIA GB300 NVL72 in FP8. NVIDIA provides recipes for SGLang, vLLM, Dynamo, and NIM, plus NeMo AutoModel for fine-tuning.
Read the article ↗NVIDIAInferenceNewsJetPack 7.2.1 introduces PyNvVideoCodec 2.2 for hardware video encode/decode, agentic video skills for workflow automation, and T3000 emulation on T5000 hardware.
Read the article ↗NVIDIAInferenceNewsNVIDIA released Nemotron 3.5 Lightning, an open 30B MoE model with 3B active parameters, plus NeMo Switchyard for routing tasks between frontier and execution models.
Read the article ↗Meta introduced Muse Glimmer, an open-weight 30B model for on-device agentic AI, with ExecuTorch support for NVIDIA GPUs and Apple silicon. Prebuilt PTE artifacts are available, enabling text and image inputs, DFlash speculative decoding, and 128K+ token context.
Read the article ↗NVIDIAInferenceNewsMeta's 30B dense model Muse Glimmer runs at over 20K tokens per second on a single NVIDIA GPU, fitting in VRAM for local agentic workflows. NVIDIA outlines deployment paths across RTX, DGX, and Jetson platforms.
Read the article ↗NVIDIAInferenceNewsFirebird launched the CIS region's largest AI factory in Armenia, with plans to install over 70,000 NVIDIA Rubin and Blackwell GPUs by end of 2027. The site targets 300 MW power capacity and early customer Perplexity will use the infrastructure for its AI agent platform.
Read the article ↗NVIDIAInferenceNewsNVIDIA's Alpamayo 2 Super combines a 32B reasoner with a 2B diffusion action expert to output trajectories, reasoning traces, and auto-labels. Open-loop and closed-loop results show improvements over Alpamayo 1.5 Nano, with weights on Hugging Face.
Read the article ↗NVIDIAInferenceNewsNVIDIA's Vera CPU, embedded in the BlueField-4 STX storage processor, outperforms a benchmarked x86 CPU in encryption, compression, integrity checking, and Reed-Solomon recovery, according to vendor benchmarks.
Read the article ↗NVIDIAInferenceAnalysisNVIDIA shares four debugging cases from partner clusters where configuration choices in the kernel, hypervisor, BIOS, and NCCL compounded into 8-12% throughput gaps, and the tuning changes that closed them.
Read the article ↗NVIDIAInferenceAnalysisNVIDIA's AI Red Team found that access control, code execution, network egress, and secret handling are the recurring weak points in AI agents. The team recommends deterministic controls outside the model's control plane.
Read the article ↗NVIDIAInferenceNewsNVIDIA's nvmath-python v1.0 wraps CUDA-X math libraries in a Pythonic API, supporting CPU, GPU, and distributed execution. It introduces a universal sparse tensor and stateful APIs that amortize planning costs across repeated runs.
Read the article ↗NVIDIAInferenceAnalysisNVIDIA details a self-hosted setup for StarCoder2-7B with NeMo Guardrails, CI checks, and metrics to keep AI-assisted code safe and auditable.
Read the article ↗NVIDIAInferenceNewsNVIDIA released GPU-native simulation modules for catheter navigation and soft-tissue surgery, plus generative world models, to train healthcare robots with reinforcement learning at scale.
Read the article ↗NVIDIAEdge AINewsNVIDIA highlights its Jetson lineup for edge AI and robotics, from the 67 TOPS Orin Nano Super developer kit to the 2070 FP4 teraflops AGX Thor with 128GB of memory, with example projects for each tier.
Read the article ↗NVIDIAInferenceNewsNVIDIA's Ising Calibration 1.5, a 31-billion-parameter vision language model, interprets quantum processor diagnostics and recommends tuning actions. It improves 86.68% over its predecessor with in-context learning and now ships in an NVFP4-quantized version for single-GPU or DGX Spark deployment.
Read the article ↗NVIDIAInferenceNewsNVIDIA's ModelExpress (MX) selects the fastest available source for model weights, prioritizing GPU-to-GPU P2P RDMA transfers via NIXL over redundant downloads from remote storage. It also reuses kernel compilation caches and coordinates fleet-wide downloads to cut cold start times dramatically.
Read the article ↗NVIDIAInferenceNewsNVIDIA and Prime Intellect Lab show a three-step workflow to customize Nemotron 3 Nano with hosted reinforcement learning, producing a downloadable LoRA adapter. Baseline accuracy on 32 math problems rose from 21.9% to 90.6% after 100 training steps.
Read the article ↗NVIDIAInferenceNewsNVIDIA's Rubin GPU targets agentic AI with up to 10x throughput per energy versus Blackwell, 288 GB HBM4, 50 petaflops NVFP4, and NVLink 6. The architecture adds TMA descriptor updates, doubled K-dimension Tensor Cores, and sparsity for long-context attention.
Read the article ↗NVIDIAInferenceNewsNVIDIA published specifications for sixth-generation NVLink, including 3.6 TB/s per GPU bandwidth, 260 TB/s rack-level throughput, and 130 TFLOPS in-network compute for the Vera Rubin NVL72 platform.
Read the article ↗NVIDIAInferenceNewsNVIDIA positions Vera Rubin as a platform for continuous post-training workloads in agentic AI, claiming it can train the largest models with one-fourth the GPUs of Blackwell. The post details benchmarks for Nemotron 3 Ultra and partner deployments from Prime Intellect, Perplexity, and Together AI.
Read the article ↗NVIDIAInferenceNewsNVIDIA's BlueField-4 DPU integrates a 64-core Grace CPU, LPDDR5X memory, and PCIe Gen6, delivering up to six times the compute of BlueField-3. It offloads networking, storage, and security to boost GPU utilization and cut inference latency.
Read the article ↗NVIDIAInferenceNewsNVIDIA's NemoClaw orchestrates the Video Search and Summarization and RAG blueprints to turn video analysis into coordinated actions like ticket creation and workflow routing, with a healthy eating coach demo creating Jira tickets from meal prep videos.
Read the article ↗NVIDIAInferenceNewsNVIDIA DeepStream 9.1 introduces AutoMagicCalib and Multi-View 3D Tracking to fuse detections from multiple cameras into a shared 3D coordinate system with consistent object IDs, plus agentic skills for natural-language deployment.
Read the article ↗NVIDIAInferenceNewsNVIDIA's nanousd-labs project uses AI agents to generate a lightweight USD runtime directly from the USD Core Specification, offering a stable C ABI and spec-compliance validation. The approach lets developers tailor runtimes to specific memory, performance, or ABI needs without adapting large existing codebases.
Read the article ↗NVIDIAInferenceNewsNVIDIA's RoboLab simulation platform evaluates general-purpose robot policies with capability-tagged tasks, graded scores, trajectory quality metrics, and failure logging to address benchmark saturation and diagnostic gaps.
Read the article ↗NVIDIAInferenceAnalysisNVIDIA's guide explains how hidden dimension, intermediate projection, and layer count shape throughput and latency, with rules for near-square weights, 128-aligned dimensions, and NVFP4 quantization.
Read the article ↗NVIDIAInferenceNewsNVIDIA's BioNeMo Agent Toolkit accelerates each stage of co-folding: GPU-based MSA search, cuEquivariance kernels, optimized OpenFold3 NIM inference, and Fold-CP context parallelism for multi-GPU scale-out.
Read the article ↗NVIDIAInferenceAnalysisNVIDIA describes replacing CPU-orchestrated MPI with NVSHMEM-based GPU-initiated remote memory access in GROMACS, fusing pack, put, and wait into single kernels to remove 12 blocking syncs per timestep and improve scaling.
Read the article ↗NVIDIAInferenceNewsNVIDIA's NeMo-based iterative pipeline produced 502,536 unique financial news headlines across 13 categories, using global deduplication and dynamic category weighting to maintain novelty and balance.
Read the article ↗NVIDIAInferenceNewsNVIDIA says Nemotron 3 Ultra, tuned via LangChain's Deep Agents harness, beats open models on accuracy and matches closed models on business tasks, at 10x lower inference cost per run.
Read the article ↗NVIDIAInferenceNewsNVIDIA and LangChain show how a harness profile with a middleware fix resolves all three read_file pagination failures and raises the evaluation score from 94 to 96 out of 127, without fine-tuning.
Read the article ↗NVIDIAInferenceNewsNVIDIA introduces a revenue-sharing and credit-support model that lets AI clouds procure its infrastructure, with Sharon AI and Firmus among the first partners building DSX AI factories.
Read the article ↗NVIDIAInferenceNewsNVIDIA reports that enabling Confidential Computing on HGX B300 with a Qwen 3.5 model costs roughly 1% to 8% in throughput and time per output token, with optimizations in FlashInfer and SGLang reducing launch overhead.
Read the article ↗NVIDIAInferenceAnalysisNVIDIA outlines a practical reinforcement learning with verifiable rewards workflow, using GRPO and tools like NeMo RL and Gym, to specialize open models for domain-specific agent tasks.
Read the article ↗Together AIInferenceNewsTogether AI announced an $800 million Series C led by investors including NVIDIA and Aramco Ventures, with commitments for over 500 MW of compute capacity. The company cites 6x to 20x cost reductions for customers using open-weights models in production.
Read the article ↗NVIDIAInferenceAnalysisNVIDIA's GQE reference architecture executes SQL on GPUs with cuDF, nvCOMP, and nvSHMEM, using pipeline parallelism, hybrid compression, and zone-map pruning. On a GB200 GPU it completes all 22 TPC-H queries in 9.0 seconds, a 7.5x speedup over DuckDB on dual-socket AMD Turin CPUs.
Read the article ↗NVIDIAInferenceAnalysisNVIDIA engineers used Nsight Systems and Nsight Compute to optimize the Omniverse NuRec neural reconstruction pipeline, fusing small kernels for a nearly 50x speedup in one function and improving renderBackward occupancy from ~15% to 30-50%.
Read the article ↗NVIDIAEdge AIGuideNVIDIA published a deployment guide for AI-Q 2.0 on Oracle Cloud Infrastructure, using Terraform for OCI resources and Helm for Kubernetes workloads on OKE. The setup creates a working AI-Q endpoint in about 20 to 25 minutes.
Read the article ↗NVIDIAInferenceAnalysisNVIDIA explains how it quantized Nemotron 3 Ultra to NVFP4, cutting model size from 1,121 GB to 352.3 GB. The post details mixed-precision layer assignments, four-over-six block scaling, and AutoQuantize's bit budget search.
Read the article ↗NVIDIAInferenceNewsNVIDIA TensorRT 11.0 introduces native multi-device inference with NCCL collectives and context parallelism strategies. Benchmarks on Cosmos 3 and FLUX.1-dev show DeepSpeed Ulysses delivering the lowest end-to-end latency for long-sequence diffusion workloads.
Read the article ↗NVIDIAInferenceAnalysisNVIDIA's BEVPoolV3 optimizes BEV pooling for GPUs by classifying the working set as DRAM-bound or L2-resident, then applying different kernel strategies. On RTX A6000 it hits 90.0 µs FP16; on RTX PRO 6000 Blackwell Max-Q it reaches 16.4 µs FP8.
Read the article ↗NVIDIAInferenceAnalysisNVIDIA outlines how power costs up to 40% of AI factory OpEx and details system, precision, and scheduling optimizations to improve performance per watt across inference and training.
Read the article ↗NVIDIAInferenceNewsNVIDIA's DFlash block-diffusion drafter generates multiple candidate tokens in one forward pass, delivering up to 15x higher throughput on Blackwell Ultra for gpt-oss-120b and nearly doubling interactivity over EAGLE-3 on Llama 3.1 8B.
Read the article ↗NVIDIAInferenceNewsSiemens EDA used NVIDIA NeMo Gym to evaluate and refine its Solido Characterization Suite agent for Liberty file generation, raising the composite verification score from about 55% to 95% while cutting agent turns from 155 to 53.
Read the article ↗NVIDIAInferenceNewsNVIDIA's cuDF skills for Apache Spark use AI agents to rewrite CPU-bound UDFs into GPU-accelerated equivalents, cutting operator time from 274.55 to 12.78 hours in a 600-billion-row workload.
Read the article ↗NVIDIAInferenceAnalysisNVIDIA and Princeton's Dao AI Lab propose representing SSM state as a checkpoint plus an input buffer, roughly halving state memory traffic and speeding up decoding kernels by 1.46x to 1.80x at batch size 256 on Blackwell Ultra B300 GPUs.
Read the article ↗NVIDIAEdge AINewsNVIDIA demonstrates a five-step workflow using Omniverse libraries and frontier AI models to convert an ABB YuMi robot from STEP files into a simulation-ready OpenUSD asset, covering appearance, physics, validation, and pick-and-place testing in Isaac Sim.
Read the article ↗NVIDIAInferenceNewsNVIDIA cuOpt introduces mPDLP, a multi-GPU solver that distributes large linear programming problems across NVLink-connected GPUs, cutting peak memory per GPU by up to 6x and speeding up solve times for problems impractical on a single GPU.
Read the article ↗NVIDIAEdge AIAnalysisNVIDIA's Seattle Robotics Lab and Isaac team detail how a modular classical pipeline, not end-to-end learning, achieved over 95% success on GB300 tester tray busbar assembly, with a 160-second cycle time.
Read the article ↗NVIDIAInferenceAnalysisNVIDIA's DOCA GPUNetIO now serves as a shared GDA-KI foundation for NCCL, NVSHMEM, UCX/NIXL, and other libraries, with an open-source Verbs-focused implementation that can optionally call SDK functions via dlopen.
Read the article ↗NVIDIAInferenceNewsNVIDIA's green contexts in CUDA 13.1 Runtime API allow explicit SM and workqueue partitioning within a process, cutting critical kernel latency from 0.140 ms to 0.007 ms on a 148-SM Blackwell GPU.
Read the article ↗Together AIInferenceNewsTogether AI is working with IBM and NVIDIA to run a dedicated NVIDIA B300 GPU cluster on IBM Cloud with Spectrum-X Ethernet, as the first customer on that infrastructure. The company says it serves hundreds of trillions of tokens per month to over a million developers.
Read the article ↗NVIDIAInferenceAnalysisNVIDIA details a NeMo-based workflow to adapt Nemotron 3.5 ASR to Saudi Najdi and Hijazi dialects using minimal curation, replay mixing, and partial encoder unfreezing, cutting WER from 55.05% to 29.96% on target speech.
Read the article ↗NVIDIAEdge AINewsNVIDIA released DOCA AI agent skills on GitHub, providing agents with verified API signatures, hardware requirements, and build constraints. In a 65-prompt evaluation, agents with skills satisfied 100% of checklist items versus 19% without, reducing handwritten code by 73% in a demo.
Read the article ↗NVIDIAInferenceNewsNVIDIA's recsys-examples adds an end-to-end HSTU generative recommender workflow using PyTorch AOTI compilation and FlexKV-backed KV caching, reporting up to 5.93x lower latency on an RTX PRO 6000 Blackwell GPU.
Read the article ↗NVIDIAInferenceNewsNVIDIA has made cuObject client and server libraries generally available, adding an RDMA wire protocol for object storage. The xio-sig consortium expands to include cuObject, with Google Cloud and Microsoft involved, and a new SCADA Server SDK targets storage providers.
Read the article ↗NVIDIAEdge AINewsNVIDIA introduced a reference design that pairs OpenShell on Vera CPUs with Sentry on BlueField-4 DPUs, placing out-of-band policy enforcement on the node's only path to the model. The platform targets agents that drift from intended tasks and need independent, hardware-level controls.
Read the article ↗Baseten details NVIDIA Nemotron 3 Diarization, an open end-to-end model that labels up to eight speakers at latencies from 0.32 to 30.4 seconds, with presets for batch, streaming, and real-time transcription. Benchmarks show up to 500 concurrent streams per RTX PRO 6000.
Read the article ↗Dynamo v1.5.0, the 18th feature release, introduces pluggable router policies, session affinity modes, a standalone Sidecar for stock engines, and a new GPU Memory Service, among 658 merged pull requests.
Read the article ↗NVIDIAInferenceNewsNVIDIA reports a 10.4x throughput gain for dropless Mixture of Experts training in JAX on GB200, using grouped GEMM, fused dispatch and combine, and other optimizations. DeepSeek-V3 performance rose from 103 to 1,068 TFLOPS/GPU.
Read the article ↗NVIDIAInferenceNewsNVIDIA announced CUDA Rust, offering two tracks: cuda-oxide for SIMT kernels via a custom rustc backend, and cutile-rs for Tile-based programming on stable Rust. Both enforce memory safety at compile time, with cutile-rs already used in production inference engines.
Read the article ↗NVIDIAInferenceAnalysisNVIDIA's walkthrough shows how Compute Sanitizer, the CCCL launch API, CUB primitives, pooled buffers, pinned memory, and per-thread streams cut an image pipeline from 6.8 seconds to 23 milliseconds.
Read the article ↗NVIDIAInferenceAnalysisNVIDIA and CrowdStrike built an adaptive agentic cybersecurity system using Nemotron 3 models, reporting a 2.5x detection improvement in backtesting and stronger generalization than a frontier system in live-fire tests.
Read the article ↗Modular announced native support for Qualcomm's Cloud AI 100 Ultra NPU in its MAX and Mojo platform, letting developers run the same code on NVIDIA, AMD, and Qualcomm silicon without rewrites. The bring-up reached Gemma 4 serving in under six months.
Read the article ↗Dynamo v1.4.0, the 17th feature release, introduces experimental cross-datacenter prefix routing, a vLLM-compatible generate endpoint, and a 22.2% throughput gain in the Mocker. The update also brings NIXL RDMA transport for multi-node disaggregation and a rebuilt documentation site.
Read the article ↗NVIDIAInferenceNewsNVIDIA's Cosmos 3 world foundation model can be post-trained into robot policies that predict actions and future camera views. The 16B Nano and 4B Edge checkpoints serve on workstation or embedded hardware, with the Edge model running at 15 Hz on Jetson Thor.
Read the article ↗NVIDIAInferenceGuideNVIDIA shows how KAI Scheduler and vCluster let three teams share one L40S GPU with per-team quotas and isolated control planes, each seeing only their own workloads.
Read the article ↗NVIDIAEdge AIAnalysisNVIDIA's Secure Agent Workspace Reference Design shifts agent execution from user devices to managed VMs, adding perimeter and runtime controls, credential proxies, and GitOps-based deployment for on-prem or cloud.
Read the article ↗We count visits on the server, cookie-free. You can also let this browser tell us how long you read, how far you scroll and which links you follow. It stays with us.
