NVIDIA Iterative Pipeline Generates 500K Unique Financial Headlines

NVIDIA published a blog post on July 9, 2026, describing an iterative synthetic data generation pipeline built with NeMo that produced 502,536 unique financial news headlines across 13 categories. The pipeline ran 82 iterations, with each batch initially generating 50,000 headlines before settling at 35,000 per batch after early iterations. Global semantic deduplication removed about 82% of generated headlines, and net-new headlines stabilized at 5,000 to 6,000 per 35,000-headline batch once the corpus exceeded 100,000 items.

Pipeline Mechanics

The iterative loop consisted of generation, quality filtering, global deduplication, few-shot selection, and weight correction. Generation used a Nemotron-3-Nano-30B-A3B-FP8 model, which has 3 billion active parameters per forward pass within a 30-billion-parameter mixture-of-experts architecture. The model was served with vLLM version 0.12.0 using 4-way tensor parallelism and 448 concurrent requests. The pipeline ran on a single 8-way NVIDIA B200 node, with GPUs 0 through 3 handling inference and GPUs 4 through 7 running NeMo Curator deduplication with Ray. Total compute time was about 6 days.

Deduplication used sentence-transformers/all-MiniLM-L6-v2 embeddings, 500 K-means clusters, and a 90% cosine similarity threshold (eps equals 0.10). NeMo Curator's TextSemanticDeduplicationWorkflow kept the headline closest to each cluster centroid as the representative. The cluster count of 500 was chosen to keep average cluster size around 1,000 headlines at the 500,000-headline endpoint, reducing pairwise comparisons to roughly 500 million. Pairwise similarity batch size was set to 4,096, embedding batch size to 1,024, and max sequence length to 512.

Steering Novelty and Balance

To maintain novelty, the pipeline used farthest-from-centroid few-shot selection with an 80% cross-iteration similarity cutoff. This meant that for each iteration, the few-shot examples chosen for the next generation were those most dissimilar from previously generated content, up to a similarity threshold. Dynamic distribution correction adjusted category weights each iteration to hit 1% targets for rare categories like Credit Ratings and Product Approval, which were brought to near 1% of the total corpus. The category sampler updated weights after each iteration to balance the distribution.

Quality filtering removed fewer than 1% of headlines per batch using word and character count rules and formatting checks. Generation parameters included temperature 0.95, top-p 0.95, frequency penalty 0.3, and presence penalty 0.2. The prompt included a system role, a category placeholder, and up to three few-shot examples per category, totaling up to 39 examples. Checkpointing persisted corpus state, category weights, few-shot examples, metrics, and embeddings each iteration for crash recovery and SLURM job chaining.

Baseline Comparison and Output

A baseline single-pass run of 50,000 headlines retained only 17,348 unique after deduplication, meaning 65% were removed as near-duplicates. The iterative approach with global deduplication and novelty steering produced the final 502,536 unique headlines. The final iteration generated a specific example headline about ThermoGlow's Q2 beat and FY guidance.

The resulting dataset, FinHeadlineMix, was released for headline classification, fine-tuning, and distillation. It supports NVIDIA's AI Model Distillation for Financial Data developer example. A 3-billion-parameter student model trained on 25,000 labeled examples reaches 95% of a 49-to-70-billion-parameter teacher's F1 score on headline classification. The pipeline used NeMo Data Designer version 0.1.5, NeMo Curator version 1.0.0rc0.dev0, and vLLM version 0.12.0.