Fine-Tune an Embedding Model from an LLM for the Price of a Coffee

Fireworks AI published a recipe for fine-tuning an LLM-based embedding model on domain-specific data, claiming it costs about as much as a coffee and takes minutes of training time. The approach starts from a strong pretrained embedding LLM, such as Qwen3-Embedding, and adapts it with a contrastive objective on pairs of queries and relevant documents. The company reports gains of up to 39% in nDCG@100 and 40% in Recall@100 on legal citation retrieval, while keeping the model's general-purpose strengths intact.

Why fine-tune instead of starting from scratch

Fireworks AI argues that off-the-shelf embedding models are excellent generalists but fail on specialized corpora. A base model does not know that a legal citation and a case name refer to the same authority, or which eligibility criteria match a patient to a clinical trial. Training an embedding model from scratch is described as a trap for most teams: with modest labeled data, the model overfits the training distribution and loses the broad linguistic competence that makes embeddings robust. Fine-tuning, by contrast, nudges the geometry of the vector space toward the domain without rebuilding it. The company says a useful run takes about 150 optimizer steps, with either full-parameter updates or a small LoRA adapter.

How the contrastive training works

The training data consists of query-positive pairs. For legal citation retrieval, the query is a court paragraph with its citation masked out, and the positive is the case it cites. For clinical trial matching, the query is a patient summary and the positive is a trial they are eligible for. The objective is contrastive: pull each query close to its positive and push it away from everything else. The efficiency trick is in-batch negatives: within a batch of B pairs, each query has one correct document, and the other B-1 documents serve as free negatives. The model computes a B by B similarity matrix, applies the InfoNCE loss in both directions, and averages the two losses. A temperature of 0.02 sharpens the softmax so positives win clearly. Fireworks AI notes that more advanced hard negative mining and cross-batch negatives could further boost performance, but the basic setup already works well.

Experimental results

Fireworks AI evaluated the fine-tuned model against the base Qwen3-Embedding-8B on three benchmarks. For LegalBench citation retrieval, with 4,899 questions over a 6,061-document corpus of U.S. court opinions, fine-tuning improved nDCG@100 from 0.462 to 0.644, a 39% gain, and Recall@100 from 0.540 to 0.758, a 40% gain. Other metrics also improved: nDCG@10 by 36%, Recall@10 by 43%, MRR by 17%, and MAP@100 by 61%. For TREC Clinical Trials, with only about 130 training queries, fine-tuning improved nDCG@100 from 0.495 to 0.556, a 12% gain, and Recall@100 from 0.303 to 0.352, a 16% gain. The company also tested on EU case-law citation retrieval with LegalPincite. All evaluations used held-out queries and the standard pytrec_eval library.

When fine-tuning did not help

Fireworks AI mentions that fine-tuning did not move the needle in some cases. The company suggests that for general-purpose text, the base model is already strong, and fine-tuning may not add value. It also notes that the recipe is designed for domain-specific retrieval, not for improving general embeddings.

Getting started

Fireworks AI offers three training modes on its platform, balancing flexibility and efficiency. The post includes a code example for training on private data. The overall message is that adapting an existing LLM-based embedding model with a few hundred contrastive steps can close most of the gap between a generalist and a domain expert, at a fraction of the cost of training from scratch.