Open-source workers with a closed-source advisor cut costs and lift quality

Fireworks AI published results on June 24, 2026, showing that an open-source worker model that consults a closed-source frontier model once per task as an advisor improves success rates on every benchmark cell tested, while cutting inference costs by 19% to 67% compared to running the frontier model as the worker.

The setup assigns two roles. The worker, either Kimi-K2.6 or GLM-5.2, runs the task end to end: it reads the problem, edits files, runs commands, and verifies its own work. At a single review step, it calls a read-only advisor, Claude Opus 4.8, which cannot edit files. The advisor reads the worker's trajectory and the working diff, then returns an assessment with pass/fail checks. The worker decides when to consult the advisor, after it has done the work and run its own verification. The advisor audits what was actually done, flags gaps, and the worker does additional work if needed. There is no external router or orchestrator.

Measured lifts and costs

Across three benchmarks, the advisor raised resolve rates for both workers. On SWE-bench Pro, Kimi-K2.6 went from 55% to 59% and GLM-5.2 from 59% to 66%. On Terminal-Bench 2.1, Kimi went from 64% to 72% and GLM from 76% to 80%. On the Legal Agent Benchmark, Kimi went from 8% to 9% and GLM from 12% to 16%. The Legal benchmark is scored as all-criteria-pass over 100 tasks, a stricter bar, so absolute numbers run lower.

Cost per task for the open-source plus advisor setup undercuts Opus-as-worker in every cell. On SWE-bench Pro, the Kimi setup cost $6.13 versus $18.28 for Opus, a 66% saving, and the GLM setup cost $6.09, a 67% saving. On Terminal-Bench, Kimi cost $4.11 versus $6.61, a 38% saving, and GLM cost $3.50, a 47% saving. On Legal, Kimi cost $7.73 versus $9.54, a 19% saving, and GLM cost $5.74, a 40% saving. GLM-5.2 plus advisor matched Opus-as-worker on Terminal-Bench at about half the cost and beat it on Legal at about 40% lower cost.

Fireworks disclosed caveats: with only 60 to 100 tasks per cell, a one-task swing is about 1 to 1.8 percentage points, so exact magnitudes are subset estimates. The Terminal-Bench tie between GLM-5.2 plus advisor and Opus-as-worker, 80.5 versus 79.5, is within sampling noise and should be read as parity. The Legal cell with Kimi-K2.6 plus advisor, a one-task lift, is also within noise and should be read as flat.

Ablations and next steps

Fireworks tested a stronger harness with two advisor calls, a plan call before building and a review call before finishing. Review-only was as good or better in five of six benchmark-worker experiments. It was distinctly better on Terminal-Bench with Kimi-K2.6, 72% versus 63%, at roughly half the advisor calls.

Replacing the frontier advisor with a same-model reviewer did not reproduce the lift. GLM-5.2 reviewing its own work was flat on SWE-bench, 58% versus 59% base, and worse on Terminal-Bench, 72% versus 76% base. A same-tier GLM-5.2 reviewer over Kimi-K2.6 actively degraded Legal results, 8% to 6%, where the frontier reviewer was neutral-to-positive. Advisor effort set to medium was the default; high effort landed within one task on SWE-bench, so medium was cheaper and faster at no quality cost.

The advisor is open-sourced as a single self-contained file in the Fireworks cookbook. The review call is one prompt designed as a skeptic, told to distrust the worker's framing and prose, verify claimed edits against the worktree, and score each issue with calibrated confidence where only scores of 80 or above become critical. The advisor reads the git diff as ground truth and marks each check as passing, failing, not implemented, or deviated.

Fireworks noted the six-cell sweep is a snapshot and listed next steps including more task types and models, such as data analysis, web, and longer workflows, to confirm the uplift holds as both open and frontier tiers advance.