Coinbase tuned an open 9-billion-parameter model and beat Opus 4.5 on all four detection metrics of its fraud benchmark. Training cost under $100. If you are a CTO, that number is the one that will show up in your next budget meeting, and it explains the least.

Four conditions explain it, and Coinbase had all of them: a narrow decision, labeled outcomes of its own, an eval it controls, and money to keep the model alive. Without all four, tuning is an expensive experiment. I covered how Bridgewater did it with financial documents. Coinbase adds what that case never showed: the test set, the baseline, and the cost of running the thing.

What Coinbase measured, and what it left out

The primary source is the Owning Intelligence series, three parts from October 6 to 8. Part 3 is signed by Coinbase engineers and the Anyscale team, the platform they trained and serve on. Keep that in mind while reading the results.

The task is classifying the risk of a transaction on Onramp, Coinbase’s crypto purchase product inside partner apps, from recent behavior. They start from Qwen3.5-9B and train it with reinforcement learning against known historical fraud, using a deterministic reward and no second model as judge. One RTX PRO 6000 with 96 GB.

The benchmark holds 16,140 transactions from 7,293 users, taken from nine weeks of production before the agent shipped, with 813 confirmed fraud cases. Against Opus 4.5, the tuned model gained 7.5 points of precision, 12.0 of recall, 9.6 of F1 and 35.4 of dollar-weighted recall. The figure caption says these are percentage-point differences, not relative gains. In production, median latency dropped from 1.515 to 0.683 seconds (55%).

Several things are missing from the series. Absolute F1 scores appear only in a chart. There is one baseline, Opus 4.5. Detection and latency were measured in separate evaluations. And across the three parts I found no comparison against a conventional classifier trained on the same labels. The LinkedIn summary by Hasanain Manesia says “legacy frontier models,” plural. Coinbase compared against one.

Why the baseline holds up

Several commenters on that post doubt that Opus 4.5 counts as frontier. Coinbase’s Part 2 answers with its own data. They ran Opus 5 on the same benchmark and it scored below 4.5 on every metric. Sonnet 5 lost 22.2 points of recall against Sonnet 4.6. GPT-5.6 (sol) gained 11.5 points of precision and lost 20.7 of recall.

If the setup was the same, the tuned model beats Opus 5 too. The series suggests it without saying it, so that part is my inference.

What matters to me is something else. Without that benchmark, Coinbase would never have known an upgrade made its detection worse. The test was the asset and the model came second. I have argued before that the benchmark only you write is the advantage that does not expire.

The cost that is not in the $100

One commenter put it briefly: maintaining and upgrading the model becomes your perpetual responsibility. Coinbase agrees. Their words: “maintaining quality requires continued evaluation and retraining.” They ask teams to version weights, prompts and decision policy together, with a tested rollback path.

Outside the $100 sit the benchmark, the labels, the serving fleet (L40S GPUs, 2 to 8 replicas) and the people watching it.

When not to tune

When there is no decision you can score. When you do not know how each case ended. And when a classic classifier is enough. Another commenter raised that, and the post’s author replied that in this case a classification model would be better suited.

Vendor numbers deserve the same suspicion. Steve Nouri’s post on Nemotron 3.5 Super VL carries the #nvidiapartner tag. On NVIDIA’s blog the 9% to 44% gains across five tasks come from partners, and Skydance’s 40x cost cut is written in the future tense (“will decrease”). VentureBeat notes NVIDIA gave no task definitions, sample sizes or protocol, and that these are separate tuned versions per task, not a general win for the base model. That base model scores 26 on the Artificial Analysis index, below GLM-5.3-Flash (42) and Qwen3.8-Flash-Next (40).

What we do at IQ Source

I would not tune anything before the test exists. In the discovery of AI Maestro, that is where we start. We pick the decision, build the case set with known outcomes, and score a frontier model with the best prompt anyone can write. If it already clears the threshold, there is nothing to build. If it does not, we ask whether the four conditions hold before anyone talks about training.

If your team repeats one decision every day and you know how each case ended, that is your raw material. The same case set also picks between models, and it may be the first thing worth building.