We Gave Our Agent's Imagination a Harder Exam — Then It Played Its Way to a Better Score
Earlier we shipped a world model whose imagination — its prediction of what each move will do — beat the "nothing changes" baseline 86% of the time, and promoted it to production the same afternoon. That post ended with a promise: the gate that made the jump routine now stands watch over every future checkpoint. This is the first thing it caught, and the first thing it let us ship because of it.
Ask the score a harder question
86% is a great number — but our agent plays a rotation of games, and one of them (call it ls20) it had mastered cold. So we asked the uncomfortable question every honest benchmark should face: is 86% the model being good, or one game carrying the average?
We re-ran the exam with every game weighted equally — no single title allowed to dominate the score. The imagination dropped to 56%. Barely above a coin flip once its best game stopped flattering it. That's not a disappointment; that's the measurement working exactly as designed. You can't fix a gap you can't see, and now we could see it precisely: the model was a specialist, strong at home and thin everywhere else.
The fix was already happening — the agent just had to train on it
Here's the elegant part. The agent had been playing the whole time, and every move it makes is captured into the model's memory. We looked at that memory and found it had quietly rebalanced itself: the over-learned game had fallen from 79% of the training data to 54%, and the harder games now carried two to five times more real, varied experience than the stale snapshot the model had last trained on.
So the fix wasn't a new loss function or a bigger model. It was use the experience the agent already generated. We retrained on the fresh, accumulated play — and the imagination got better everywhere at once: 92% on its home game (up from 86%) and 66% on the honest cross-family exam (up from 56%). Same architecture. The only change was training on what the agent taught itself by playing.
The gate's real job: catching the shortcuts that don't work
The reason we can take swings like this is that the gate lets us fail cheaply. Two tempting ideas got tested and rejected on the way to the win:
- The obvious shortcut — synthetically rebalance the old data by repeating the rare examples. It cratered to 15%. Replaying a handful of transitions louder isn't the same as having more of them; the gate said so in eleven minutes and we moved on.
- The fashionable technique — a popular auxiliary training objective that, in theory, should sharpen the model. Measured head-to-head, it cost nineteen points of prediction skill. So we didn't ship it as a training signal — we found a way to get its one genuine benefit (calibrated confidence scores, so the planner knows when to trust a prediction) with the predictor frozen and provably untouched. Same skill, plus the imagination now tells the agent how sure it is.
None of those dead ends reached production. That's the entire point of a gate: it turns "we think this helps" into "we measured it, and here's which half was right."
Now it improves on its own
The deepest lesson is that our agent gets smarter the more it plays — the buffer self-balances, the hard games accumulate, and a retrain turns that into sharper foresight. So we closed the loop: a scheduled job now retrains on the freshest play each week, runs the same honest exam, and promotes the result only if it strictly beats the model in production — banking a rollback every time. No human in the critical path, but the gate in the critical path.
From "is 86% even real?" to a model that's better on every game and keeps improving by itself — in a single day, with every number reproducible from the exam that produced it. The imagination is live at arc.aitherium.com, and it's still playing its way smarter.