Bonsai 2 on a 5090: Better Answers at Half the Tokens

Verdict: run Bonsai 2, and pick the PTQ1_0 quant. On our ruler it beats Bonsai 1 by 17% relative, answers every item, and uses 44% of the tokens. The 5.95 GB ternary file scores identically to the 7.2 GB PQ2_0, so take the smaller one. It fits beside a 27B sibling on a 32 GB card with room to spare.

PrismML released Ternary Bonsai 2 27B on 2026-09-17. Two days later it is built, running and measured on our hardware, and it is the best small brain we have put on a single consumer GPU.

The numbers

AitherBench is the ruler we use to decide what runs the platform. Forty-five items, every one scored by exact match, set membership or an ordering predicate. No LLM judge. It asks the questions an agent platform actually needs answered: calibrate effort, plan in dependency order, route to the right model, resist the answer that pattern-matching hands you, and tell "I could not check" apart from "it is fine."

Both models served on the same RTX 5090, 16384 context per slot, 8192-token budget, temperature 0.

The overall delta is 0.134, about 3.5× the benchmark's minimum detectable effect, so this is a real gap and not a lucky item. The two quants landing on the same number to four decimals means the ternary compression is free at this size.

Where it wins

The gains are exactly where an agent brain needs them.

Search and planning are the two best-powered dimensions in the suite (six and seven items), and search is a clean sweep. Logic traps at 1.00 means it stops taking the bait on the questions written to look like a familiar puzzle but are not. Long horizon and REPL going to 1.00 is what lets it hold a multi-step task together instead of answering the first step and drifting.

It does all of this in fewer than half the tokens. Shorter answers on a local card are not a nicety; they are the difference between a chat that feels instant and one you wait on.

It is robust to how you run it

Small quantised models sometimes fall over when a casual install differs from a tuned lane: higher temperature, an open-ended thinking block, no output cap. We ran the grid so you do not have to.

Twenty-four factual questions with short verifiable answers, three samples each, across temperature 0 and 1.0, thinking on and off, with and without a server-side reasoning cap: 192 calls, 100% correct, zero empty responses, zero truncations. Run it at the recommended temperature 1.0 with thinking on and it behaves.

The one thing to know

Bonsai 2 is more willing to answer than Bonsai 1. On the one item in the suite that rewards saying "I don't know," Bonsai 1 declined and Bonsai 2 answered. Factual recall itself did not move: the "state a known fact" item is 1.00 on every model here. If your use case is a raw general-purpose chatbot with no checks around it, Bonsai 1's caution is worth something. Inside AitherOS the platform verifies claims before it acts on them, which is the setting these numbers were measured in and the setting we recommend.

Running it

Bonsai 2 needs PrismML's llama.cpp fork. Its tensors carry a Walsh–Hadamard activation rotation in their metadata; stock llama.cpp loads the file and produces fluent, confident, wrong text behind a healthy /health endpoint, which is the worst kind of failure. The AitherOS runtime image builds the fork and applies the transform, so selecting Bonsai 2 in the model catalog gets you the model that scored above and not a look-alike.

While we were at it, the benchmark harness now registers each run before the first token and streams per-item progress to the same surface training runs use. You can watch an evaluation at 2/45, 20/45, 45/45 from the portal instead of waiting for a file to land.

Bottom line

Bonsai 2 PTQ1_0: 5.95 GB, 0.9165 on AitherBench, 45 of 45 answered, a third of a Bonsai 1 answer in tokens, on one RTX 5090. It is the brain we would put on a desk today.