A 5.9 GB Model That Also Sees

Verdict: make the ternary 27B your default multimodal lane, and keep a small dense multimodal model around only for literal transcription of small text. On one RTX 5090, Bonsai 2 27B — 7.2 GB of PQ2_0 weights plus a 629 MB vision projector — answered every image question we put to it faster than gemma4-12b, sustained 54.6 tok/s against 9.5, and got a bar-chart ordering right that the 12B got wrong. The one thing it lost was character-level OCR: it read "AITHERIUM" as "AETHERIUM".

That trade is worth stating plainly, because it is the whole decision. A model whose weights are three values reasons about what is in a picture as well as a dense model five times its bit-width, and stumbles on the exact shape of small glyphs.

What ternary means here

Every weight in Bonsai 2 is -1, 0 or +1. Embeddings, attention, MLP and the LM head are all ternary; 26.2M parameters — 0.0976% of the model — stay in higher precision, and the headline 1.76 bits per weight counts them. PrismML released it under Apache 2.0 with a vision tower included, which is the part most people miss: this is not a text-only compression demo. It takes images.

One engine caveat, one sentence: the weights carry a Walsh-Hadamard activation rotation, so they need the PrismML llama.cpp fork — stock llama.cpp loads the file happily and answers wrong.

The head-to-head

Both models, same prompts, same generated PNGs, temperature 0.3, called from inside our fleet. Bonsai 2 on the 5090 through llama.cpp; gemma4-12b on a DGX Spark through vLLM.

Three results carry the decision.

It reasons over the image, not just labels it. Asked to order four bars by height, the 27B returned red < orange < blue < green, which is the picture. The 12B named a tallest and a shortest correctly and then produced an ordering that was not the chart. Labelling is not reading.

It does what you asked. Given "transcribe every line of text", the 12B returned its bounding-box detection JSON — accurate content in a format nobody requested. Downstream, that is a parser you have to write and a failure mode you have to handle.

It sustains real throughput. 54.6 tok/s on a 400-token answer against 9.5 is not a micro-benchmark difference; it is the difference between an agent that streams and one you wait for. Short answers read lower on both models because prefill dominates — and 54.6 is itself under the 130-143 tok/s the model card quotes for this format on a 5090, because our card is shared with two other resident models.

What it means for video and for puzzle solving

Neither model ingests a video file. Both take a sequence of frames as images in one request, which is how frame-sampled video reasoning actually works in practice. On a three-frame clip the 27B named both the moving object and its direction while the 12B named only the direction, in a third of the time.

Then the frame count decides it. Asked for five frames, the 12B's server answered At most 4 image(s) may be provided in one prompt — a hard cap, and four frames is not a clip. The ternary 27B took 5, 8 and 16 frames and answered in 2.3 s at sixteen. One honest limit with it: in a clip where a circle moved and a dot grew, it reported both changes as motion at every frame count. It tracks position across frames well; it read a size change as a second trajectory. Ask it where things went, and verify anything you are told about size.

Faster frame reasoning is what makes "watch this and tell me when X happens" affordable at all: at 9.5 tok/s you sample sparsely and hope; at 54.6, with a sixteen-frame window, you can look often.

For grid-puzzle work the interesting result is that both models solved the ARC-style transform, and the ternary model did it in 2.2 s. ARC-style reasoning does not need pixels — the grids are structured data — so the win here is not vision at all. It is that the cheap model is now good enough to be the perception and hypothesis lane, and fast enough to try many hypotheses instead of one.

What to run

Default to the ternary 27B for multimodal work: describing, comparing, chart and diagram reading, frame sequences, and anything where the answer must come back in a format you specified. Keep a small dense multimodal model mounted for one job — transcribing small text exactly, where an extra character error is not acceptable.

And take the footprint seriously. 7.8 GB of weights plus projector, on one consumer card, co-resident with two other models, answering image questions in under two seconds. The reason to care about ternary is not that it is small. It is that small stopped costing you the answer.

Every number above is from one box, one day, one prompt per row — enough to choose a default, not enough to call a trend. The harness is bench_model_remote.py --endpoint <url> --served-name <name>.