All Four Bonsai Sizes Now Run in Your Browser

Open a tab on aitherium.com, pick a brain size, and a language model downloads into your browser and starts answering — on your GPU, through WebGPU, with no server in the loop. No account, no API key, no Hugging Face token. Close the tab and it's gone; open it again and the weights come back from cache.

As of this week, that works for every size in the Bonsai family:

That table is the news. Until this week only the 27B ran in-browser, because our WGSL kernels implemented just the hybrid architecture — and 3.6 GB is nobody's idea of a phone download. The three dense sizes needed their own attention path (ungated projections, a different norm layout, no SSM state), a per-layer architecture classifier that reads each tensor's measured shape rather than trusting the file's architecture string, and 2-D dispatch so long prompts don't exceed WebGPU's 65,535-workgroups-per-dimension grid limit.

The 236 MB 1.7B is the size that changes who this is for. It is small enough for a phone to hold, fast enough to feel live, and — the part we weren't sure about — capable enough to run an agent.

Verified, not vibes

Every size is validated the same way: run the byte-identical GGUF through llama.cpp on CPU, then through our runtime in a real browser, and compare answers on fixed prompts. When both stacks say "Paris." and "1, 2, 3, 4, 5" from the same file, the runtime is doing what the reference does. All four sizes pass, and the harness that checks them scores degenerate output explicitly rather than trusting length or letter ratios.

The numbers, with their variance

Measured on an RTX 5090 in headless Chrome, greedy-ish decoding, short prompts. These are full ranges across every run we did, not best samples — decode throughput on a shared desktop GPU swings meaningfully run to run, and a single flattering number is how benchmarks lie:

For scale: the same 27B file through llama.cpp on the same machine's CPU/GPU node does ~50 tok/s single-stream. The browser is not the fast path and doesn't claim to be — it's the zero-install path, and the small sizes are quick enough there to hold a conversation.

These numbers are also several times better than they were at the start of the week: a per-token profiling probe (?timing=1 on our harness) took the GPU forward pass from 116.8 ms to 23.9 ms per token on the 1.7B by eliminating per-allocation queue submits in the buffer pool. The details live in our debt ledger; the receipts ship with the runtime as reproducible harnesses.

An agent on one bit per weight

The interesting question was never throughput — it's whether a model this small can do anything. The greeter on our landing page is a real agent: it can open apps, start a model download, or show the installer, through a deliberately tiny tool grammar ([[do:open sprite]]) with a whitelist — no verb in the vocabulary can spend money, write to disk, or touch an account.

We benchmark tool-calling the way we benchmark decoding: a fixed six-case suite, scored strictly, unanswered cases counted as no-data rather than as passes.

The 1.7B opens the right app on request, downloads the model you asked for, and — the case small models usually fail — declines to act on questions that just want an answer.

It also taught us the loop's most important rule. The agent observes what its action did and may continue; told "the sprite window is open," the 1.7B's instinct is to open two more. Small quantized models do not honor polite instructions like "only act again if needed" — so the bound is structural: a step whose actions all succeeded may speak but not act again. Only a failed action earns another attempt, because recovery is the reason the loop exists.

Phones

Phones now take the in-browser path too. The auto-selected size on mobile is the 1.7B or 4B — never the 27B — and nothing above ~600 MB downloads without an explicit tap on a button that shows the size. The loader was also taught not to ask a phone for a 341 MB allocation in one fetch, which is the kind of thing you learn when a real Pixel meets your code minutes after you ship it. For phones that want the stronger ternary builds instead, the one-line installer remains the better path — the browser and a local node find each other automatically.

Honest footnotes

The runtime is two weeks old. Getting the dense sizes working surfaced real bugs — the kind that never raise errors — and fixing them is what also made the 27B noticeably more coherent this week; the full postmortem lives in our engineering ledger rather than here. The 27B in a browser remains slow, and the fastest experience is still a local node, which the page auto-detects and prefers. Ternary (Q2_0) kernels for the higher-quality builds are wired but not yet GPU-validated against real weights. And every number above came from our own machine — your GPU will differ, which is rather the point: it runs on yours.