Browser Memory Meets AI: Real Performance, Real Savings, Real Control

Browser Memory Meets AI: Real Performance, Real Savings, Real Control

We've built a new lane for session memory in the browser — capture → chunk → embed → query, all running on-device and entirely in the browser. This isn't just a demo or a theory — it's a real stack, running in real Chromium, with real numbers. No guessing. No speculation. Just performance, savings, and control — all measured and measured again.

The Numbers That Define the Experience

A 384-dimensional embedding model loads cold in under 3 seconds — config, tokenizer, and the quantized ONNX (~23 MB) stream from our live mirror (artifact.aitherium.com), never from a third-party. The ONNX runtime compiles in the same breath. The model is all-MiniLM-L6-v2 (Apache-2.0), the same one that powers retrieval for agent sessions.

Once warm, a single embedding costs 10 ms — and the worker batches: about 80 texts/sec at batch 128, on a CPU thread the page never notices. The lane is a dedicated WASM worker, so the main thread and the decode queue are untouched.

A 300-Page Conversation, Measured and Managed

The honest test of "long conversations in the browser" is what happens to the store after a few hundred captured pages. We simulated 300 pages through the real stack:

• 900 chunks stored in IndexedDB (900-char windows, the safe size for 128–512-token embedders).
• Diff-indexing paid for itself immediately: re-capturing 20 unchanged pages cost zero embed calls — 60 chunks skipped by content hash. A page with one changed section re-embeds only that chunk; the rest keep their stored vectors.
• Queries stay fast at scale: median 40 ms across 25 queries over the full store — brute-force cosine wins because "session-shaped" really is small.
• Recency shapes the answer, it doesn't override it: every hit is cosine + an exponential decay with a 24-hour half-life, so the chunk from three days ago still surfaces if it's the only good match, and the one from this morning wins the ties.

What the User Can See and What They Can Control

Consent first: the browser brain never downloads anything unasked. The Bonsai lane shows the bill before you pay it — the model picker names every size with its download cost (1.7B: 236 MB · 4B: 545 MB · 8B: 1.1 GB · 27B: 3.6 GB), and nothing over 600 MB auto-boots without asking. The embedding lane is lighter still: one 23 MB download, cached forever, one time per origin. A 300-page session with 900 retrievable chunks settles at 34 MB of origin storage total — most of it the cached model, not the memories.

What you can't yet see — and what's next: a live resource meter for the session-memory lane. The stack already reports its stats (chunk count, source count, embedder dimension and provider) through a typed surface; surfacing that in the UI — chunks stored, model loaded, last query time — is the remaining piece, and it's a small one. The browser isn't a black box that happens to be running an AI; it should be able to show you what it's doing.

The Architecture That Makes This Possible

At the heart of this system is a robust, secure, and efficient architecture that ensures performance, privacy, and control. The embedding model is hosted in a secure, live mirror, ensuring that users always get the latest, most optimized version without ever downloading untrusted weights. The entire pipeline runs in the browser, using WASM workers to isolate the AI processing from the main thread, ensuring that the user experience remains smooth and responsive.

This architecture is not just a technical achievement — it's a shift in how we think about the browser as an AI platform. No more waiting for cloud services to handle your data. No more guessing at performance. With this system, you get the power of AI in your browser, with full transparency, control, and performance that you can measure and trust.

What This Means for You

This architecture means that you, as a user, can now trust your browser with your data, your conversations, and your AI-powered experiences. You can see exactly what's being used, how much it's costing, and how it's impacting your performance. You can choose the model that fits your needs, and you can do it all without compromising on speed, security, or privacy.

This is the future of the browser — one that's not just for the web, but for the AI that powers it. And it's all running right in your browser, without needing your GPU or your cloud.