Search That Gets Faster Every Time You Use It

A search company posted a benchmark this week: 62ms P50 against Exa's 372ms, captioned "we just killed Exa." It's a genuinely good number, and it's the right thing to optimize — agents don't search at human speed, and every hundred milliseconds you spend on retrieval is latency your reasoning never gets back.

We had one constraint when we went after it: don't buy anyone's index. No Brave key, no Exa key, no Octen key. Whatever we shipped had to run on hardware we already own.

Here's what came out of that.

A ladder, not an endpoint

Most search APIs are one thing: a request goes out, an index answers, you pay. AitherSearch is four things stacked by cost, each one answering if it can and stepping aside if it can't.

Three of those four tiers never touch the network. The fourth one is what makes the other three smarter, and that's the whole design.

The corpus builds itself

Every live search writes back: the synthesized answer, the sources, and a semantic index over the whole thing. That corpus is the product. It isn't a cache in the "hold a response for 300 seconds" sense — it's an accumulating body of research that our agents mine before they ever consider going outside.

Which means the interesting question is not "how fast is a search?" It's how fast is the tenth question about a subject you've touched once?

Ask AitherSearch about Python async patterns and it goes to the web once, at web speed. Ask it eleven differently-worded versions of that question over the next month and it answers all eleven from its own memory, in tens of milliseconds, at zero marginal cost.

The corpus on a node is deliberately bounded — it runs against a token budget and decays entries by relevance, so it stays a working set rather than an archive that grows until it slows down. What survives is what the fleet keeps asking about.

The tier that catches what keyword matching can't

TF-IDF is fast and cheap and completely blind to a paraphrase with no shared vocabulary. So when it comes back weak, we spend half a second on a 4096-dimension embedding of the query and search the corpus by meaning instead of by words.

The result that sold us on it:

query : "which restaurants should a foodie visit in LA"
match : "best places to eat los angeles"

  TF-IDF cosine   0.327   -> under its gate, would have gone to the web
  embedding       0.631   -> served from corpus

Not one shared keyword. The embedding tier found it anyway, and the answer came back without a network call.

That tier runs on our own DGX, next to the reasoner, on the same box that serves the rest of the stack. No API key, no per-token bill, no third party learning what our agents are curious about.

The thresholds behind it are measured, not guessed. On the live corpus:

The gate sits at 0.55, in the gap. Above it we serve from memory; below it we go to the web and grow the corpus. Both outcomes are wins — one is fast, the other makes the next one fast.

It repairs itself

The embedding model shares a GPU with our 27B reasoner. When memory got tight it fell over, and the tempting fix was to shrink the reasoner's KV cache to make room.

We measured instead: the reasoner's KV holds 145,729 tokens against a 131,072 context — 1.11×. Cutting it would have permanently cost the ability to serve one full-context request. Trading reasoning depth for search recall is a bad trade, and it would have been invisible until the day someone sent a long prompt.

The real answer was ordering, not capacity: free the neighbour, load the model into the gap, restore the neighbour. Everything fits when it boots in the right sequence, and the reasoner was never touched.

That recovery is now a routine that runs every twenty minutes without a human. It only acts on the specific stuck state — an unreachable endpoint is a network fault and gets skipped, because bouncing two GPU services over a failed HTTP probe is repairing the wrong layer — it holds a cooldown so a genuinely dead model can never flap its neighbour, and it restores that neighbour on every path including its own failures.

Self-healing infrastructure isn't a slogan here. It's the difference between a tier that's up and a tier that's up at 3am.

Constant versus slope

Here's the honest comparison, including the part that doesn't flatter us.

On a question nobody has ever asked, we're slower. A cold topic costs us 1.7–2.5s, because that's what the open web costs when you refuse to rent an index.

On anything we've seen the shape of — the second question on a subject, the tenth paraphrase of it, the follow-up an agent asks itself while reasoning — we serve in 15–40ms, under the 62ms that was just announced as the fastest agent search on Earth, from our own corpus, on our own metal, with nothing billed and no one else in the path.

A hosted index gives every customer the same number forever. That's a constant. Ours converges toward ~30ms on precisely the questions our agents actually ask, and every miss makes the next hit cheaper.

One is a constant. The other is a slope. Over the lifetime of an agent that asks thousands of related questions, the slope wins — and it keeps winning on hardware you already paid for.

That's the part a benchmark screenshot can't show you: not how fast one query was, but which direction the curve is pointing.