The #1 MTEB Embedder Now Runs Next to a 27B on 120 Watts

An embedding model turns text into a point in space, and everything downstream — search, retrieval-augmented generation, memory — lives or dies on how well-separated those points are. Our production embedder, nomic-embed-text-v1.5, is a 137M-parameter workhorse: fast, cheap, 768 dimensions. NV-Embed-v2 is the other end of the spectrum — 7.85B parameters, 4096 dimensions, and the model that sat at #1 on the MTEB leaderboard with a 72.31 average across 56 tasks (nomic's is ~62.3).

Ten MTEB points is an abstract number. So we measured the gap ourselves, on our own hardware, twice — once on CPU to validate the model, once through the served endpoint.

The margin experiment

We took 24 real query→passage pairs from Natural Questions and asked each model to rank all 24 passages for every query. Both models got every single answer right — accuracy@1 of 1.000, a saturated test. The interesting number is the margin: how far the correct passage's cosine score sits above the best wrong passage. Margin is your safety buffer — it's what stands between "right answer" and "confidently wrong answer" when the corpus gets bigger and the distractors get closer.

NV-Embed-v2 separates the right passage from the nearest distractor 2.4× more cleanly on average, and 3.7× more cleanly in the worst case. That worst case is the number that matters: nomic's tightest call was a 0.059-cosine whisker — the kind of gap that flips to a wrong retrieval when the corpus grows tenfold. NV-Embed-v2's tightest call kept a 0.220 buffer.

Exactness, verified twice

NVIDIA publishes an exact expected score matrix for the model card's example — two queries, two passages, scores scaled ×100. Our CPU validation run (float32, pinned revision) reproduced it to about six decimal places. The served endpoint — bfloat16 on the GPU — reproduces it to within bf16 tolerance:

Same model, byte-pinned revision, quantization drift under two tenths of a point on an 87-point score. If your deployment can't reproduce the model card, you're not serving the model you think you are — so we check.

Three models, one 120-watt box

The serving venue is the DGX Spark — a GB10 Grace Blackwell SoC with 121 GB of unified memory at 120 W. NV-Embed-v2 didn't get the box to itself. It moved in next to the residents:

That packing is a budget, not an accident: every gigabyte of KV cache, weight file, and activation headroom is accounted for, and boot order is sequenced so each engine sizes its cache against a settled picture. The payoff shows up in the load path too — the 16 GB of bf16 weights stream from NVMe straight into GPU memory in 5.7 seconds (shard-direct loading; the naive load path takes minutes and twice the transient memory). The weights themselves downloaded at fleet speed: 16 GB in 2 minutes 21 seconds.

Where it fits

NV-Embed-v2 is CC-BY-NC licensed, so it serves internal surfaces only — the memory graph, research retrieval, agent context ranking — while nomic (Apache-2.0) remains the commercial embedding lane on the RTX 5090, where it embeds 48 texts in 6.4 seconds and costs a rounding error to run. That split is the point: the cheap fast embedder handles volume, and the #1-on-MTEB embedder handles the retrievals where a 0.06 margin isn't good enough.

One box under a desk now runs a 27B at full context, a 12B that reads images, and the best open embedding model on the leaderboard — simultaneously, on less power than a bright light bulb.