<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:content="http://purl.org/rss/1.0/modules/content/"><channel><title>Ollama Embedding Models on RockB</title><link>https://baeseokjae.github.io/tags/ollama-embedding-models/</link><description>Recent content in Ollama Embedding Models on RockB</description><image><title>RockB</title><url>https://baeseokjae.github.io/images/og-default.png</url><link>https://baeseokjae.github.io/images/og-default.png</link></image><generator>Hugo</generator><language>en-us</language><lastBuildDate>Sun, 15 Nov 2026 08:55:36 +0000</lastBuildDate><atom:link href="https://baeseokjae.github.io/tags/ollama-embedding-models/index.xml" rel="self" type="application/rss+xml"/><item><title>One Fact per Vector: What Chunk Granularity Does to Agent-Memory Recall, Measured on Three Local Embedders (2026)</title><link>https://baeseokjae.github.io/posts/agent-memory-chunk-granularity-recall-2026/</link><pubDate>Sun, 15 Nov 2026 08:55:36 +0000</pubDate><guid>https://baeseokjae.github.io/posts/agent-memory-chunk-granularity-recall-2026/</guid><description>An 8-document, 13-question fixture on three local Ollama embedders: two-fact hit@1 falls from 13/15 with whole-document vectors to 0/15 at 3-sentence and single-sentence chunks, while mean top-3 context drops from 1,444 to 210 characters. Includes the probe script and fixture.</description><content:encoded><![CDATA[<p>An agent memory store is usually built by a chunker that nobody re-examines after the first week. One vector per conversation turn, per note, or per fixed block of text. The store looks healthy, because the questions used to check it are single-fact questions about a short note, and those pass at almost any granularity. Then a user asks a two-part question — &ldquo;how do access-token and signing-key lifetimes compare?&rdquo;, &ldquo;who owns the deploy key and how often does it rotate?&rdquo; — and the retriever returns one half of the answer with no error at all. The other half is in a different chunk and did not fit in the top-k.</p>
<p>The opposite mistake is just as common. Keeping whole documents to preserve context means every retrieval drags 1,400 characters of mostly irrelevant prose into the prompt instead of 210.</p>
<p>This article measures the tradeoff instead of asserting it. On one host, with three locally served embedding models, an 8-document fixture and 13 questions, the short answer is:</p>
<ul>
<li>Single-fact recall did not degrade as chunks got smaller (21/24 pooled whole-document, 23/24 at 3-sentence windows, 23/24 at single sentences).</li>
<li>Two-fact recall at k=1 collapsed from 13/15 to 0/15 the moment document-level vectors were replaced by 3-sentence windows, and stayed at 0/15 at sentence level.</li>
<li>The two facts usually survived somewhere in a deeper ranking: both required sentences appeared in the top 8 for 15/15, 13/15 and 12/15 respectively.</li>
<li>Mean characters in the top-3 retrieved chunks fell from 1,444 to 475 to 210 — about 6.9x less prompt context for the same k.</li>
</ul>
<p>So the decision is not &ldquo;smaller chunks retrieve better&rdquo; or &ldquo;bigger chunks preserve meaning&rdquo;. It is: what query shape do you serve, and can your retriever merge two chunks before the prompt budget closes?</p>
<h2 id="the-two-questions-your-chunker-decides-for-you">The two questions your chunker decides for you</h2>
<p>A chunk boundary is a retrieval contract. Whatever else you configure, two properties of your system are fixed the moment the chunker runs.</p>
<p>The first is <strong>composability</strong>: can any single retrieved unit contain all the facts that one realistic question needs? A sentence chunk can answer &ldquo;what is the retry interval&rdquo; and can never answer &ldquo;how does the retry interval compare with the escalation delay&rdquo;, because a comparison needs two sentences and one chunk holds one. Nothing errors. The ranker returns its best single sentence, which is a correct answer to half of the question.</p>
<p>The second is <strong>context cost per retrieval</strong>: how much text does k=3 hand to the model? With whole-document vectors, three hits are three documents. That is fine when documents are 480 characters and expensive when they are 8,000.</p>
<p>Both properties are measurable before you pick a chunk size, and neither is answered by vendor documentation. Vendor pages tell you the context window and the API shape, not what your boundaries do to your recall.</p>
<h2 id="what-the-fixture-measures-and-what-it-does-not">What the fixture measures, and what it does not</h2>
<p>Eight invented project documents (deploy, billing, auth, search, mobile, pipeline, support, infra), seven sentences each, 456–500 characters per document, 52–99 characters per sentence. Each document carries one needle fact, plus facts that only mean something when combined with an adjacent sentence. Eight single-fact questions (one per document) and five two-fact questions whose two required sentences sit on opposite sides of a 3-sentence window boundary.</p>
<p>Three granularities of the same corpus:</p>
<ul>
<li>whole document: 8 vectors per model</li>
<li>non-overlapping 3-sentence windows: 24 vectors (sentences 1–3, 4–6, 7)</li>
<li>single sentences: 56 vectors</li>
</ul>
<p>Three embedders served by local Ollama 0.20.3 through <code>/api/embed</code>: all-minilm:latest (384-d), snowflake-arctic-embed:33m (384-d) and nomic-embed-text:latest (768-d). Cosine on L2-normalised vectors, a deterministic full sort, no generation, no re-ranker, no hybrid search, no metadata filters. Each row reports single-fact hit@1 (the needle sentence is inside the top-1 chunk), two-fact hit@1 (both required sentences are inside the top-1 chunk), and the mean character count of the top-3 chunks. The full sweep took 54 seconds.</p>
<p>What this does <strong>not</strong> measure: answer quality, latency, throughput, token counts, production recall, or user satisfaction. hit@1 here means &ldquo;the right text was in the returned chunk&rdquo;. Nothing else.</p>
<h2 id="result-1-fine-chunks-cost-nothing-on-single-fact-questions">Result 1: fine chunks cost nothing on single-fact questions</h2>
<p>Pooled over the three embedders and the eight single-fact questions — 24 model-question pairs per granularity:</p>
<table>
	<thead>
			<tr>
					<th>Granularity</th>
					<th>Vectors per model</th>
					<th>Single-fact hit@1</th>
					<th>Two-fact hit@1</th>
					<th>Both facts in top 8</th>
					<th>Mean top-3 chars</th>
			</tr>
	</thead>
	<tbody>
			<tr>
					<td>Whole document</td>
					<td>8</td>
					<td>21/24</td>
					<td>13/15</td>
					<td>15/15</td>
					<td>1,444</td>
			</tr>
			<tr>
					<td>3-sentence windows</td>
					<td>24</td>
					<td>23/24</td>
					<td>0/15</td>
					<td>13/15</td>
					<td>475</td>
			</tr>
			<tr>
					<td>Single sentence</td>
					<td>56</td>
					<td>23/24</td>
					<td>0/15</td>
					<td>12/15</td>
					<td>210</td>
			</tr>
	</tbody>
</table>
<p>Per model, single-fact hit@1 moved 8/8 → 8/8 → 8/8 (all-minilm), 6/8 → 7/8 → 8/8 (arctic-33m) and 7/8 → 8/8 → 7/8 (nomic). Those are one-query swings on an eight-question set. The honest reading is &ldquo;no measurable damage from finer chunks&rdquo;, not &ldquo;fine chunks beat documents&rdquo;. Anyone who has read a single-fact recall table will recognise the shape: at 480-character documents, the whole-document vector is not diluted enough to hurt, and the small deficits are noise.</p>
<p>This is the result that makes chunking decisions go wrong. A team shrinks chunks, reruns its single-fact regression suite, sees 8/8 unchanged, and concludes the change was free. It was not free. The cost is simply invisible to that test suite.</p>
<h2 id="result-2-fine-chunks-lose-the-two-fact-answer-completely-at-k1">Result 2: fine chunks lose the two-fact answer completely at k=1</h2>
<p>Same fixture, five two-fact questions, top-1 only:</p>
<table>
	<thead>
			<tr>
					<th>Granularity</th>
					<th>Two-fact hit@1 (pooled)</th>
					<th>all-minilm</th>
					<th>arctic-33m</th>
					<th>nomic</th>
			</tr>
	</thead>
	<tbody>
			<tr>
					<td>Whole document</td>
					<td>13/15</td>
					<td>4/5</td>
					<td>4/5</td>
					<td>5/5</td>
			</tr>
			<tr>
					<td>3-sentence windows</td>
					<td>0/15</td>
					<td>0/5</td>
					<td>0/5</td>
					<td>0/5</td>
			</tr>
			<tr>
					<td>Single sentence</td>
					<td>0/15</td>
					<td>0/5</td>
					<td>0/5</td>
					<td>0/5</td>
			</tr>
	</tbody>
</table>
<p>Two mechanisms produce that zero, and they are not equally general.</p>
<p>At <strong>sentence granularity</strong> the zero is structural: one sentence cannot contain two sentences, so no ranking can ever satisfy the condition. Any corpus, any embedder, any question shape. If your questions ever require two facts, single-sentence chunks at k=1 are guaranteed to fail, and the failure is silent because the top hit is still a sensible sentence.</p>
<p>At <strong>3-sentence windows</strong> the zero is constructed. The five pairs were deliberately placed to straddle a window boundary, so each question&rsquo;s two sentences necessarily land in different windows. That is a worst case, and a corpus whose two-fact answers happen to fall inside one window would score differently. It is also not an artificial worry: any fixed-width window will straddle some fraction of real co-reference pairs, and the fraction grows as the window shrinks.</p>
<p>At <strong>top-3</strong> the loss softens but does not disappear: both required sentences appear among the three returned chunks for 15/15 whole-document, 10/15 at 3-sentence windows and 8/15 at single sentences. A retriever that merges its top-3 into one context window recovers some of the damage — but only if it merges, and only if the ranker put both halves inside k.</p>
<h2 id="result-3-the-facts-are-usually-still-there-just-not-together">Result 3: the facts are usually still there, just not together</h2>
<p>The most useful number in the run is the top-8 check. Both required sentences appeared somewhere in ranks 1–8 for 15/15, 13/15 and 12/15 across the three granularities. At sentence level, twelve of fifteen two-fact questions had both halves one deeper ranking away.</p>
<p>That reframes the fix. If your retriever can see the fragments, the problem is assembly, not indexing, and assembly is cheaper than re-chunking a live store:</p>
<ul>
<li><strong>Merge before the prompt.</strong> Retrieve 8–10 chunks, then concatenate neighbours from the same source document (or group by document id) before the model sees them. This converts a ranking problem into a grouping problem.</li>
<li><strong>Two-stage retrieval.</strong> Retrieve wide at fine granularity, then re-rank or expand the winning chunks to their parent window. The classic parent-document pattern.</li>
<li><strong>Keep a dual index.</strong> Fine chunks for precision, whole documents for context, and choose per query shape.</li>
</ul>
<p>All three need the store to record what the chunk <em>is</em>: source document id, sentence offsets, the encoder&rsquo;s identity and digest, and the vector&rsquo;s dimension. Without that metadata you cannot expand a hit to its neighbours, and you cannot tell whether a stored vector is comparable to a query vector. <a href="https://baeseokjae.github.io/posts/agent-memory-write-path-reliability-2026/">The write-path article covers the record each chunk needs</a>.</p>
<h2 id="result-4-what-coarse-chunks-cost-in-prompt-characters">Result 4: what coarse chunks cost in prompt characters</h2>
<p>Mean characters of the top-3 retrieved chunks, averaged over the eight single-fact queries per row: 1,444 for whole documents, 475 for 3-sentence windows, 210 for single sentences. That is 6.9x less text for the same k.</p>
<p>Two cautions, both of which matter more than the ratio itself.</p>
<p>It is characters, not tokens. No tokenizer was run, so this is not a cost figure and should not be converted into one. If you need tokens, measure them; the Ollama embed response carries <code>prompt_eval_count</code> specifically so you do not have to guess (<a href="https://docs.ollama.com/api/embed">embed API reference</a>).</p>
<p>The ratio is a property of <em>your</em> document lengths, not a constant. These documents are 482 characters on average, so a whole-document chunk is about six sentences&rsquo; worth of prose — the coarse setting here is only mildly coarse. At 8,000-character documents the same table would show a far larger ratio, and at 200-character notes it would nearly vanish. The transferable finding is the direction and the mechanism, not 6.9.</p>
<h2 id="does-the-embedder-change-the-answer">Does the embedder change the answer?</h2>
<p>No — the shape held on all three models. Every embedder went 4/5-or-5/5 → 0/5 → 0/5 on two-fact hit@1 while staying flat on single-fact recall. That consistency is why the effect reads as a property of the chunk boundary rather than of one model&rsquo;s geometry.</p>
<p>The nomic row needs a specific caveat. nomic-embed-text-v1.5&rsquo;s <a href="https://huggingface.co/nomic-ai/nomic-embed-text-v1.5">model card</a> states that the text must carry a task instruction prefix: <code>search_document: </code> for indexed text and <code>search_query: </code> for queries. This harness sent no prefixes to any model, and Ollama&rsquo;s own <a href="https://ollama.com/library/nomic-embed-text">nomic-embed-text library page</a> shows an example without them. So nomic&rsquo;s absolute numbers here are not a fair model comparison and must not be read as a ranking. The prefix omission applies in code to all three rows equally, but only nomic documents the requirement — meaning nomic is the row most likely to be depressed by the choice.</p>
<p>The practical consequence: choose the chunk size before you choose the embedder. The boundary failure shows up on every model tested; the encoder choice changes absolute recall, not whether a two-sentence question can be answered by a one-sentence chunk.</p>
<p>One sizing check belongs here, because it explains why truncation is not a confound in this run. all-minilm and snowflake-arctic-embed:33m both publish a 512-token context window on their Ollama library pages (<a href="https://ollama.com/library/all-minilm">all-minilm</a>, <a href="https://ollama.com/library/snowflake-arctic-embed">snowflake-arctic-embed</a>), and <code>/api/embed</code> defaults to <code>truncate: true</code>, silently clipping longer inputs. At roughly 482 characters per document — on the order of 120 tokens at a rule-of-thumb 4 characters per token — no indexed chunk here came close to that ceiling. That is a sizing argument, not a measurement: <code>prompt_eval_count</code> was not recorded in this run. The truncation ceiling itself is a different failure and is <a href="https://baeseokjae.github.io/posts/local-embedding-context-ceiling-2026/">measured separately</a>.</p>
<p>This also separates the result from the encoder-side failure measured in <a href="https://baeseokjae.github.io/posts/agent-memory-embedding-swap-recall-2026/">the embedding-swap harness</a>, where replacing an encoder with a same-width model turned ranking into noise. Here the encoder is fixed within each row and only the boundary moves, which is the only way to attribute the change to chunking.</p>
<h2 id="a-decision-procedure-for-your-own-corpus">A decision procedure for your own corpus</h2>
<p>The fixture is small on purpose: it makes the mechanism visible in under a minute, and it is meant to be replaced by your own documents. The harness is published — point it at your corpus, not at mine.</p>
<p>To reproduce this exact run you need Ollama serving the three models locally. Then:</p>
<pre tabindex="0"><code>python3 chunk_probe2.py fixture.json results2.json
</code></pre><p><code>chunk_probe2.py</code> records each needle&rsquo;s rank to depth 8 and reports <code>both_in_top8</code> alongside hit@1 and hit@3; <code>chunk_probe.py</code> is the earlier hits-only sweep. Both read the fixture JSON, call <code>http://127.0.0.1:11434/api/embed</code> only, and print one row per (model, granularity).</p>
<p>The published files, all original and inspectable before you run them:</p>
<ul>
<li><a href="https://raw.githubusercontent.com/baeseokjae/baeseokjae.github.io/main/research/evidence/agent-memory-chunk-granularity-recall-2026/chunk_probe2.py">chunk_probe2.py</a> — the probe, parameterised by fixture and output path</li>
<li><a href="https://raw.githubusercontent.com/baeseokjae/baeseokjae.github.io/main/research/evidence/agent-memory-chunk-granularity-recall-2026/fixture.json">fixture.json</a> — 8 documents, 8 single-fact and 5 two-fact questions</li>
<li><a href="https://raw.githubusercontent.com/baeseokjae/baeseokjae.github.io/main/research/evidence/agent-memory-chunk-granularity-recall-2026/results2.json">results2.json</a> — the raw per-row output behind every number above</li>
<li><a href="https://raw.githubusercontent.com/baeseokjae/baeseokjae.github.io/main/research/evidence/agent-memory-chunk-granularity-recall-2026/environment.json">environment.json</a> — model names, digests, host and scoring settings</li>
<li><a href="https://raw.githubusercontent.com/baeseokjae/baeseokjae.github.io/main/research/evidence/agent-memory-chunk-granularity-recall-2026/METHOD.md">METHOD.md</a> — method, commands and known deviations</li>
</ul>
<p>Note that the fixture text ships with the probe and is entirely invented; nothing in it comes from a customer, a vendor, or a public benchmark.</p>
<p>The procedure to apply to your own store:</p>
<ol>
<li><strong>Write your real question shapes down first</strong>, separated into single-fact and multi-fact. If you have no multi-fact questions in that list, this entire article is not your problem yet — but your users will find some.</li>
<li><strong>Build a labelled set of 30–100 questions</strong> over your own documents, each with the sentence or span that answers it. Thirty is enough to see a collapse from 13/15 to 0/15; it is not enough to resolve a two-query difference.</li>
<li><strong>Run candidate chunk sizes</strong> with the embedder you actually deploy, at your real k. Report hit@1 <em>per question shape</em>, never pooled across shapes — pooling is exactly how a two-fact regression hides inside a flat single-fact score.</li>
<li><strong>Read the two numbers together.</strong> If multi-fact hit@1 collapses but deep-rank coverage holds, fix assembly: raise k and merge by parent document. If coverage also collapses at your k, the chunker is too fine for that query shape and the boundary has to move.</li>
<li><strong>Report the context cost of whatever you choose.</strong> Doubling k to recover assembly costs prompt characters on every turn; measure it in characters or tokens, not in vibes.</li>
</ol>
<h2 id="limitations">Limitations</h2>
<ul>
<li>Eight documents, thirteen questions, three embedders: a fixture sized to expose a mechanism in under a minute, not a benchmark. No production recall, answer-quality, latency or satisfaction claim follows from it.</li>
<li>The five two-fact pairs were placed to straddle a window boundary by construction, so 0/15 at fine granularity is a designed worst case.</li>
<li>Retrieval only: no generation, no re-ranker, no hybrid BM25, no metadata filters.</li>
<li>Character counts are not tokens, and the 6.9x ratio must not be converted to token cost without measuring a tokenizer.</li>
<li>nomic-embed-text ran without its required task prefixes, so its absolute numbers are not a fair model comparison.</li>
<li>Single-fact differences between granularities are one-query swings on an eight-question set and are not statistically meaningful.</li>
<li>One host (2 vCPU, 11 GiB, Ollama 0.20.3), localhost only; elapsed times are load-in dominated and are not throughput figures.</li>
<li>Embeddings are reproducible only for an identical model digest. Any rerun with a different digest is a different experiment, and results2.json records the digests it ran against.</li>
</ul>
<p>The result should be falsifiable in public: rerun the fixture, and if 3-sentence windows do not beat whole-document chunks on single-fact hit@1 for at least two of three embedders — or if two-fact hit@1 does not fall when moving from whole documents to sentence chunks — then this fixture is not reproducing and you should trust your own numbers over this table.</p>
<p>If you are still choosing where memory lives at all, <a href="https://baeseokjae.github.io/posts/agent-memory-architecture-guide-2026/">the framework comparison</a> covers the vendor side. This article assumes you already have a store and asks the smaller question that decides whether it works: how big is a vector&rsquo;s worth of truth.</p>
]]></content:encoded></item><item><title>Swap the Embedding Model, Keep the Store: Agent Memory That Returns the Wrong Neighbour Without an Error</title><link>https://baeseokjae.github.io/posts/agent-memory-embedding-swap-recall-2026/</link><pubDate>Mon, 02 Nov 2026 15:40:00 +0000</pubDate><guid>https://baeseokjae.github.io/posts/agent-memory-embedding-swap-recall-2026/</guid><description>A 40-document recall harness shows what happens when the encoder behind an existing vector store changes: same-width swaps return wrong neighbours with no exception, while width mismatches fail loudly. Includes the guard that records encoder id, digest and dimension per vector.</description><content:encoded><![CDATA[<p>An agent memory store can survive an embedding-model change while quietly returning the wrong fact. Nothing in the stack raises, because nothing in the stack compares the encoder that wrote a vector with the encoder that now reads it. In a 40-document fixture with 15 single-answer queries, one encoder on both sides ranked the labelled document first for 15 of 15 queries. Replacing the document encoder with a different model of the same width dropped that to 1 of 15, and no call failed. Replacing it with a model of a different width failed all 15 queries with a length error — the loud case, and the easy one.</p>
<p>That contrast is the whole point. The dimension check most teams already have catches one of these two failures. The other one changes what your agent believes.</p>
<h2 id="the-failure-neighbours-come-back-the-space-does-not">The failure: neighbours come back, the space does not</h2>
<p>The scenario is ordinary maintenance. A team moves memory embeddings from a hosted model to a local one, upgrades a model version, or switches providers during an outage. The write path keeps running, the table keeps its shape, and <code>search(query, k=5)</code> keeps returning five rows. If the new encoder has the same output width as the old one, even a strict database column will accept the new vectors: a <code>vector(384)</code> column takes 384 floats from any model on earth.</p>
<p>What changed is not the schema. It is the coordinate system. Cosine similarity only means something when both vectors were produced by the same function; compare two different functions and the ranking is noise with a plausible-looking shape. The agent does not report &ldquo;no match&rdquo; — it reports the nearest row in a meaningless space, phrased fluently. That is worse than an outage, because an outage is visible.</p>
<p>This is a different failure from the one where long inputs are clipped before they become vectors. <a href="https://baeseokjae.github.io/posts/local-embedding-context-ceiling-2026/">That truncation problem is measured here</a>. The swap problem happens after a correct write: the vector is complete, the row is intact, and only its provenance is wrong.</p>
<h2 id="what-actually-changes-when-the-encoder-changes">What actually changes when the encoder changes</h2>
<p>Three properties decide whether two encoders share a space, and a dimension check tests only the third.</p>
<p>The training objective and data decide the geometry. <a href="https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2">all-MiniLM-L6-v2</a> is documented as mapping sentences to a 384-dimensional dense space for semantic search; a different 384-dimensional model is trained on a different objective and arranges the same words differently.</p>
<p>The input convention decides the point, even for one model. <a href="https://huggingface.co/nomic-ai/nomic-embed-text-v1.5">nomic-embed-text-v1.5&rsquo;s card</a> requires a task instruction prefix — <code>search_document: </code> for stored text and <code>search_query: </code> for queries — and says the prefix is mandatory. Feeding that model unprefixed text produces a vector that is valid, 768-dimensional, and not the vector the model was trained to produce for retrieval.</p>
<p>The width decides only whether the error is loud. Ollama&rsquo;s <a href="https://github.com/ollama/ollama/blob/main/docs/api.md">embed API reference</a> documents a <code>dimensions</code> parameter that sets the number of dimensions for the embedding, so width is not necessarily fixed per model: a client can request a shorter vector from the same weights, and a store keyed only on <code>(model, width)</code> would accept that output as if it belonged to the full-width space.</p>
<p>Input length is a separate lever in the same API and does not touch width. The reference&rsquo;s <code>truncate</code> option defaults to <code>true</code> and clips the end of each <em>input</em> to fit the model&rsquo;s context length — the vector still comes out at full width. That is the write-time information loss measured in <a href="https://baeseokjae.github.io/posts/local-embedding-context-ceiling-2026/">the embedding input-length article</a>, and it can occur with no model change at all. Keep the two apart: clipping changes what the model saw, a swap changes which function produced the vector.</p>
<h2 id="the-harness-40-documents-15-queries-three-encoders">The harness: 40 documents, 15 queries, three encoders</h2>
<p>The measurement is deliberately small so it can be re-run on a laptop. The fixture is 40 synthetic agent-memory statements — deployment windows, retention periods, ports, contact preferences, identifiers — and 15 questions, each with exactly one defensible gold document. No real user data is involved. The baseline result is also the fixture&rsquo;s label check: if a competent encoder cannot put the gold document first for every query, the labels are ambiguous and the fixture is not usable.</p>
<p>Documents and queries are embedded through the local Ollama endpoint, stored as plain Python lists, and scored by brute-force cosine over every query-document pair. There is no ANN index, so no approximation error can be confused with the effect being measured. Encoders are pinned by digest rather than tag, because <code>:latest</code> can be repointed: nomic-embed-text <code>0a109f422b47</code> at 768 dimensions, all-minilm <code>1b226e2802db</code> at 384, and snowflake-arctic-embed:33m <code>e8db018629b4</code> at 384, read from the host&rsquo;s tag list on 2026-11-02 with Ollama 0.20.3. nomic was called with the prefixes its card requires; the two 384-dimension models were called without any task prefix, so the swap rows do not separate a prefix effect from a space effect.</p>
<p>Five configurations were scored: a matched baseline, both directions of a same-width cross-model swap, a half-migrated store, and a cross-width query. Each produced one record per query plus an aggregate.</p>
<h2 id="the-result-matrix">The result matrix</h2>
<table>
	<thead>
			<tr>
					<th>Configuration</th>
					<th>Document encoder</th>
					<th>Query encoder</th>
					<th>Width</th>
					<th>recall@1</th>
					<th>recall@5</th>
			</tr>
	</thead>
	<tbody>
			<tr>
					<td>Baseline</td>
					<td>nomic-embed-text</td>
					<td>nomic-embed-text</td>
					<td>768/768</td>
					<td>15/15</td>
					<td>15/15</td>
			</tr>
			<tr>
					<td>Same-width swap A</td>
					<td>all-minilm</td>
					<td>snowflake-arctic-embed:33m</td>
					<td>384/384</td>
					<td>1/15</td>
					<td>2/15</td>
			</tr>
			<tr>
					<td>Same-width swap B</td>
					<td>snowflake-arctic-embed:33m</td>
					<td>all-minilm</td>
					<td>384/384</td>
					<td>1/15</td>
					<td>3/15</td>
			</tr>
			<tr>
					<td>Half-migrated store</td>
					<td>nomic + all-minilm (20/20)</td>
					<td>nomic-embed-text</td>
					<td>768+384 / 768</td>
					<td>7/15</td>
					<td>7/15</td>
			</tr>
			<tr>
					<td>Cross-width</td>
					<td>nomic-embed-text</td>
					<td>all-minilm</td>
					<td>768/384</td>
					<td>—</td>
					<td>—</td>
			</tr>
	</tbody>
</table>
<p>The two same-width rows are the alarming ones. Recall at rank one fell from 15 of 15 to 1 of 15 in both directions, and the best of the two still missed 12 of 15 gold documents inside the top five. No embedding call raised, every vector came back at the recorded width, and this harness kept its vectors in a plain Python list, so no column type was consulted at all — but the 384-dimension vectors would have satisfied a <code>vector(384)</code> column just as readily, because their width was correct; no width constraint anywhere in this stack would have rejected them. If the only monitoring you have is &ldquo;the query returned k rows&rdquo;, this failure is invisible.</p>
<p>The mixed store is the migration-in-progress case. Twenty documents were embedded by all-minilm and twenty by nomic, alternating; queries used nomic, so half the table belonged to the query&rsquo;s space and half did not. Recall@1 came out at 7 of 15 — better than the full swap, because the matching half was reachable, and still less than half of the baseline. The harness skipped 300 query-document comparisons (20 mismatched documents across 15 queries) rather than pretending to score them. In a real store the same state is worse than a ranking problem: part of the table cannot be served at all, and nothing in the application knows which part.</p>
<h2 id="the-loud-failure-is-the-safe-one">The loud failure is the safe one</h2>
<p>Querying the 768-dimension store with the 384-dimension encoder raised <code>ValueError: vector length mismatch: 384 vs 768</code> for all 15 queries. That is the outcome to want. A width mismatch stops the request at the first comparison, which is why a schema like <a href="https://github.com/pgvector/pgvector">pgvector&rsquo;s <code>vector(n)</code> column</a> is worth having: it fixes the width at write time, so a differently sized vector is rejected on insert instead of discovered on query.</p>
<p>Two caveats keep this from being a complete defence. First, the exception text above belongs to this harness&rsquo;s pure-Python cosine; the failure is client-dependent. A client that pads, slices or truncates to the stored width — or a provider that honours a <code>dimensions</code> parameter — will return ranked neighbours with no error at all. Second, the width is the only property a database column checks. Same-width models pass every constraint in the stack, which is exactly the configuration that measured 1 of 15.</p>
<p>One hypothesis was tested and dropped rather than published as a maybe: whether Ollama&rsquo;s native <code>/api/embed</code> and its OpenAI-compatible <code>/v1/embeddings</code> return different spaces for the same input. For one model and one input string, the vectors were identical to six decimal places (cosine 1.0, 384 dimensions on both paths). There is no endpoint-divergence warning to give here.</p>
<h2 id="the-guard-one-identifier-per-vector-checked-before-scoring">The guard: one identifier per vector, checked before scoring</h2>
<p>The fix is not a better model. It is provenance: every vector records which encoder produced it, and every query refuses to run against a store written by a different one. A model tag alone is insufficient, because tags move; a digest alone is insufficient, because width can change under one model through <code>dimensions</code>. Concatenate them.</p>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-text-size-adjust:none;"><code class="language-sql" data-lang="sql"><span style="display:flex;"><span><span style="color:#66d9ef">CREATE</span> <span style="color:#66d9ef">TABLE</span> memory_vectors (
</span></span><span style="display:flex;"><span>  doc_id      TEXT <span style="color:#66d9ef">NOT</span> <span style="color:#66d9ef">NULL</span>,
</span></span><span style="display:flex;"><span>  encoder_id  TEXT <span style="color:#66d9ef">NOT</span> <span style="color:#66d9ef">NULL</span>,          <span style="color:#75715e">-- e.g. nomic-embed-text:latest@0a109f42:768
</span></span></span><span style="display:flex;"><span>  dim         INTEGER <span style="color:#66d9ef">NOT</span> <span style="color:#66d9ef">NULL</span>,
</span></span><span style="display:flex;"><span>  content_sha TEXT <span style="color:#66d9ef">NOT</span> <span style="color:#66d9ef">NULL</span>,
</span></span><span style="display:flex;"><span>  vector      BLOB <span style="color:#66d9ef">NOT</span> <span style="color:#66d9ef">NULL</span>,          <span style="color:#75715e">-- packed float32
</span></span></span><span style="display:flex;"><span>  <span style="color:#66d9ef">PRIMARY</span> <span style="color:#66d9ef">KEY</span> (doc_id, encoder_id)
</span></span><span style="display:flex;"><span>);
</span></span></code></pre></div><p>The <code>content_sha</code> column answers a second question the recall harness cannot: whether the stored vector still corresponds to the text it is supposed to represent. If the document changed and the vector did not, the row is stale even though its encoder id matches.</p>
<p>The read path then becomes a refusal rather than a hope:</p>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-text-size-adjust:none;"><code class="language-python" data-lang="python"><span style="display:flex;"><span><span style="color:#66d9ef">def</span> <span style="color:#a6e22e">require_same_space</span>(store_encoder_id, store_dim, query_encoder_id, query_dim):
</span></span><span style="display:flex;"><span>    problems <span style="color:#f92672">=</span> []
</span></span><span style="display:flex;"><span>    <span style="color:#66d9ef">if</span> query_encoder_id <span style="color:#f92672">!=</span> store_encoder_id:
</span></span><span style="display:flex;"><span>        problems<span style="color:#f92672">.</span>append(<span style="color:#e6db74">f</span><span style="color:#e6db74">&#34;encoder mismatch: store=</span><span style="color:#e6db74">{</span>store_encoder_id<span style="color:#e6db74">}</span><span style="color:#e6db74"> query=</span><span style="color:#e6db74">{</span>query_encoder_id<span style="color:#e6db74">}</span><span style="color:#e6db74">&#34;</span>)
</span></span><span style="display:flex;"><span>    <span style="color:#66d9ef">if</span> query_dim <span style="color:#f92672">!=</span> store_dim:
</span></span><span style="display:flex;"><span>        problems<span style="color:#f92672">.</span>append(<span style="color:#e6db74">f</span><span style="color:#e6db74">&#34;dimension mismatch: store=</span><span style="color:#e6db74">{</span>store_dim<span style="color:#e6db74">}</span><span style="color:#e6db74"> query=</span><span style="color:#e6db74">{</span>query_dim<span style="color:#e6db74">}</span><span style="color:#e6db74">&#34;</span>)
</span></span><span style="display:flex;"><span>    <span style="color:#66d9ef">if</span> problems:
</span></span><span style="display:flex;"><span>        <span style="color:#66d9ef">raise</span> <span style="color:#a6e22e">RuntimeError</span>(<span style="color:#e6db74">&#34;refusing cross-space search: &#34;</span> <span style="color:#f92672">+</span> <span style="color:#e6db74">&#34;; &#34;</span><span style="color:#f92672">.</span>join(problems))
</span></span></code></pre></div><p>During a migration, &ldquo;refuse&rdquo; would take the feature down, so the guard also needs a read-only mode that reports exposure instead of blocking: count rows by <code>encoder_id</code>, return the fraction the target encoder can actually serve, and fail the migration gate until that fraction is 1.0 with exactly one encoder id present.</p>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-text-size-adjust:none;"><code class="language-python" data-lang="python"><span style="display:flex;"><span><span style="color:#66d9ef">def</span> <span style="color:#a6e22e">reindex_state</span>(rows, target_dim):
</span></span><span style="display:flex;"><span>    total <span style="color:#f92672">=</span> len(rows)
</span></span><span style="display:flex;"><span>    live <span style="color:#f92672">=</span> sum(<span style="color:#ae81ff">1</span> <span style="color:#66d9ef">for</span> r <span style="color:#f92672">in</span> rows <span style="color:#66d9ef">if</span> r[<span style="color:#e6db74">&#34;dim&#34;</span>] <span style="color:#f92672">==</span> target_dim)
</span></span><span style="display:flex;"><span>    <span style="color:#66d9ef">return</span> {<span style="color:#e6db74">&#34;total&#34;</span>: total, <span style="color:#e6db74">&#34;servable&#34;</span>: live,
</span></span><span style="display:flex;"><span>            <span style="color:#e6db74">&#34;silently_unreachable&#34;</span>: total <span style="color:#f92672">-</span> live,
</span></span><span style="display:flex;"><span>            <span style="color:#e6db74">&#34;ready&#34;</span>: total <span style="color:#f92672">==</span> live <span style="color:#f92672">and</span> len({r[<span style="color:#e6db74">&#34;encoder_id&#34;</span>] <span style="color:#66d9ef">for</span> r <span style="color:#f92672">in</span> rows}) <span style="color:#f92672">==</span> <span style="color:#ae81ff">1</span>}
</span></span></code></pre></div><p>That function is the difference between the mixed-store row and a migration you can reason about. It converts &ldquo;we re-embedded most of the table&rdquo; into a number.</p>
<h2 id="migration-checklist">Migration checklist</h2>
<ol>
<li>Build a shadow index under a new <code>encoder_id</code>. Write both encoders&rsquo; vectors for new documents (&ldquo;dual write&rdquo;) so the old path keeps serving while the new one fills.</li>
<li>Run a labelled recall check against both indexes before cutover, on the same queries. Compare recall@5, not just &ldquo;does search return rows&rdquo;. A drop from 15/15 to 1/15 is what a cross-space query looks like when nobody is looking for it.</li>
<li>Backfill by <code>content_sha</code>, not by <code>updated_at</code>: a row whose text did not change still needs a new vector, and a row whose text changed twice needs one vector, not three.</li>
<li>Cut over by changing the stored <code>encoder_id</code> the reader expects — the guard will refuse mixed states — then delete the old rows only after the new index serves the full table.</li>
<li>Keep the labelled query set. It is the only regression test that fails when a future model swap reintroduces the silent case: the width check in this harness raised on every cross-width pair — the whole of the last configuration and the mismatched half of the mixed store — while every same-width comparison returned a full ranking.</li>
</ol>
<p>For stores that expose a fixed vector width, an insert at the wrong width fails at the column. That covers only part of step 3: a same-width vector from another model is accepted by every schema constraint, so the rest is enforced only by the <code>encoder_id</code> check, which is why the check belongs in the read path and not in a deployment checklist.</p>
<h2 id="reproduce-it-in-a-few-minutes">Reproduce it in a few minutes</h2>
<p>The measured run used the 40-document fixture described above; the scoring loop is small enough to paste, and swapping in your own documents and queries is the intended use. The core is a brute-force cosine over two lists of vectors:</p>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-text-size-adjust:none;"><code class="language-python" data-lang="python"><span style="display:flex;"><span><span style="color:#f92672">from</span> math <span style="color:#f92672">import</span> sqrt
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#66d9ef">def</span> <span style="color:#a6e22e">cosine</span>(a, b):
</span></span><span style="display:flex;"><span>    <span style="color:#66d9ef">if</span> len(a) <span style="color:#f92672">!=</span> len(b):
</span></span><span style="display:flex;"><span>        <span style="color:#66d9ef">raise</span> <span style="color:#a6e22e">ValueError</span>(<span style="color:#e6db74">f</span><span style="color:#e6db74">&#34;vector length mismatch: </span><span style="color:#e6db74">{</span>len(a)<span style="color:#e6db74">}</span><span style="color:#e6db74"> vs </span><span style="color:#e6db74">{</span>len(b)<span style="color:#e6db74">}</span><span style="color:#e6db74">&#34;</span>)
</span></span><span style="display:flex;"><span>    dot <span style="color:#f92672">=</span> sum(x <span style="color:#f92672">*</span> y <span style="color:#66d9ef">for</span> x, y <span style="color:#f92672">in</span> zip(a, b))
</span></span><span style="display:flex;"><span>    na, nb <span style="color:#f92672">=</span> sqrt(sum(x <span style="color:#f92672">*</span> x <span style="color:#66d9ef">for</span> x <span style="color:#f92672">in</span> a)), sqrt(sum(y <span style="color:#f92672">*</span> y <span style="color:#66d9ef">for</span> y <span style="color:#f92672">in</span> b))
</span></span><span style="display:flex;"><span>    <span style="color:#66d9ef">return</span> dot <span style="color:#f92672">/</span> (na <span style="color:#f92672">*</span> nb) <span style="color:#66d9ef">if</span> na <span style="color:#f92672">and</span> nb <span style="color:#66d9ef">else</span> <span style="color:#ae81ff">0.0</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#66d9ef">def</span> <span style="color:#a6e22e">recall_at_k</span>(docs, queries, k):
</span></span><span style="display:flex;"><span>    hits <span style="color:#f92672">=</span> <span style="color:#ae81ff">0</span>
</span></span><span style="display:flex;"><span>    <span style="color:#66d9ef">for</span> q <span style="color:#f92672">in</span> queries:
</span></span><span style="display:flex;"><span>        ranked <span style="color:#f92672">=</span> sorted(docs, key<span style="color:#f92672">=</span><span style="color:#66d9ef">lambda</span> d: cosine(q[<span style="color:#e6db74">&#34;vector&#34;</span>], d[<span style="color:#e6db74">&#34;vector&#34;</span>]), reverse<span style="color:#f92672">=</span><span style="color:#66d9ef">True</span>)
</span></span><span style="display:flex;"><span>        hits <span style="color:#f92672">+=</span> q[<span style="color:#e6db74">&#34;gold&#34;</span>] <span style="color:#f92672">in</span> [d[<span style="color:#e6db74">&#34;id&#34;</span>] <span style="color:#66d9ef">for</span> d <span style="color:#f92672">in</span> ranked[:k]]
</span></span><span style="display:flex;"><span>    <span style="color:#66d9ef">return</span> hits <span style="color:#f92672">/</span> len(queries)
</span></span></code></pre></div><p>Embed <code>docs</code> and <code>queries</code> with the same model first, confirm recall@1 is at or near 1.0, and only then re-embed the documents with the candidate model and re-run against the original query vectors. The difference between those two numbers is the cost of the swap on your corpus.</p>
<p>The evidence behind the numbers in this article — fixture, harness, per-configuration JSONL and the digest ledger — is a private author-side working record kept with the run; it is not offered as a download from this page. What is reproducible without it is the procedure above and the two guard functions, which are complete as written.</p>
<h2 id="limitations">Limitations</h2>
<p>The fixture has 40 documents and 15 single-answer queries, and the baseline is saturated at 15 of 15. The measured collapse therefore shows that cross-space ranking is near-random on this fixture; it does not estimate the size of the drop on a larger, harder corpus, where a matched encoder would also miss some queries. All numbers come from a single run on one 2-vCPU host with Ollama 0.20.3, two 384-dimension open models and one 768-dimension model. No proprietary embedder, reranker, hybrid BM25 leg or ANN index was involved, so nothing here measures approximation error, latency or the cost of re-embedding a production table.</p>
<p>The mixed-store result depends on a lenient client: this harness skips width-mismatched pairs, while pgvector would reject them at the column type and an application-level store might accept them silently. The 768-vs-384 exception text is Python-specific for the same reason. The 384-dimension pair was embedded without task prefixes, so part of that drop may belong to input convention rather than to a different vector space; the two causes are not separated here, which is another reason to treat the number as a demonstration of the failure mode rather than a measurement of any model. And no search-demand data supports the topic: the memory cluster has measured impressions, but no embedding-migration query appears in the samples behind this page, so treat it as an unmeasured failure mode rather than a searched-for one.</p>
<p>If you are still choosing a store rather than migrating one, <a href="https://baeseokjae.github.io/posts/mem0-vs-zep-production-guide-2026/">the store comparison</a> and <a href="https://baeseokjae.github.io/posts/zep-ai-agent-memory-review-2026/">the Zep review</a> cover that decision. If you are designing the write path this guard sits on, <a href="https://baeseokjae.github.io/posts/agent-memory-write-path-reliability-2026/">the write-path reliability protocol</a> is the companion piece, and <a href="https://baeseokjae.github.io/posts/agent-memory-architecture-guide-2026/">the architecture guide</a> puts the vector layer in context.</p>
]]></content:encoded></item></channel></rss>