<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:content="http://purl.org/rss/1.0/modules/content/"><channel><title>Model Sizing on RockB</title><link>https://baeseokjae.github.io/tags/model-sizing/</link><description>Recent content in Model Sizing on RockB</description><image><title>RockB</title><url>https://baeseokjae.github.io/images/og-default.png</url><link>https://baeseokjae.github.io/images/og-default.png</link></image><generator>Hugo</generator><language>en-us</language><lastBuildDate>Mon, 09 Nov 2026 07:40:19 +0000</lastBuildDate><atom:link href="https://baeseokjae.github.io/tags/model-sizing/index.xml" rel="self" type="application/rss+xml"/><item><title>What You Actually Pulled: Reading Quantisation, Parameter Count and Context Out of an Ollama GGUF File</title><link>https://baeseokjae.github.io/posts/ollama-gguf-artifact-forensics-2026/</link><pubDate>Fri, 06 Nov 2026 00:00:00 +0000</pubDate><guid>https://baeseokjae.github.io/posts/ollama-gguf-artifact-forensics-2026/</guid><description>A 31B tag pulls 18.99 GiB of layers — an 18,670,033,120-byte model blob against 61.4 GB for the same weights in bf16 — and nothing ollama prints says which quantisation is inside. An 8–32 MiB ranged read of the GGUF header returns the tensor table, so bits per weight, parameter count and the declared context field become numbers you can check before the download finishes.</description><content:encoded><![CDATA[<p>You pulled a tag. The download is 18.99 GiB for a model the card calls 31B. The agent that needed long history behaves as if it had a fraction of the context the card advertises. The quality does not match the benchmark chart that made you choose the model in the first place. None of those three surprises is a property of the model. All three are properties of the artifact: which quantisation the tag actually resolved to, how many bits per weight that quantisation really costs, and which metadata value declares the context this particular build was made for.</p>
<p>This article answers those three questions from the bytes rather than from the tag name. Every figure below comes from a ranged HTTP read of the file header — 8 to 32 MiB out of a 5–19 GB file — so no weights were downloaded, no server was started, and no throughput, quality or VRAM claim is made or implied. Five artifacts were read on 2026-11-06: three published GGUF files and two Ollama registry blobs.</p>
<h2 id="the-file-on-disk-does-not-match-the-card">The file on disk does not match the card</h2>
<p>The unsloth repository <code>unsloth/gemma-4-31B-it-GGUF</code> lists its bf16 build as two shards of 49,950,156,960 and 11,463,033,568 bytes, which is 61,413,190,528 bytes or 57.2 GiB (<a href="https://huggingface.co/api/models/unsloth/gemma-4-31B-it-GGUF?blobs=true">repository index</a>). The same index lists <code>gemma-4-31B-it-Q4_K_M.gguf</code> at 18,323,733,440 bytes. That is 17.07 GiB, 18.32 decimal GB, and 3.35× smaller than the bf16 build.</p>
<p>So &ldquo;18 GB, not 48 GB&rdquo; contains two separate confusions. The 18 figure is real: it is the 4-bit file, and it looks smaller in GiB than in decimal GB because binary units divide by 1,073,741,824. The 48 figure corresponds to nothing in that repository — the bf16 weights there are 61.41 decimal GB, and a Q8_0 build of the same weights lands at 8.5 bits per weight rather than 16. Once the quantisation is known, both sizes are arithmetic.</p>
<p>The catch is that <code>31B</code> is a name, not a specification. The name tells you the parameter class; it says nothing about the 4.77 bits per weight the file actually spends. <code>ollama show</code> prints a Modelfile back at you, and that Modelfile is a description of how to load the blob, not of what is inside it. The manifest audit in <a href="https://baeseokjae.github.io/posts/ollama-tag-quant-bits-per-weight-audit-2026/">the tag-size article</a> takes the first step: it divides a manifest&rsquo;s model-layer bytes by the parameter total published by the model repository, and states plainly that it did not inspect tensors. This piece takes the second step and reads the tensor table, so numerator and denominator come from the same file.</p>
<h2 id="gguf-is-a-directory-inside-a-file">GGUF is a directory inside a file</h2>
<p>A GGUF file begins with the magic <code>GGUF</code>, a version, a tensor count and a metadata key/value count, then a metadata block of typed key/value pairs, then one tensor-info record per tensor holding its name, dimensions, quantisation type and byte offset. The weight data starts after that array, padded to the alignment boundary, and each tensor offset is relative to the start of the data section rather than to the start of the file (<a href="https://github.com/ggml-org/ggml/blob/master/docs/gguf.md">GGUF specification</a>).</p>
<p>That layout is what makes a header read sufficient. The tensor-info array carries the type of every tensor and the size of every dimension, so the full weight map — how many bytes at which precision, how many elements in total — is available before a single weight is fetched. The weights themselves are only needed to run the model, not to describe it.</p>
<h2 id="reading-it-without-downloading-it">Reading it without downloading it</h2>
<p>The probe sends two requests per artifact. A <code>HEAD</code> gives <code>Content-Length</code>, the exact size of the file. A <code>GET</code> with <code>Range: bytes=0-33554431</code> returns the first 32 MiB, which is enough for the metadata block, the tokenizer arrays and the tensor-info array of a 262,144-token vocabulary.</p>
<p>The window has to clear the header, and how much that takes is a property of the file, not a constant. A 1 MiB window fails on the Gemma file before the tensor table is complete (<code>EOFError: window ended at 1048576; need 13 more</code>), and 8 MiB still fails there (<code>window ended at 8388608; need 8 more</code>) because the 262,144-token vocabulary pushes the weight section out to byte 15,834,176. The same 8 MiB window was already enough for both Qwen3 files, whose weight sections start at 5,956,416 and 5,956,384 bytes. In the saved batch three artifacts parsed at 32 MiB — the Gemma file and both registry blobs — and two parsed at 8 MiB (Qwen3-8B-Q4_K_M at 14:34:01Z, Qwen3-8B-Q8_0 at 14:34:04Z); a later 32 MiB rerun of Qwen3-8B-Q4_K_M at 14:41:15Z reproduced the same values. The parser raises on a short window instead of returning the tensors it managed to read, which matters: a silently truncated tensor list still produces a plausible-looking bits-per-weight number that is simply wrong.</p>
<p>The five successful reads ran between 14:34:01Z and 14:34:33Z on 2026-11-06, each returning HTTP 206 and taking 1.41–4.03 s of wall clock. The two registry blobs answered fastest (1.41 s and 1.62 s) and the Gemma file slowest (4.03 s), a spread that tracks the origin rather than the 8- or 32-MiB span. The SHA-256 of each fetched span was recorded so a later rerun can prove it read the same bytes.</p>
<h2 id="worked-example-gemma-4-31b-it-q4_k_m">Worked example: gemma-4-31B-it Q4_K_M</h2>
<p>The file&rsquo;s header declares architecture <code>gemma4</code>, 60 blocks, 262,144 tokens of context, and 833 tensors. Adding up the dimensions gives 30,697,345,596 stored elements — the parameter count of this file, not of the family. Multiplying each tensor&rsquo;s element count by its type&rsquo;s block width gives 18,307,897,584 bytes of weights.</p>
<table>
	<thead>
			<tr>
					<th>Quantisation</th>
					<th>Tensors</th>
					<th>Bytes</th>
					<th>Share of weight bytes</th>
			</tr>
	</thead>
	<tbody>
			<tr>
					<td>Q4_K (4.5 bpw nominal)</td>
					<td>356</td>
					<td>15,006,007,296</td>
					<td>81.96%</td>
			</tr>
			<tr>
					<td>Q6_K (6.5625 bpw nominal)</td>
					<td>55</td>
					<td>3,296,563,200</td>
					<td>18.01%</td>
			</tr>
			<tr>
					<td>F32 (32 bpw)</td>
					<td>422</td>
					<td>5,327,088</td>
					<td>0.03%</td>
			</tr>
	</tbody>
</table>
<p>8 × 18,307,897,584 ÷ 30,697,345,596 = 4.7712 bits per weight. The 422 F32 tensors are almost all small norms and biases: they are 422 of the 833 tensor records but 0.03% of the bytes.</p>
<p>The reconciliation is what makes that ratio trustworthy rather than decorative. The weight section begins at byte 15,834,176, so the file&rsquo;s published size leaves 18,307,899,264 bytes for weights. The tensor table accounts for 18,307,897,584 of them — a residue of 1,680 bytes, 0.00001%, which is the alignment padding after the final tensor. A wrong block table, a missed tensor or a misread dimension cannot land on that total by accident.</p>
<h2 id="why-4-bit-is-477-and-8-bit-is-850">Why &ldquo;4-bit&rdquo; is 4.77 and &ldquo;8-bit&rdquo; is 8.50</h2>
<p>Block geometry explains the gap between the label and the measurement. In llama.cpp&rsquo;s quantisation table a K-quant block covers 256 elements: Q4_K spends 144 bytes per block (4.5 bpw), Q6_K spends 210 (6.5625 bpw), and Q8_0 — a 32-element block — spends 34 (8.5 bpw) (<a href="https://raw.githubusercontent.com/ggml-org/llama.cpp/master/gguf-py/gguf/constants.py">constants.py</a>).</p>
<p>A <code>Q4_K_M</code> file is a mixture. The <code>_M</code> variant keeps selected tensors at a higher precision, so on the Gemma file 18.01% of weight bytes are Q6_K and the blended figure is 4.7712 rather than 4.5. The Qwen3-8B Q4_K_M file shows the same mechanism at a different ratio: 3,704,979,456 bytes of Q4_K, 1,315,614,720 of Q6_K and 1,232,896 of F32 over 8,190,735,360 elements, giving 4.9049 bpw with 26.20% of weight bytes at Q6_K. The Q8_0 build of the same 8.19B parameters spends 8,702,328,832 bytes at Q8_0 plus the same 1,232,896 bytes of F32 and measures 8.5009.</p>
<p>The practical consequence: &ldquo;4-bit&rdquo; and &ldquo;8-bit&rdquo; are labels for a mixture, and the blended figure moves with how much of the file the quantizer kept at higher precision. If the number matters — and it does when you are deciding whether a file fits in VRAM or on a laptop disk — it has to be summed from the tensors.</p>
<h2 id="the-tag-is-not-the-file">The tag is not the file</h2>
<p>An Ollama tag resolves to a registry manifest, and that manifest names layers with exact byte sizes and digests. Reading the model layer of <code>gemma4:31b</code> and parsing it the same way gives a different file from the same-named Hugging Face build.</p>
<table>
	<thead>
			<tr>
					<th>Artifact</th>
					<th>Bytes</th>
					<th>Q4_K bytes</th>
					<th>Q6_K bytes</th>
					<th>Parameters</th>
					<th>bpw</th>
			</tr>
	</thead>
	<tbody>
			<tr>
					<td>HF <code>gemma-4-31B-it-Q4_K_M.gguf</code></td>
					<td>18,323,733,440</td>
					<td>15,006,007,296</td>
					<td>3,296,563,200</td>
					<td>30,697,345,596</td>
					<td>4.7712</td>
			</tr>
			<tr>
					<td>Ollama <code>gemma4:31b</code> model layer</td>
					<td>18,670,033,120</td>
					<td>14,250,442,752</td>
					<td>4,398,428,160</td>
					<td>30,697,345,596</td>
					<td>4.8614</td>
			</tr>
	</tbody>
</table>
<p>Same architecture, same 60 blocks, same 262,144-token context field, same parameter count — 346,299,680 more bytes, and a different mixture: 23.58% of the tag&rsquo;s weight bytes are Q6_K against 18.01% in the Hugging Face file. The manifest&rsquo;s own <code>from</code> field names the upstream publisher tag <code>dhiltgen/gemma4:31b-it-mtp-q4_K_M-llamacpp</code>, not a Hugging Face repository, so size and mixture similarity cannot establish which repository this blob was converted from. Treat provenance as unknown and check the bytes you have.</p>
<p>The same divergence appears at 8B. The Ollama <code>qwen3:8b</code> model layer is 5,225,374,496 bytes against 5,027,783,488 for <code>Qwen3-8B-Q4_K_M.gguf</code>, and it carries 301,989,888 bytes of F16 tensors the Hugging Face Q4_K_M does not, with 1,253,683,200 bytes of Q6_K against 1,315,614,720. Bits per weight: 5.0979 versus 4.9049, over an identical 8,190,735,360 parameters.</p>
<p>A tag can also be more than one model. <code>gemma4:31b</code> is projector 1,200,726,368 + model 18,670,033,120 + draft 514,687,072 bytes — 20,385,446,560 weight-bearing bytes, and 20,385,457,992 bytes across every layer including template, license and params. That is 18.99 GiB of download for a file advertised as 31B at 4-bit. The projector is a separate artifact with its own cost profile, covered in <a href="https://baeseokjae.github.io/posts/local-vlm-tag-projector-image-token-cost-2026/">the projector article</a>; it is not part of a weights-only bits-per-weight ratio, and adding it to the denominator would make the number meaningless.</p>
<h2 id="which-context-value-wins">Which context value wins</h2>
<p>Three different quantities get called &ldquo;the context length&rdquo;, and they are not interchangeable.</p>
<p>The first is the field inside the artifact. Parsing the metadata block gives <code>gemma4.context_length = 262144</code> and <code>qwen3.context_length = 40960</code>. For the Qwen3-8B files that 40,960 is the positional-embedding allocation the vendor ships in <code>config.json</code> as <code>max_position_embeddings</code>; it is not a statement of the length the weights were trained on.</p>
<p>The second is the model card&rsquo;s own framing, which states the limits separately: &ldquo;Context Length: 32,768 natively and 131,072 tokens with YaRN&rdquo;. The card then explains where 40,960 comes from: &ldquo;The default max_position_embeddings in config.json is set to 40,960. This allocation includes reserving 32,768 tokens for outputs and 8,192 tokens for typical prompts, which is sufficient for most scenarios involving short text processing&rdquo; (<a href="https://huggingface.co/Qwen/Qwen3-8B">Qwen3-8B card</a>). Read as three numbers rather than one: 32,768 native, 131,072 with YaRN scaling, and 40,960 as a padded default that splits into an 8,192-token prompt budget and 32,768 reserved for generation. An agent carrying a 30k-token tool history is asking for more prompt than that arrangement allows, on a card whose headline number reads &ldquo;40,960&rdquo;.</p>
<p>The third is the runtime option, and it is the one that decides what is actually served. Ollama&rsquo;s Modelfile reference defines <code>PARAMETER num_ctx 4096</code> as setting &ldquo;the context window size to 4096, this controls how many tokens the LLM can use as context to generate the next token&rdquo; (<a href="https://docs.ollama.com/modelfile.md">Modelfile reference</a>), and a tag&rsquo;s <code>params</code> layer is where such a value would be baked in. Both tags&rsquo; params layers are readable: the registry blob URL answers HTTP 307 to a presigned object URL, and following it returns 64 bytes for <code>gemma4:31b</code> (<code>draft_num_predict</code>, <code>temperature</code>, <code>top_k</code>, <code>top_p</code>) and 120 bytes for <code>qwen3:8b</code> (<code>repeat_penalty</code>, <code>stop</code>, <code>temperature</code>, <code>top_k</code>, <code>top_p</code>). Neither contains <code>num_ctx</code>, so neither tag pins a window. What is established is the boundary: the artifact&rsquo;s declared length is not the served length, the tag&rsquo;s params do not set it, and the value that does comes from a server default or a per-request option. <a href="https://baeseokjae.github.io/posts/ollama-silent-context-truncation-2026/">The truncation diagnosis</a> covers what the server does when a conversation exceeds that cap.</p>
<h2 id="a-five-minute-checklist-before-trusting-a-tag">A five-minute checklist before trusting a tag</h2>
<ol>
<li>Run <code>ollama show --modelfile qwen3:8b</code> and read the <code>PARAMETER</code> lines and the architecture line. That is the loading recipe, not the file contents.</li>
<li>Take the file&rsquo;s exact size: <code>curl -sI &lt;resolve-url&gt; | grep -i content-length</code>, or read the model layer&rsquo;s <code>size</code> from the registry manifest at <code>https://registry.ollama.ai/v2/library/&lt;family&gt;/manifests/&lt;tag&gt;</code>.</li>
<li>Compare that size with the size the same-named file has in the model repository. A mismatch means the tag is a different requantisation, which is normal, not a bug — but it means benchmarks measured on the repository file do not automatically describe your file.</li>
<li>Fetch the first 32 MiB and parse the header with the script below. You get the quantisation mix, the parameter count, the declared context and the bits per weight in one pass.</li>
<li>Sanity-check the ratio against block geometry: a Q4_K_M file at 7B or larger typically blends to the high 4s, and anything inside 4.5–5.2 is a normal mixture; a Q8_0 file lands near 8.5. A figure outside that band means the parser or the fixture is wrong.</li>
<li>Read <code>context_length</code> from the same output and set <code>num_ctx</code> deliberately. Do not inherit a default and then debug the model&rsquo;s memory.</li>
</ol>
<h2 id="what-this-check-cannot-tell-you">What this check cannot tell you</h2>
<p>The measurement is a weights-only ratio: metadata, tokenizer arrays and alignment padding are excluded by construction, which is why the reconciliation compares against <code>Content-Length</code> minus the data offset rather than the whole file. It says nothing about output quality, tokens per second, VRAM in use or whether the mixture is a good one — all of those need a running server and a workload, and none was measured here. It covers five artifacts: three published GGUF files and two registry blobs, with no IQ-series, Q3, Q5 or MXFP4 file parsed. Ollama tags are mutable, so every size and digest here is true for 2026-11-06 and can change when a tag is re-pushed. And the ratio cannot prove where a blob came from; it can only prove what is inside the bytes you fetched.</p>
<h2 id="reproduce-it">Reproduce it</h2>
<p>Save this as <code>probe_range.py</code> and run <code>python3 probe_range.py &lt;gguf-url&gt;</code>. It downloads no weights, needs no account, and prints one JSON record per artifact.</p>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-text-size-adjust:none;"><code class="language-python" data-lang="python"><span style="display:flex;"><span><span style="color:#75715e">#!/usr/bin/env python3</span>
</span></span><span style="display:flex;"><span><span style="color:#e6db74">&#34;&#34;&#34;Read a GGUF header over HTTP Range: quant mix, bits-per-weight, parameters, context.&#34;&#34;&#34;</span>
</span></span><span style="display:flex;"><span><span style="color:#f92672">import</span> json
</span></span><span style="display:flex;"><span><span style="color:#f92672">import</span> struct
</span></span><span style="display:flex;"><span><span style="color:#f92672">import</span> sys
</span></span><span style="display:flex;"><span><span style="color:#f92672">import</span> urllib.request
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#75715e"># (elements_per_block, bytes_per_block) from llama.cpp gguf-py/gguf/constants.py</span>
</span></span><span style="display:flex;"><span>BLOCK <span style="color:#f92672">=</span> {<span style="color:#e6db74">&#34;F32&#34;</span>: (<span style="color:#ae81ff">1</span>, <span style="color:#ae81ff">4</span>), <span style="color:#e6db74">&#34;F16&#34;</span>: (<span style="color:#ae81ff">1</span>, <span style="color:#ae81ff">2</span>), <span style="color:#e6db74">&#34;BF16&#34;</span>: (<span style="color:#ae81ff">1</span>, <span style="color:#ae81ff">2</span>), <span style="color:#e6db74">&#34;Q4_0&#34;</span>: (<span style="color:#ae81ff">32</span>, <span style="color:#ae81ff">18</span>), <span style="color:#e6db74">&#34;Q4_1&#34;</span>: (<span style="color:#ae81ff">32</span>, <span style="color:#ae81ff">20</span>),
</span></span><span style="display:flex;"><span>         <span style="color:#e6db74">&#34;Q5_0&#34;</span>: (<span style="color:#ae81ff">32</span>, <span style="color:#ae81ff">22</span>), <span style="color:#e6db74">&#34;Q5_1&#34;</span>: (<span style="color:#ae81ff">32</span>, <span style="color:#ae81ff">24</span>), <span style="color:#e6db74">&#34;Q8_0&#34;</span>: (<span style="color:#ae81ff">32</span>, <span style="color:#ae81ff">34</span>), <span style="color:#e6db74">&#34;Q2_K&#34;</span>: (<span style="color:#ae81ff">256</span>, <span style="color:#ae81ff">84</span>),
</span></span><span style="display:flex;"><span>         <span style="color:#e6db74">&#34;Q3_K&#34;</span>: (<span style="color:#ae81ff">256</span>, <span style="color:#ae81ff">110</span>), <span style="color:#e6db74">&#34;Q4_K&#34;</span>: (<span style="color:#ae81ff">256</span>, <span style="color:#ae81ff">144</span>), <span style="color:#e6db74">&#34;Q5_K&#34;</span>: (<span style="color:#ae81ff">256</span>, <span style="color:#ae81ff">176</span>), <span style="color:#e6db74">&#34;Q6_K&#34;</span>: (<span style="color:#ae81ff">256</span>, <span style="color:#ae81ff">210</span>),
</span></span><span style="display:flex;"><span>         <span style="color:#e6db74">&#34;Q8_K&#34;</span>: (<span style="color:#ae81ff">256</span>, <span style="color:#ae81ff">292</span>)}
</span></span><span style="display:flex;"><span>SCALAR <span style="color:#f92672">=</span> {<span style="color:#ae81ff">0</span>: <span style="color:#e6db74">&#34;B&#34;</span>, <span style="color:#ae81ff">1</span>: <span style="color:#e6db74">&#34;b&#34;</span>, <span style="color:#ae81ff">2</span>: <span style="color:#e6db74">&#34;H&#34;</span>, <span style="color:#ae81ff">3</span>: <span style="color:#e6db74">&#34;h&#34;</span>, <span style="color:#ae81ff">4</span>: <span style="color:#e6db74">&#34;I&#34;</span>, <span style="color:#ae81ff">5</span>: <span style="color:#e6db74">&#34;i&#34;</span>, <span style="color:#ae81ff">6</span>: <span style="color:#e6db74">&#34;f&#34;</span>, <span style="color:#ae81ff">7</span>: <span style="color:#e6db74">&#34;?&#34;</span>,
</span></span><span style="display:flex;"><span>          <span style="color:#ae81ff">10</span>: <span style="color:#e6db74">&#34;Q&#34;</span>, <span style="color:#ae81ff">11</span>: <span style="color:#e6db74">&#34;q&#34;</span>, <span style="color:#ae81ff">12</span>: <span style="color:#e6db74">&#34;d&#34;</span>}
</span></span><span style="display:flex;"><span>ENUM <span style="color:#f92672">=</span> {<span style="color:#ae81ff">0</span>: <span style="color:#e6db74">&#34;F32&#34;</span>, <span style="color:#ae81ff">1</span>: <span style="color:#e6db74">&#34;F16&#34;</span>, <span style="color:#ae81ff">2</span>: <span style="color:#e6db74">&#34;Q4_0&#34;</span>, <span style="color:#ae81ff">3</span>: <span style="color:#e6db74">&#34;Q4_1&#34;</span>, <span style="color:#ae81ff">4</span>: <span style="color:#e6db74">&#34;Q4_1&#34;</span>, <span style="color:#ae81ff">5</span>: <span style="color:#e6db74">&#34;Q5_0&#34;</span>, <span style="color:#ae81ff">6</span>: <span style="color:#e6db74">&#34;Q5_0&#34;</span>,
</span></span><span style="display:flex;"><span>        <span style="color:#ae81ff">7</span>: <span style="color:#e6db74">&#34;Q5_1&#34;</span>, <span style="color:#ae81ff">8</span>: <span style="color:#e6db74">&#34;Q8_0&#34;</span>, <span style="color:#ae81ff">9</span>: <span style="color:#e6db74">&#34;Q8_1&#34;</span>, <span style="color:#ae81ff">10</span>: <span style="color:#e6db74">&#34;Q2_K&#34;</span>, <span style="color:#ae81ff">11</span>: <span style="color:#e6db74">&#34;Q3_K&#34;</span>, <span style="color:#ae81ff">12</span>: <span style="color:#e6db74">&#34;Q4_K&#34;</span>,
</span></span><span style="display:flex;"><span>        <span style="color:#ae81ff">13</span>: <span style="color:#e6db74">&#34;Q5_K&#34;</span>, <span style="color:#ae81ff">14</span>: <span style="color:#e6db74">&#34;Q6_K&#34;</span>, <span style="color:#ae81ff">15</span>: <span style="color:#e6db74">&#34;Q8_K&#34;</span>, <span style="color:#ae81ff">30</span>: <span style="color:#e6db74">&#34;BF16&#34;</span>}
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#66d9ef">class</span> <span style="color:#a6e22e">Reader</span>:
</span></span><span style="display:flex;"><span>    <span style="color:#66d9ef">def</span> <span style="color:#a6e22e">__init__</span>(self, buf):
</span></span><span style="display:flex;"><span>        self<span style="color:#f92672">.</span>b, self<span style="color:#f92672">.</span>p <span style="color:#f92672">=</span> buf, <span style="color:#ae81ff">0</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span>    <span style="color:#66d9ef">def</span> <span style="color:#a6e22e">take</span>(self, n):
</span></span><span style="display:flex;"><span>        <span style="color:#66d9ef">if</span> self<span style="color:#f92672">.</span>p <span style="color:#f92672">+</span> n <span style="color:#f92672">&gt;</span> len(self<span style="color:#f92672">.</span>b):
</span></span><span style="display:flex;"><span>            <span style="color:#66d9ef">raise</span> <span style="color:#a6e22e">EOFError</span>(<span style="color:#e6db74">f</span><span style="color:#e6db74">&#34;window ended at </span><span style="color:#e6db74">{</span>len(self<span style="color:#f92672">.</span>b)<span style="color:#e6db74">}</span><span style="color:#e6db74">; need </span><span style="color:#e6db74">{</span>n<span style="color:#e6db74">}</span><span style="color:#e6db74"> more&#34;</span>)
</span></span><span style="display:flex;"><span>        out <span style="color:#f92672">=</span> self<span style="color:#f92672">.</span>b[self<span style="color:#f92672">.</span>p:self<span style="color:#f92672">.</span>p <span style="color:#f92672">+</span> n]
</span></span><span style="display:flex;"><span>        self<span style="color:#f92672">.</span>p <span style="color:#f92672">+=</span> n
</span></span><span style="display:flex;"><span>        <span style="color:#66d9ef">return</span> out
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span>    <span style="color:#66d9ef">def</span> <span style="color:#a6e22e">scalar</span>(self, fmt):
</span></span><span style="display:flex;"><span>        <span style="color:#66d9ef">return</span> struct<span style="color:#f92672">.</span>unpack(<span style="color:#e6db74">&#34;&lt;&#34;</span> <span style="color:#f92672">+</span> fmt, self<span style="color:#f92672">.</span>take(struct<span style="color:#f92672">.</span>calcsize(fmt)))[<span style="color:#ae81ff">0</span>]
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span>    <span style="color:#66d9ef">def</span> <span style="color:#a6e22e">string</span>(self):
</span></span><span style="display:flex;"><span>        <span style="color:#66d9ef">return</span> self<span style="color:#f92672">.</span>take(self<span style="color:#f92672">.</span>scalar(<span style="color:#e6db74">&#34;Q&#34;</span>))<span style="color:#f92672">.</span>decode(<span style="color:#e6db74">&#34;utf-8&#34;</span>, <span style="color:#e6db74">&#34;replace&#34;</span>)
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span>    <span style="color:#66d9ef">def</span> <span style="color:#a6e22e">skip_value</span>(self, vtype):
</span></span><span style="display:flex;"><span>        <span style="color:#66d9ef">if</span> vtype <span style="color:#f92672">in</span> SCALAR:
</span></span><span style="display:flex;"><span>            self<span style="color:#f92672">.</span>take(struct<span style="color:#f92672">.</span>calcsize(SCALAR[vtype]))
</span></span><span style="display:flex;"><span>        <span style="color:#66d9ef">elif</span> vtype <span style="color:#f92672">==</span> <span style="color:#ae81ff">8</span>:
</span></span><span style="display:flex;"><span>            self<span style="color:#f92672">.</span>take(self<span style="color:#f92672">.</span>scalar(<span style="color:#e6db74">&#34;Q&#34;</span>))
</span></span><span style="display:flex;"><span>        <span style="color:#66d9ef">elif</span> vtype <span style="color:#f92672">==</span> <span style="color:#ae81ff">9</span>:
</span></span><span style="display:flex;"><span>            inner, count <span style="color:#f92672">=</span> self<span style="color:#f92672">.</span>scalar(<span style="color:#e6db74">&#34;I&#34;</span>), self<span style="color:#f92672">.</span>scalar(<span style="color:#e6db74">&#34;Q&#34;</span>)
</span></span><span style="display:flex;"><span>            <span style="color:#66d9ef">for</span> _ <span style="color:#f92672">in</span> range(count):
</span></span><span style="display:flex;"><span>                self<span style="color:#f92672">.</span>skip_value(inner)
</span></span><span style="display:flex;"><span>        <span style="color:#66d9ef">else</span>:
</span></span><span style="display:flex;"><span>            <span style="color:#66d9ef">raise</span> <span style="color:#a6e22e">ValueError</span>(<span style="color:#e6db74">f</span><span style="color:#e6db74">&#34;unknown metadata value type </span><span style="color:#e6db74">{</span>vtype<span style="color:#e6db74">}</span><span style="color:#e6db74">&#34;</span>)
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#66d9ef">def</span> <span style="color:#a6e22e">probe</span>(url, span<span style="color:#f92672">=</span><span style="color:#ae81ff">32</span> <span style="color:#f92672">*</span> <span style="color:#ae81ff">1024</span> <span style="color:#f92672">*</span> <span style="color:#ae81ff">1024</span>):
</span></span><span style="display:flex;"><span>    req <span style="color:#f92672">=</span> urllib<span style="color:#f92672">.</span>request<span style="color:#f92672">.</span>Request(url, headers<span style="color:#f92672">=</span>{<span style="color:#e6db74">&#34;Range&#34;</span>: <span style="color:#e6db74">f</span><span style="color:#e6db74">&#34;bytes=0-</span><span style="color:#e6db74">{</span>span <span style="color:#f92672">-</span> <span style="color:#ae81ff">1</span><span style="color:#e6db74">}</span><span style="color:#e6db74">&#34;</span>,
</span></span><span style="display:flex;"><span>                                              <span style="color:#e6db74">&#34;User-Agent&#34;</span>: <span style="color:#e6db74">&#34;gguf-range-probe/1.0&#34;</span>})
</span></span><span style="display:flex;"><span>    <span style="color:#66d9ef">with</span> urllib<span style="color:#f92672">.</span>request<span style="color:#f92672">.</span>urlopen(req, timeout<span style="color:#f92672">=</span><span style="color:#ae81ff">60</span>) <span style="color:#66d9ef">as</span> resp:
</span></span><span style="display:flex;"><span>        buf, status <span style="color:#f92672">=</span> resp<span style="color:#f92672">.</span>read(), resp<span style="color:#f92672">.</span>status
</span></span><span style="display:flex;"><span>        length <span style="color:#f92672">=</span> int(resp<span style="color:#f92672">.</span>headers[<span style="color:#e6db74">&#34;Content-Range&#34;</span>]<span style="color:#f92672">.</span>rsplit(<span style="color:#e6db74">&#34;/&#34;</span>, <span style="color:#ae81ff">1</span>)[<span style="color:#ae81ff">1</span>])
</span></span><span style="display:flex;"><span>    r <span style="color:#f92672">=</span> Reader(buf)
</span></span><span style="display:flex;"><span>    <span style="color:#66d9ef">if</span> r<span style="color:#f92672">.</span>take(<span style="color:#ae81ff">4</span>) <span style="color:#f92672">!=</span> <span style="color:#e6db74">b</span><span style="color:#e6db74">&#34;GGUF&#34;</span>:
</span></span><span style="display:flex;"><span>        <span style="color:#66d9ef">raise</span> <span style="color:#a6e22e">ValueError</span>(<span style="color:#e6db74">&#34;not a GGUF file&#34;</span>)
</span></span><span style="display:flex;"><span>    version, n_tensors, n_kv <span style="color:#f92672">=</span> r<span style="color:#f92672">.</span>scalar(<span style="color:#e6db74">&#34;I&#34;</span>), r<span style="color:#f92672">.</span>scalar(<span style="color:#e6db74">&#34;Q&#34;</span>), r<span style="color:#f92672">.</span>scalar(<span style="color:#e6db74">&#34;Q&#34;</span>)
</span></span><span style="display:flex;"><span>    meta <span style="color:#f92672">=</span> {}
</span></span><span style="display:flex;"><span>    <span style="color:#66d9ef">for</span> _ <span style="color:#f92672">in</span> range(n_kv):
</span></span><span style="display:flex;"><span>        key <span style="color:#f92672">=</span> r<span style="color:#f92672">.</span>string()
</span></span><span style="display:flex;"><span>        <span style="color:#66d9ef">if</span> SCALAR<span style="color:#f92672">.</span>get((vtype <span style="color:#f92672">:=</span> r<span style="color:#f92672">.</span>scalar(<span style="color:#e6db74">&#34;I&#34;</span>))):
</span></span><span style="display:flex;"><span>            meta[key] <span style="color:#f92672">=</span> r<span style="color:#f92672">.</span>scalar(SCALAR[vtype])
</span></span><span style="display:flex;"><span>        <span style="color:#66d9ef">elif</span> vtype <span style="color:#f92672">==</span> <span style="color:#ae81ff">8</span>:
</span></span><span style="display:flex;"><span>            meta[key] <span style="color:#f92672">=</span> r<span style="color:#f92672">.</span>string()
</span></span><span style="display:flex;"><span>        <span style="color:#66d9ef">else</span>:
</span></span><span style="display:flex;"><span>            r<span style="color:#f92672">.</span>skip_value(vtype)
</span></span><span style="display:flex;"><span>    tensors <span style="color:#f92672">=</span> []
</span></span><span style="display:flex;"><span>    <span style="color:#66d9ef">for</span> _ <span style="color:#f92672">in</span> range(n_tensors):
</span></span><span style="display:flex;"><span>        name <span style="color:#f92672">=</span> r<span style="color:#f92672">.</span>string()
</span></span><span style="display:flex;"><span>        dims <span style="color:#f92672">=</span> [r<span style="color:#f92672">.</span>scalar(<span style="color:#e6db74">&#34;Q&#34;</span>) <span style="color:#66d9ef">for</span> _ <span style="color:#f92672">in</span> range(r<span style="color:#f92672">.</span>scalar(<span style="color:#e6db74">&#34;I&#34;</span>))]
</span></span><span style="display:flex;"><span>        tensors<span style="color:#f92672">.</span>append((name, dims, ENUM<span style="color:#f92672">.</span>get(r<span style="color:#f92672">.</span>scalar(<span style="color:#e6db74">&#34;I&#34;</span>), <span style="color:#e6db74">&#34;?&#34;</span>), r<span style="color:#f92672">.</span>scalar(<span style="color:#e6db74">&#34;Q&#34;</span>)))
</span></span><span style="display:flex;"><span>    data_start <span style="color:#f92672">=</span> (r<span style="color:#f92672">.</span>p <span style="color:#f92672">+</span> <span style="color:#ae81ff">31</span>) <span style="color:#f92672">//</span> <span style="color:#ae81ff">32</span> <span style="color:#f92672">*</span> <span style="color:#ae81ff">32</span>
</span></span><span style="display:flex;"><span>    mix, elements, nbytes <span style="color:#f92672">=</span> {}, <span style="color:#ae81ff">0</span>, <span style="color:#ae81ff">0</span>
</span></span><span style="display:flex;"><span>    <span style="color:#66d9ef">for</span> _name, dims, qtype, _off <span style="color:#f92672">in</span> tensors:
</span></span><span style="display:flex;"><span>        count <span style="color:#f92672">=</span> <span style="color:#ae81ff">1</span>
</span></span><span style="display:flex;"><span>        <span style="color:#66d9ef">for</span> d <span style="color:#f92672">in</span> dims:
</span></span><span style="display:flex;"><span>            count <span style="color:#f92672">*=</span> d
</span></span><span style="display:flex;"><span>        <span style="color:#66d9ef">if</span> qtype <span style="color:#f92672">not</span> <span style="color:#f92672">in</span> BLOCK:
</span></span><span style="display:flex;"><span>            <span style="color:#66d9ef">raise</span> <span style="color:#a6e22e">ValueError</span>(<span style="color:#e6db74">f</span><span style="color:#e6db74">&#34;block table missing </span><span style="color:#e6db74">{</span>qtype<span style="color:#e6db74">}</span><span style="color:#e6db74">&#34;</span>)
</span></span><span style="display:flex;"><span>        per, width <span style="color:#f92672">=</span> BLOCK[qtype]
</span></span><span style="display:flex;"><span>        size <span style="color:#f92672">=</span> count <span style="color:#f92672">//</span> per <span style="color:#f92672">*</span> width
</span></span><span style="display:flex;"><span>        mix[qtype] <span style="color:#f92672">=</span> mix<span style="color:#f92672">.</span>get(qtype, <span style="color:#ae81ff">0</span>) <span style="color:#f92672">+</span> size
</span></span><span style="display:flex;"><span>        elements <span style="color:#f92672">+=</span> count
</span></span><span style="display:flex;"><span>        nbytes <span style="color:#f92672">+=</span> size
</span></span><span style="display:flex;"><span>    arch <span style="color:#f92672">=</span> meta<span style="color:#f92672">.</span>get(<span style="color:#e6db74">&#34;general.architecture&#34;</span>, <span style="color:#e6db74">&#34;?&#34;</span>)
</span></span><span style="display:flex;"><span>    <span style="color:#66d9ef">return</span> {<span style="color:#e6db74">&#34;url&#34;</span>: url, <span style="color:#e6db74">&#34;http_status&#34;</span>: status, <span style="color:#e6db74">&#34;content_length&#34;</span>: length,
</span></span><span style="display:flex;"><span>            <span style="color:#e6db74">&#34;range_bytes&#34;</span>: len(buf), <span style="color:#e6db74">&#34;gguf_version&#34;</span>: version,
</span></span><span style="display:flex;"><span>            <span style="color:#e6db74">&#34;architecture&#34;</span>: arch, <span style="color:#e6db74">&#34;file_type&#34;</span>: meta<span style="color:#f92672">.</span>get(<span style="color:#e6db74">&#34;general.file_type&#34;</span>),
</span></span><span style="display:flex;"><span>            <span style="color:#e6db74">&#34;context_length&#34;</span>: meta<span style="color:#f92672">.</span>get(<span style="color:#e6db74">f</span><span style="color:#e6db74">&#34;</span><span style="color:#e6db74">{</span>arch<span style="color:#e6db74">}</span><span style="color:#e6db74">.context_length&#34;</span>),
</span></span><span style="display:flex;"><span>            <span style="color:#e6db74">&#34;block_count&#34;</span>: meta<span style="color:#f92672">.</span>get(<span style="color:#e6db74">f</span><span style="color:#e6db74">&#34;</span><span style="color:#e6db74">{</span>arch<span style="color:#e6db74">}</span><span style="color:#e6db74">.block_count&#34;</span>),
</span></span><span style="display:flex;"><span>            <span style="color:#e6db74">&#34;parameter_count&#34;</span>: elements, <span style="color:#e6db74">&#34;weight_bytes&#34;</span>: nbytes,
</span></span><span style="display:flex;"><span>            <span style="color:#e6db74">&#34;bits_per_weight&#34;</span>: round(<span style="color:#ae81ff">8</span> <span style="color:#f92672">*</span> nbytes <span style="color:#f92672">/</span> elements, <span style="color:#ae81ff">4</span>),
</span></span><span style="display:flex;"><span>            <span style="color:#e6db74">&#34;quant_mix_bytes&#34;</span>: dict(sorted(mix<span style="color:#f92672">.</span>items())),
</span></span><span style="display:flex;"><span>            <span style="color:#e6db74">&#34;tensor_count&#34;</span>: n_tensors, <span style="color:#e6db74">&#34;header_data_start&#34;</span>: data_start,
</span></span><span style="display:flex;"><span>            <span style="color:#e6db74">&#34;weight_section_bytes&#34;</span>: length <span style="color:#f92672">-</span> data_start}
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#66d9ef">if</span> __name__ <span style="color:#f92672">==</span> <span style="color:#e6db74">&#34;__main__&#34;</span>:
</span></span><span style="display:flex;"><span>    print(json<span style="color:#f92672">.</span>dumps(probe(sys<span style="color:#f92672">.</span>argv[<span style="color:#ae81ff">1</span>], int(sys<span style="color:#f92672">.</span>argv[<span style="color:#ae81ff">2</span>]) <span style="color:#66d9ef">if</span> len(sys<span style="color:#f92672">.</span>argv) <span style="color:#f92672">&gt;</span> <span style="color:#ae81ff">2</span> <span style="color:#66d9ef">else</span> <span style="color:#ae81ff">32</span> <span style="color:#f92672">&lt;&lt;</span> <span style="color:#ae81ff">20</span>), indent<span style="color:#f92672">=</span><span style="color:#ae81ff">1</span>))
</span></span></code></pre></div><p>Run against the Gemma file, the script prints <code>content_length: 18323733440</code>, <code>parameter_count: 30697345596</code>, <code>bits_per_weight: 4.7712</code>, <code>header_data_start: 15834176</code> and the three-entry quantisation mix — the figures in the table above, reproduced from a fresh ranged read on 2026-11-06 at 14:41Z.</p>
<p>The run records behind those numbers — the probe output, the window-failure records, the registry manifests, the two params-layer bodies, the fetched spec and the method note — are kept in this article&rsquo;s evidence directory under <code>research/evidence/ollama-gguf-artifact-forensics-2026/</code> as a private author-side verification set, hashed in the review receipt so each figure can be re-checked against what was actually fetched. They are not offered as a downloadable dataset, and no reader-facing archive is promised; the script above regenerates every row on your machine.</p>
<h2 id="sources">Sources</h2>
<ul>
<li>GGUF file format specification, <code>ggml-org/ggml</code>: <a href="https://github.com/ggml-org/ggml/blob/master/docs/gguf.md">https://github.com/ggml-org/ggml/blob/master/docs/gguf.md</a></li>
<li>Ollama Modelfile reference (<code>PARAMETER num_ctx</code>): <a href="https://docs.ollama.com/modelfile.md">https://docs.ollama.com/modelfile.md</a></li>
<li><code>unsloth/gemma-4-31B-it-GGUF</code> file index with per-file sizes and SHA-256: <a href="https://huggingface.co/api/models/unsloth/gemma-4-31B-it-GGUF?blobs=true">https://huggingface.co/api/models/unsloth/gemma-4-31B-it-GGUF?blobs=true</a></li>
<li><code>Qwen/Qwen3-8B-GGUF</code> file index: <a href="https://huggingface.co/api/models/Qwen/Qwen3-8B-GGUF?blobs=true">https://huggingface.co/api/models/Qwen/Qwen3-8B-GGUF?blobs=true</a></li>
<li>Ollama registry manifest for <code>gemma4:31b</code>: <a href="https://registry.ollama.ai/v2/library/gemma4/manifests/31b">https://registry.ollama.ai/v2/library/gemma4/manifests/31b</a></li>
<li>Ollama registry manifest for <code>qwen3:8b</code>: <a href="https://registry.ollama.ai/v2/library/qwen3/manifests/8b">https://registry.ollama.ai/v2/library/qwen3/manifests/8b</a></li>
<li>Ollama <code>gemma4:31b</code> params layer (64 B, follows a 307 to object storage): <a href="https://registry.ollama.ai/v2/library/gemma4/blobs/sha256:2365fbb6d97bf654fdda0353b7505baa2254c662122cff0f153e4b47e6220ded">https://registry.ollama.ai/v2/library/gemma4/blobs/sha256:2365fbb6d97bf654fdda0353b7505baa2254c662122cff0f153e4b47e6220ded</a></li>
<li>Ollama <code>qwen3:8b</code> params layer (120 B, follows a 307 to object storage): <a href="https://registry.ollama.ai/v2/library/qwen3/blobs/sha256:cff3f395ef3756ab63e58b0ad1b32bb6f802905cae1472e6a12034e4246fbbdb">https://registry.ollama.ai/v2/library/qwen3/blobs/sha256:cff3f395ef3756ab63e58b0ad1b32bb6f802905cae1472e6a12034e4246fbbdb</a></li>
<li>Qwen3-8B model card (40,960-token allocation): <a href="https://huggingface.co/Qwen/Qwen3-8B">https://huggingface.co/Qwen/Qwen3-8B</a></li>
<li>llama.cpp quantisation block sizes, <code>gguf-py/gguf/constants.py</code>: <a href="https://raw.githubusercontent.com/ggml-org/llama.cpp/master/gguf-py/gguf/constants.py">https://raw.githubusercontent.com/ggml-org/llama.cpp/master/gguf-py/gguf/constants.py</a></li>
</ul>
]]></content:encoded></item><item><title>Reading an Ollama Model's Quantization and Size From Its Registry Manifest</title><link>https://baeseokjae.github.io/posts/ollama-tag-quant-bits-per-weight-audit-2026/</link><pubDate>Tue, 03 Nov 2026 00:30:00 +0000</pubDate><guid>https://baeseokjae.github.io/posts/ollama-tag-quant-bits-per-weight-audit-2026/</guid><description>A 32B Ollama tag can serve a 20.2 GB file. Read its declared quantization, separate model and projector bytes, and use matching tensor counts for bits per weight.</description><content:encoded><![CDATA[<p>You ran <code>ollama pull qwen3:32b</code> and got a model file of about 20.2 GB, or 18.8 GiB. A nominal 32-billion-parameter model needs 64 GB for bf16 weights alone. The downloaded model file is about 31.6% of that nominal baseline, not one-fifth. The tag says <code>32b</code>, but it does not spell out the quantization recipe.</p>
<p>You can check the registry&rsquo;s declared quantization and file sizes before downloading model-weight blobs. An Ollama manifest lists layers and byte sizes; its small config blob supplies <code>file_type</code>. Turning those bytes into bits per weight takes another step: the parameter count must describe the tensors in the same file.</p>
<p><strong>Correction, November 9, 2026:</strong> The original version confused ideal four-bit storage with a mixed-quantization file, used mismatched denominators for DeepSeek, Gemma and one Mistral tag, and treated an unsupported ±3% rule as an integrity check. This revision corrects the arithmetic, checks the tensor counts, and separates advertised file sizes from actual network and disk use.</p>
<h2 id="the-download-that-looks-broken">The download that looks broken</h2>
<p>For <code>qwen3:32b</code>, the manifest&rsquo;s model layer is 20,201,240,160 bytes:</p>
<ul>
<li><strong>20.201 decimal GB</strong>, using 1 GB = 1,000,000,000 bytes.</li>
<li><strong>18.814 GiB</strong>, using 1 GiB = 1,073,741,824 bytes.</li>
</ul>
<p>Those are two representations of the same file. Unit conversion accounts for that difference; it cannot by itself explain a much larger gap such as 18 GB versus 48 GB.</p>
<p>The idealized four-bit calculation is <code>32,000,000,000 × 4 / 8 = 16,000,000,000</code> bytes, or 16 GB. Real <code>Q4_K_M</code> files use a mixture of tensor types plus block scales, other quantization data, GGUF metadata and alignment. They need not average exactly four bits per parameter.</p>
<p>Using this artifact&rsquo;s exact 32,762,123,264 tensor elements, the ideal four-bit payload would be 16.381 GB and a bf16 payload would be 65.524 GB. Its 20.201 GB model file is therefore about <strong>30.8% of that exact bf16 baseline</strong>, with a file-level average of <strong>4.933 bits per stored tensor element</strong>. These are weight-storage comparisons, not estimates of total inference RAM or VRAM.</p>
<p>The November 3 snapshot recorded manifest SHA256 <code>030ee887880fc378860c2dd35101da424377520441ae4bfe7be6deff8ade7840</code>. The November 9 recheck returned the same hash. Tags can move; a digest identifies the bytes referenced by that particular snapshot.</p>
<h2 id="what-the-manifest-tells-you">What the manifest tells you</h2>
<p>A tag name is a label. Some names include a quantization suffix, such as <code>qwen3:32b-q8_0</code>; others leave it out. Resolve the tag and inspect these fields:</p>
<ol>
<li><strong><code>config.digest</code></strong> points to a small JSON blob. For <a href="https://registry.ollama.ai/v2/library/qwen3/manifests/32b">qwen3:32b</a>, the <a href="https://registry.ollama.ai/v2/library/qwen3/blobs/sha256:afdf5c7585b363536f93782f1e44f1b81db4237847cd8b4bd2c5c33b0d76c94b">488-byte config</a> declares <code>file_type: Q4_K_M</code> and <code>model_family: qwen3</code>. This is a file-level label, not a promise that every tensor has one storage type.</li>
<li><strong><code>layers[]</code></strong> lists each layer&rsquo;s <code>mediaType</code>, <code>size</code> and <code>digest</code>. This audit uses the complete <code>application/vnd.ollama.image.model</code> file for its numerator. That includes the GGUF container&rsquo;s metadata and alignment as well as tensor data.</li>
<li><strong>Other layers have separate roles.</strong> Templates, parameters and licenses are small in this sample. Multimodal Gemma also has an <code>application/vnd.ollama.image.projector</code> layer. On <code>gemma3:4b</code>, that layer is 851,251,104 bytes, about 34.1% of the 2,499,012,288-byte language-model file. Their sizes must be kept separate when calculating language-model bpw.</li>
<li><strong>The model-layer digest identifies the advertised artifact.</strong> Checking a downloaded blob against that digest verifies its bytes against the manifest. Reading a matching size or quant label alone does not do that.</li>
</ol>
<p>The sum of <code>layers[].size</code> is useful capacity information. It is <strong>not an exact network-transfer or incremental-disk figure</strong>: it excludes the config and manifest, downloads can reuse or resume cached blobs, and several tags can share one stored blob. HTTP overhead and filesystem allocation are separate concerns. <a href="https://github.com/ollama/ollama/blob/main/server/images.go">Ollama&rsquo;s pull implementation</a> handles config downloads, cache reuse and digest verification separately.</p>
<h2 id="method-match-the-file-and-the-denominator">Method: match the file and the denominator</h2>
<p>The table calculates:</p>
<p><code>file_bpw = model_layer_bytes × 8 / stored_tensor_elements</code></p>
<p>For the denominator, count the elements represented by the model file&rsquo;s tensor metadata: for each unique tensor name, multiply its dimensions, then sum across tensors. The tensor-count links in the table point to Ollama&rsquo;s tensor listings for the exact model blob. The correction recheck parsed those listings, rejected duplicate tensor names, and counted the tensors without downloading their weight data.</p>
<p>A Hugging Face repository&rsquo;s <code>safetensors.total</code> can be a useful cross-check, but it is not automatically the right divisor. The source repository must match the actual model version, and its scope must match the file. A multimodal total can include a vision component stored in another blob. Tied or duplicated weights also require attention when comparing source parameters with serialized tensor elements. Both Llama 3.1 blobs here store an additional 64-element <code>rope_freqs.weight</code> tensor. The table includes it, so its denominator describes serialized tensor elements rather than strictly learned parameters; the difference does not affect the displayed three-decimal bpw.</p>
<p>Exact quant-block sizes come from <a href="https://github.com/ggml-org/llama.cpp/blob/master/gguf-py/gguf/constants.py">llama.cpp&rsquo;s GGUF constants</a>: <code>Q4_K</code> stores 256 values in 144 bytes, or 4.5 bpw; <code>Q6_K</code> uses 210 bytes, or 6.5625 bpw; <code>Q8_0</code> stores 32 values in 34 bytes, or 8.5 bpw. Those block rates include scales and other block data, but not the whole GGUF container.</p>
<p><code>Q4_K_M</code> is a mixed quantization recipe, not a universal 4.85 bpw constant. Architecture, tensor shapes, quantizer choices and higher-precision tensors affect the file average. The <a href="https://github.com/ggml-org/llama.cpp/blob/master/src/llama-quant.cpp">quantizer source</a> shows both tensor-specific choices and fallbacks for incompatible shapes. There is no supported universal ±3% pass/fail test here. A plausible file size cannot prove the quantization layout, model quality or download integrity.</p>
<h2 id="the-audit-14-tags-7-families">The audit: 14 tags, 7 families</h2>
<p>The original manifest reads ran on November 3, 2026. During the November 9 correction recheck, all 14 manifest hashes were unchanged. The byte sizes below are therefore the same artifacts; the denominator corrections change their interpretation. Each row links to its registry manifest and to its model-blob tensor listing. All 14 captured configs declare <code>Q4_K_M</code> or <code>Q8_0</code> as shown.</p>
<p>“Model bytes” means the complete model GGUF file. “Layer sum” means only the manifest&rsquo;s layer sizes, excluding config and manifest bytes. GB is decimal; GiB is binary. The tensor count excludes any separate projector blob.</p>
<table>
	<thead>
			<tr>
					<th>Tag</th>
					<th>Declared type</th>
					<th style="text-align: right">Model bytes</th>
					<th style="text-align: right">GB / GiB</th>
					<th style="text-align: right">Stored tensor elements</th>
					<th style="text-align: right">File bpw</th>
					<th style="text-align: right">Layer sum (bytes)</th>
			</tr>
	</thead>
	<tbody>
			<tr>
					<td><a href="https://registry.ollama.ai/v2/library/qwen3/manifests/4b">qwen3:4b</a></td>
					<td>Q4_K_M</td>
					<td style="text-align: right">2,497,280,480</td>
					<td style="text-align: right">2.497 / 2.326</td>
					<td style="text-align: right"><a href="https://ollama.com/library/qwen3:4b/blobs/3e4cb1417446">4,022,468,096</a></td>
					<td style="text-align: right">4.967</td>
					<td style="text-align: right">2,497,293,444</td>
			</tr>
			<tr>
					<td><a href="https://registry.ollama.ai/v2/library/qwen3/manifests/8b">qwen3:8b</a></td>
					<td>Q4_K_M</td>
					<td style="text-align: right">5,225,374,496</td>
					<td style="text-align: right">5.225 / 4.867</td>
					<td style="text-align: right"><a href="https://ollama.com/library/qwen3:8b/blobs/a3de86cd1c13">8,190,735,360</a></td>
					<td style="text-align: right">5.104</td>
					<td style="text-align: right">5,225,387,677</td>
			</tr>
			<tr>
					<td><a href="https://registry.ollama.ai/v2/library/qwen3/manifests/32b">qwen3:32b</a></td>
					<td>Q4_K_M</td>
					<td style="text-align: right">20,201,240,160</td>
					<td style="text-align: right">20.201 / 18.814</td>
					<td style="text-align: right"><a href="https://ollama.com/library/qwen3:32b/blobs/3291abe70f16">32,762,123,264</a></td>
					<td style="text-align: right">4.933</td>
					<td style="text-align: right">20,201,253,341</td>
			</tr>
			<tr>
					<td><a href="https://registry.ollama.ai/v2/library/qwen3/manifests/32b-q8_0">qwen3:32b-q8_0</a></td>
					<td>Q8_0</td>
					<td style="text-align: right">35,132,291,680</td>
					<td style="text-align: right">35.132 / 32.719</td>
					<td style="text-align: right"><a href="https://ollama.com/library/qwen3:32b-q8_0/blobs/de447d788da3">32,762,123,264</a></td>
					<td style="text-align: right">8.579</td>
					<td style="text-align: right">35,132,304,861</td>
			</tr>
			<tr>
					<td><a href="https://registry.ollama.ai/v2/library/llama3.1/manifests/8b">llama3.1:8b</a></td>
					<td>Q4_K_M</td>
					<td style="text-align: right">4,920,738,944</td>
					<td style="text-align: right">4.921 / 4.583</td>
					<td style="text-align: right"><a href="https://ollama.com/library/llama3.1:8b/blobs/667b0c1932bc">8,030,261,312</a></td>
					<td style="text-align: right">4.902</td>
					<td style="text-align: right">4,920,752,841</td>
			</tr>
			<tr>
					<td><a href="https://registry.ollama.ai/v2/library/llama3.1/manifests/70b">llama3.1:70b</a></td>
					<td>Q4_K_M</td>
					<td style="text-align: right">42,520,398,176</td>
					<td style="text-align: right">42.520 / 39.600</td>
					<td style="text-align: right"><a href="https://ollama.com/library/llama3.1:70b/blobs/de20d2cf2dc4">70,553,706,560</a></td>
					<td style="text-align: right">4.821</td>
					<td style="text-align: right">42,520,412,073</td>
			</tr>
			<tr>
					<td><a href="https://registry.ollama.ai/v2/library/gemma3/manifests/4b">gemma3:4b</a></td>
					<td>Q4_K_M</td>
					<td style="text-align: right">2,499,012,288</td>
					<td style="text-align: right">2.499 / 2.327</td>
					<td style="text-align: right"><a href="https://ollama.com/library/gemma3:4b/blobs/f06e91e18fc7">3,880,101,888</a></td>
					<td style="text-align: right">5.152</td>
					<td style="text-align: right">3,350,272,259</td>
			</tr>
			<tr>
					<td><a href="https://registry.ollama.ai/v2/library/gemma3/manifests/12b">gemma3:12b</a></td>
					<td>Q4_K_M</td>
					<td style="text-align: right">7,309,830,496</td>
					<td style="text-align: right">7.310 / 6.808</td>
					<td style="text-align: right"><a href="https://ollama.com/library/gemma3:12b/blobs/13576eb6dd8d">11,765,792,256</a></td>
					<td style="text-align: right">4.970</td>
					<td style="text-align: right">8,164,039,587</td>
			</tr>
			<tr>
					<td><a href="https://registry.ollama.ai/v2/library/gemma3/manifests/27b">gemma3:27b</a></td>
					<td>Q4_K_M</td>
					<td style="text-align: right">16,555,661,472</td>
					<td style="text-align: right">16.556 / 15.419</td>
					<td style="text-align: right"><a href="https://ollama.com/library/gemma3:27b/blobs/3eb75691d2db">27,009,007,616</a></td>
					<td style="text-align: right">4.904</td>
					<td style="text-align: right">17,413,409,507</td>
			</tr>
			<tr>
					<td><a href="https://registry.ollama.ai/v2/library/mistral/manifests/7b">mistral:7b</a></td>
					<td>Q4_K_M</td>
					<td style="text-align: right">4,372,811,712</td>
					<td style="text-align: right">4.373 / 4.072</td>
					<td style="text-align: right"><a href="https://ollama.com/library/mistral:7b/blobs/f5074b1221da">7,248,023,552</a></td>
					<td style="text-align: right">4.826</td>
					<td style="text-align: right">4,372,823,897</td>
			</tr>
			<tr>
					<td><a href="https://registry.ollama.ai/v2/library/mistral/manifests/7b-instruct-q8_0">mistral:7b-instruct-q8_0</a></td>
					<td>Q8_0</td>
					<td style="text-align: right">7,696,794,208</td>
					<td style="text-align: right">7.697 / 7.168</td>
					<td style="text-align: right"><a href="https://ollama.com/library/mistral:7b-instruct-q8_0/blobs/0624ba75c3ce">7,241,732,096</a></td>
					<td style="text-align: right">8.503</td>
					<td style="text-align: right">7,696,805,637</td>
			</tr>
			<tr>
					<td><a href="https://registry.ollama.ai/v2/library/deepseek-r1/manifests/8b">deepseek-r1:8b</a></td>
					<td>Q4_K_M</td>
					<td style="text-align: right">5,225,373,760</td>
					<td style="text-align: right">5.225 / 4.867</td>
					<td style="text-align: right"><a href="https://ollama.com/library/deepseek-r1:8b/blobs/e6a7edc1a4d7">8,190,735,360</a></td>
					<td style="text-align: right">5.104</td>
					<td style="text-align: right">5,225,375,560</td>
			</tr>
			<tr>
					<td><a href="https://registry.ollama.ai/v2/library/qwen2.5/manifests/0.5b">qwen2.5:0.5b</a></td>
					<td>Q4_K_M</td>
					<td style="text-align: right">397,807,936</td>
					<td style="text-align: right">0.398 / 0.370</td>
					<td style="text-align: right"><a href="https://ollama.com/library/qwen2.5:0.5b/blobs/c5396e06af29">494,032,768</a></td>
					<td style="text-align: right">6.442</td>
					<td style="text-align: right">397,820,829</td>
			</tr>
			<tr>
					<td><a href="https://registry.ollama.ai/v2/library/phi4/manifests/14b">phi4:14b</a></td>
					<td>Q4_K_M</td>
					<td style="text-align: right">9,053,114,464</td>
					<td style="text-align: right">9.053 / 8.431</td>
					<td style="text-align: right"><a href="https://ollama.com/library/phi4:14b/blobs/fd7b6731c33c">14,659,507,200</a></td>
					<td style="text-align: right">4.940</td>
					<td style="text-align: right">9,053,115,905</td>
			</tr>
	</tbody>
</table>
<p>The corrected rows reveal why the denominator matters:</p>
<ul>
<li><strong>DeepSeek-R1 8B:</strong> The <a href="https://ollama.com/library/deepseek-r1:8b">official tag page</a> identifies <code>DeepSeek-R1-0528-Qwen3-8B</code>, not the earlier Llama-based distilled model. The <a href="https://huggingface.co/api/models/deepseek-ai/DeepSeek-R1-0528-Qwen3-8B">matching repository</a> has 8,190,735,360 parameters, agreeing with the blob&rsquo;s tensor count. That changes the result from <strong>5.206 to 5.104 bpw</strong>. The manifest hash is unchanged, so this was an incorrect source-model match, not a later tag update.</li>
<li><strong>Gemma 3:</strong> The language-model blobs contain 3,880,101,888, 11,765,792,256 and 27,009,007,616 tensor elements for the 4B, 12B and 27B tags. The original calculation divided those files by larger multimodal repository totals. With matching denominators, their file averages are <strong>5.152, 4.970 and 4.904 bpw</strong>. The old 4.649 figure for Gemma 4B did not demonstrate unusually low tensor precision. The separate projector layers are 851,251,104, 854,200,224 and 857,739,168 bytes.</li>
<li><strong>Mistral Q8_0:</strong> The <code>7b-instruct-q8_0</code> blob has 7,241,732,096 tensor elements and a 32,000-token vocabulary. The original denominator, 7,248,023,552, came from a different Mistral version. The matching blob count changes <strong>8.495 to 8.503 bpw</strong>. Similar family names do not establish identical source models.</li>
<li><strong>Qwen2.5 0.5B:</strong> Its <strong>6.442 bpw</strong> calculation is unchanged, but its explanation needs more than a general claim about embeddings. The blob&rsquo;s tensor listing shows Q8_0 token embeddings, many Q5_0 matrices, and a mixture of Q4_K, Q6_K and other tensor types. Its hidden width is 896, which is not divisible by the 256-value K-block size. llama.cpp&rsquo;s quantizer supports fallbacks from Q4_K to Q5_0 and from Q6_K to Q8_0 for incompatible shapes. The listed layout is consistent with that mechanism; it is not evidence of a corrupt file. The 494,032,768-element total includes no separately stored output.weight tensor.</li>
</ul>
<h2 id="one-model-two-quantizations">One model, two quantizations</h2>
<p>The two Qwen3 32B blobs have the same 32,762,123,264 tensor elements. Their model files grow from 20,201,240,160 bytes at <code>Q4_K_M</code> to 35,132,291,680 bytes at <code>Q8_0</code>: <strong>1.739 times the storage</strong>, or about <strong>73.9% more</strong>. The file averages are 4.933 and 8.579 bpw.</p>
<p>That is a file-size comparison. It does not measure quality, tokens per second, runtime memory or memory traffic. Those depend on the implementation, offloading, batching, cache behavior and workload, and need their own measurements.</p>
<p>For a particular resolved model-blob digest, the artifact bytes do not change with your VRAM. Hardware still affects whether and how the model runs. This observation concerns the captured plain GGUF manifests; it does not assert that every tag, platform or future manifest-list selection always resolves to the same artifact.</p>
<h2 id="run-it-yourself">Run it yourself</h2>
<p>Save this as <code>manifest_audit.py</code>. It reads a public library manifest and config, checks the config&rsquo;s declared digest and size, and reports model sizes and the layer sum. It does not download or verify the model-weight blobs. An optional parameter count produces file bpw; leave it out until you have matched the count to the blob&rsquo;s tensor metadata.</p>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-text-size-adjust:none;"><code class="language-python" data-lang="python"><span style="display:flex;"><span><span style="color:#75715e">#!/usr/bin/env python3</span>
</span></span><span style="display:flex;"><span><span style="color:#e6db74">&#34;&#34;&#34;Read public library manifests; optional family:tag=parameter_count.&#34;&#34;&#34;</span>
</span></span><span style="display:flex;"><span><span style="color:#f92672">import</span> hashlib
</span></span><span style="display:flex;"><span><span style="color:#f92672">import</span> json
</span></span><span style="display:flex;"><span><span style="color:#f92672">import</span> re
</span></span><span style="display:flex;"><span><span style="color:#f92672">import</span> sys
</span></span><span style="display:flex;"><span><span style="color:#f92672">import</span> urllib.request
</span></span><span style="display:flex;"><span><span style="color:#f92672">from</span> datetime <span style="color:#f92672">import</span> datetime, timezone
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#66d9ef">def</span> <span style="color:#a6e22e">get</span>(url):
</span></span><span style="display:flex;"><span>    req <span style="color:#f92672">=</span> urllib<span style="color:#f92672">.</span>request<span style="color:#f92672">.</span>Request(url, headers<span style="color:#f92672">=</span>{<span style="color:#e6db74">&#34;User-Agent&#34;</span>: <span style="color:#e6db74">&#34;manifest-audit/2.0&#34;</span>})
</span></span><span style="display:flex;"><span>    <span style="color:#66d9ef">with</span> urllib<span style="color:#f92672">.</span>request<span style="color:#f92672">.</span>urlopen(req, timeout<span style="color:#f92672">=</span><span style="color:#ae81ff">30</span>) <span style="color:#66d9ef">as</span> response:
</span></span><span style="display:flex;"><span>        body <span style="color:#f92672">=</span> response<span style="color:#f92672">.</span>read()
</span></span><span style="display:flex;"><span>        <span style="color:#66d9ef">return</span> body, response<span style="color:#f92672">.</span>headers<span style="color:#f92672">.</span>get(<span style="color:#e6db74">&#34;Date&#34;</span>), datetime<span style="color:#f92672">.</span>now(timezone<span style="color:#f92672">.</span>utc)<span style="color:#f92672">.</span>isoformat()
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#66d9ef">def</span> <span style="color:#a6e22e">audit</span>(spec):
</span></span><span style="display:flex;"><span>    <span style="color:#66d9ef">match</span> <span style="color:#f92672">=</span> re<span style="color:#f92672">.</span>fullmatch(<span style="color:#e6db74">r</span><span style="color:#e6db74">&#34;([A-Za-z0-9_.-]+):([A-Za-z0-9_.-]+)(?:=([1-9][0-9]*))?&#34;</span>, spec)
</span></span><span style="display:flex;"><span>    <span style="color:#66d9ef">if</span> <span style="color:#f92672">not</span> <span style="color:#66d9ef">match</span>:
</span></span><span style="display:flex;"><span>        <span style="color:#66d9ef">raise</span> <span style="color:#a6e22e">ValueError</span>(<span style="color:#e6db74">&#34;Expected family:tag or family:tag=positive_parameter_count&#34;</span>)
</span></span><span style="display:flex;"><span>    family, tag, count <span style="color:#f92672">=</span> <span style="color:#66d9ef">match</span><span style="color:#f92672">.</span>groups()
</span></span><span style="display:flex;"><span>    base <span style="color:#f92672">=</span> <span style="color:#e6db74">f</span><span style="color:#e6db74">&#34;https://registry.ollama.ai/v2/library/</span><span style="color:#e6db74">{</span>family<span style="color:#e6db74">}</span><span style="color:#e6db74">&#34;</span>
</span></span><span style="display:flex;"><span>    body, server_date, read_at <span style="color:#f92672">=</span> get(<span style="color:#e6db74">f</span><span style="color:#e6db74">&#34;</span><span style="color:#e6db74">{</span>base<span style="color:#e6db74">}</span><span style="color:#e6db74">/manifests/</span><span style="color:#e6db74">{</span>tag<span style="color:#e6db74">}</span><span style="color:#e6db74">&#34;</span>)
</span></span><span style="display:flex;"><span>    manifest <span style="color:#f92672">=</span> json<span style="color:#f92672">.</span>loads(body)
</span></span><span style="display:flex;"><span>    <span style="color:#66d9ef">if</span> <span style="color:#e6db74">&#34;layers&#34;</span> <span style="color:#f92672">not</span> <span style="color:#f92672">in</span> manifest:
</span></span><span style="display:flex;"><span>        <span style="color:#66d9ef">raise</span> <span style="color:#a6e22e">ValueError</span>(<span style="color:#e6db74">&#34;Not a plain model manifest; resolve its manifest list first&#34;</span>)
</span></span><span style="display:flex;"><span>    layers <span style="color:#f92672">=</span> manifest[<span style="color:#e6db74">&#34;layers&#34;</span>]
</span></span><span style="display:flex;"><span>    model_layers <span style="color:#f92672">=</span> [x <span style="color:#66d9ef">for</span> x <span style="color:#f92672">in</span> layers <span style="color:#66d9ef">if</span> x[<span style="color:#e6db74">&#34;mediaType&#34;</span>] <span style="color:#f92672">==</span> <span style="color:#e6db74">&#34;application/vnd.ollama.image.model&#34;</span>]
</span></span><span style="display:flex;"><span>    <span style="color:#66d9ef">if</span> <span style="color:#f92672">not</span> model_layers:
</span></span><span style="display:flex;"><span>        <span style="color:#66d9ef">raise</span> <span style="color:#a6e22e">ValueError</span>(<span style="color:#e6db74">&#34;No model layer; do not infer a parameter ratio&#34;</span>)
</span></span><span style="display:flex;"><span>    config_ref <span style="color:#f92672">=</span> manifest[<span style="color:#e6db74">&#34;config&#34;</span>]
</span></span><span style="display:flex;"><span>    <span style="color:#66d9ef">if</span> <span style="color:#f92672">not</span> re<span style="color:#f92672">.</span>fullmatch(<span style="color:#e6db74">r</span><span style="color:#e6db74">&#34;sha256:[0-9a-f]</span><span style="color:#e6db74">{64}</span><span style="color:#e6db74">&#34;</span>, config_ref[<span style="color:#e6db74">&#34;digest&#34;</span>]):
</span></span><span style="display:flex;"><span>        <span style="color:#66d9ef">raise</span> <span style="color:#a6e22e">ValueError</span>(<span style="color:#e6db74">&#34;Unsupported config digest&#34;</span>)
</span></span><span style="display:flex;"><span>    config_body, _, _ <span style="color:#f92672">=</span> get(<span style="color:#e6db74">f</span><span style="color:#e6db74">&#34;</span><span style="color:#e6db74">{</span>base<span style="color:#e6db74">}</span><span style="color:#e6db74">/blobs/</span><span style="color:#e6db74">{</span>config_ref[<span style="color:#e6db74">&#39;digest&#39;</span>]<span style="color:#e6db74">}</span><span style="color:#e6db74">&#34;</span>)
</span></span><span style="display:flex;"><span>    <span style="color:#66d9ef">if</span> (len(config_body) <span style="color:#f92672">!=</span> config_ref[<span style="color:#e6db74">&#34;size&#34;</span>] <span style="color:#f92672">or</span>
</span></span><span style="display:flex;"><span>            <span style="color:#e6db74">&#34;sha256:&#34;</span> <span style="color:#f92672">+</span> hashlib<span style="color:#f92672">.</span>sha256(config_body)<span style="color:#f92672">.</span>hexdigest() <span style="color:#f92672">!=</span> config_ref[<span style="color:#e6db74">&#34;digest&#34;</span>]):
</span></span><span style="display:flex;"><span>        <span style="color:#66d9ef">raise</span> <span style="color:#a6e22e">ValueError</span>(<span style="color:#e6db74">&#34;Config size or digest mismatch&#34;</span>)
</span></span><span style="display:flex;"><span>    config <span style="color:#f92672">=</span> json<span style="color:#f92672">.</span>loads(config_body)
</span></span><span style="display:flex;"><span>    model_bytes <span style="color:#f92672">=</span> sum(x[<span style="color:#e6db74">&#34;size&#34;</span>] <span style="color:#66d9ef">for</span> x <span style="color:#f92672">in</span> model_layers)
</span></span><span style="display:flex;"><span>    parameters <span style="color:#f92672">=</span> int(count) <span style="color:#66d9ef">if</span> count <span style="color:#66d9ef">else</span> <span style="color:#66d9ef">None</span>
</span></span><span style="display:flex;"><span>    <span style="color:#66d9ef">return</span> {
</span></span><span style="display:flex;"><span>        <span style="color:#e6db74">&#34;tag&#34;</span>: <span style="color:#e6db74">f</span><span style="color:#e6db74">&#34;</span><span style="color:#e6db74">{</span>family<span style="color:#e6db74">}</span><span style="color:#e6db74">:</span><span style="color:#e6db74">{</span>tag<span style="color:#e6db74">}</span><span style="color:#e6db74">&#34;</span>, <span style="color:#e6db74">&#34;read_at_utc&#34;</span>: read_at, <span style="color:#e6db74">&#34;response_date&#34;</span>: server_date,
</span></span><span style="display:flex;"><span>        <span style="color:#e6db74">&#34;manifest_sha256&#34;</span>: hashlib<span style="color:#f92672">.</span>sha256(body)<span style="color:#f92672">.</span>hexdigest(),
</span></span><span style="display:flex;"><span>        <span style="color:#e6db74">&#34;model_family&#34;</span>: config<span style="color:#f92672">.</span>get(<span style="color:#e6db74">&#34;model_family&#34;</span>), <span style="color:#e6db74">&#34;file_type&#34;</span>: config<span style="color:#f92672">.</span>get(<span style="color:#e6db74">&#34;file_type&#34;</span>),
</span></span><span style="display:flex;"><span>        <span style="color:#e6db74">&#34;model_layers&#34;</span>: model_layers, <span style="color:#e6db74">&#34;model_bytes&#34;</span>: model_bytes,
</span></span><span style="display:flex;"><span>        <span style="color:#e6db74">&#34;decimal_gb&#34;</span>: round(model_bytes <span style="color:#f92672">/</span> <span style="color:#ae81ff">10</span><span style="color:#f92672">**</span><span style="color:#ae81ff">9</span>, <span style="color:#ae81ff">3</span>), <span style="color:#e6db74">&#34;gib&#34;</span>: round(model_bytes <span style="color:#f92672">/</span> <span style="color:#ae81ff">2</span><span style="color:#f92672">**</span><span style="color:#ae81ff">30</span>, <span style="color:#ae81ff">3</span>),
</span></span><span style="display:flex;"><span>        <span style="color:#e6db74">&#34;all_layer_bytes&#34;</span>: sum(x[<span style="color:#e6db74">&#34;size&#34;</span>] <span style="color:#66d9ef">for</span> x <span style="color:#f92672">in</span> layers),
</span></span><span style="display:flex;"><span>        <span style="color:#e6db74">&#34;config_bytes&#34;</span>: config_ref[<span style="color:#e6db74">&#34;size&#34;</span>], <span style="color:#e6db74">&#34;manifest_bytes&#34;</span>: len(body),
</span></span><span style="display:flex;"><span>        <span style="color:#e6db74">&#34;parameters_supplied&#34;</span>: parameters,
</span></span><span style="display:flex;"><span>        <span style="color:#e6db74">&#34;file_bpw&#34;</span>: round(model_bytes <span style="color:#f92672">*</span> <span style="color:#ae81ff">8</span> <span style="color:#f92672">/</span> parameters, <span style="color:#ae81ff">3</span>) <span style="color:#66d9ef">if</span> parameters <span style="color:#66d9ef">else</span> <span style="color:#66d9ef">None</span>,
</span></span><span style="display:flex;"><span>    }
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#66d9ef">if</span> __name__ <span style="color:#f92672">==</span> <span style="color:#e6db74">&#34;__main__&#34;</span>:
</span></span><span style="display:flex;"><span>    <span style="color:#66d9ef">if</span> <span style="color:#f92672">not</span> sys<span style="color:#f92672">.</span>argv[<span style="color:#ae81ff">1</span>:]:
</span></span><span style="display:flex;"><span>        <span style="color:#66d9ef">raise</span> <span style="color:#a6e22e">SystemExit</span>(<span style="color:#e6db74">&#34;Usage: python3 manifest_audit.py family:tag[=parameter_count] ...&#34;</span>)
</span></span><span style="display:flex;"><span>    <span style="color:#66d9ef">for</span> spec <span style="color:#f92672">in</span> sys<span style="color:#f92672">.</span>argv[<span style="color:#ae81ff">1</span>:]:
</span></span><span style="display:flex;"><span>        print(json<span style="color:#f92672">.</span>dumps(audit(spec)))
</span></span></code></pre></div><p>Run <code>python3 manifest_audit.py qwen3:32b=32762123264 phi4:14b</code>. The Qwen row includes <code>file_bpw: 4.933</code>; the Phi row leaves it <code>null</code> because no denominator was supplied. Compare the returned hashes with your saved snapshot. A later run is not guaranteed to return the same hash because tags can move.</p>
<p><code>read_at_utc</code> is taken after the manifest body is read. <code>response_date</code> is the HTTP Date header and may reflect an intermediary&rsquo;s cached response; it is not a substitute for recording your own observation time. The script deliberately rejects manifest lists rather than silently choosing an artifact. It supports the plain public <code>library/family:tag</code> examples in this article, not every registry or authenticated model.</p>
<h2 id="what-this-does-not-tell-you">What this does not tell you</h2>
<ul>
<li><strong>File bpw is storage accounting, not quality.</strong> Similar averages can hide different tensor layouts and quantization choices. An accuracy or perplexity claim needs an evaluation.</li>
<li><strong>The registry declares the bytes.</strong> This correction fetched manifests, configs and tensor metadata, not full model-weight blobs. It did not independently check every advertised tensor against downloaded weights.</li>
<li><strong>Tags move and model identities matter.</strong> Keep the manifest hash, layer digest, observation time and denominator provenance together. A family name and rounded “B” count are insufficient.</li>
<li><strong>Fourteen tags are a sample.</strong> These observations do not establish a universal range or pass/fail threshold for a quantization recipe.</li>
<li><strong>Storage is not runtime memory.</strong> KV cache, activations, work buffers, offloaded layers and backend choices are outside this audit.</li>
</ul>
<p>Read <code>file_type</code> to learn the declared quantization recipe. Read layer sizes to plan storage. Match tensor scope before calculating file bpw, and verify blob digests when checking a download. Each step answers a different question.</p>
<h2 id="sources-and-correction-evidence">Sources and correction evidence</h2>
<p>The row links identify the primary registry manifests and digest-specific tensor listings. The November 9 recheck preserved response times, response hashes, manifests, configs and the parsed tensor names, types and dimensions in the repository&rsquo;s correction evidence. The original November 3 CSV is retained as the historical record; it has not been silently rewritten to erase the corrected denominators.</p>
<p>For implementation details, see the linked llama.cpp constants and quantizer, and Ollama&rsquo;s pull implementation. The <a href="https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct/raw/main/config.json">Qwen2.5-0.5B-Instruct config</a> documents its tied embeddings and dimensions. The <a href="https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.1/raw/main/config.json">Mistral-7B-Instruct-v0.1 config</a> matches the Q8 artifact&rsquo;s vocabulary and RoPE settings, but matching architecture alone does not prove the exact weight lineage; the table uses the blob&rsquo;s counted elements instead.</p>
<p>Related reading on this site: <a href="https://baeseokjae.github.io/posts/best-local-llm-models-2026/">the idealized params × bits ÷ 8 sizing formula and why real artifacts differ</a>, <a href="https://baeseokjae.github.io/posts/qwen-3-32b-local-guide-2026/">the Qwen3 32B setup guide</a>, and <a href="https://baeseokjae.github.io/posts/qwen-3-full-lineup-guide-2026/">the deployment checklist that asks you to record quantization</a>.</p>
]]></content:encoded></item></channel></rss>