What You Actually Pulled: Reading Quantisation, Parameter Count and Context Out of an Ollama GGUF File

You pulled a tag. The download is 18.99 GiB for a model the card calls 31B. The agent that needed long history behaves as if it had a fraction of the context the card advertises. The quality does not match the benchmark chart that made you choose the model in the first place. None of those three surprises is a property of the model. All three are properties of the artifact: which quantisation the tag actually resolved to, how many bits per weight that quantisation really costs, and which metadata value declares the context this particular build was made for. ...

November 6, 2026 · 14 min · baeseokjae

Reading an Ollama Model's Quantization and Size From Its Registry Manifest

You ran ollama pull qwen3:32b and got a model file of about 20.2 GB, or 18.8 GiB. A nominal 32-billion-parameter model needs 64 GB for bf16 weights alone. The downloaded model file is about 31.6% of that nominal baseline, not one-fifth. The tag says 32b, but it does not spell out the quantization recipe. You can check the registry’s declared quantization and file sizes before downloading model-weight blobs. An Ollama manifest lists layers and byte sizes; its small config blob supplies file_type. Turning those bytes into bits per weight takes another step: the parameter count must describe the tensors in the same file. ...

November 3, 2026 · 11 min · baeseokjae
Unsloth Dynamic 3.0 GGUF quantization improves local LLM inference accuracy at matched size

Unsloth Dynamic GGUF 3.0: Faster Local Inference Without Sacrificing Accuracy

Unsloth Dynamic GGUF 3.0 is the latest iteration of Unsloth’s quantization format for running large models locally. It delivers more than 10% better top-1% accuracy at the same file size as every other GGUF provider, works with llama.cpp and Unsloth Desktop, and makes a 27B-parameter model runnable on as little as 7-8GB of RAM. This review explains what changed, how the quality claims are measured, and exactly which quant to pick for your hardware. ...

September 18, 2026 · 12 min · baeseokjae
SLM on ESP32-S3: Training a Small Language Model on an $8 Microcontroller 2026

SLM on ESP32-S3: Training a Small Language Model on an $8 Microcontroller 2026

Introduction — Why Run a Language Model on an $8 Microcontroller? The idea of running a language model on a microcontroller that costs less than a cup of coffee sounds improbable, but the ESP32-S3 makes it a reality. With dual-core Xtensa LX7 processors running at up to 240 MHz, 512 KB of internal SRAM, and support for up to 8 MB of external octal SPI PSRAM, this $6–8 chip can execute small language models (SLMs) entirely offline — no cloud, no WiFi dependency, no API keys required. ...

August 6, 2026 · 14 min · baeseokjae