Choosing a local LLM starts with the exact model identity and the resources it needs. This guide covers the families discussed in the original article and explains how to evaluate their fit. It does not present a locally measured benchmark leaderboard.
Correction — October 7, 2026: Llama 3.3 was incorrectly described as an 8B model, and Mistral Small 3 as a 7B model. Their correct sizes are 70B and 24B. The previous hardware-speed table, blanket privacy claims and inconsistent cloud-cost calculations have also been removed or corrected.
Model Identities to Check Before Downloading
| Model | Correct identity | Selection consideration |
|---|---|---|
| Llama 3.3 | 70B instruction-tuned model | A large-memory option, not an 8B laptop model |
| Qwen2.5-Coder | Code-specialized family with size-specific tags | Distinguish it from general Qwen2.5 models |
| Phi-4 | 14B model | Evaluate the exact checkpoint rather than combining Phi variants |
| Mistral Small 3 | 24B model | Distinct from Mistral 7B and later Small releases |
| DeepSeek-R1 | Full 671B-total MoE model plus separately named distilled models | Smaller distills are not the full model at a different precision |
These are candidates from this article’s scope, not a claim that all newer releases have been assessed. For the original Qwen3 family, use the size- and version-specific guide.
Llama 3.3 70B
The checked Ollama listing contains the 70B model and shows an approximately 43 GB default artifact. That download size is not a complete runtime memory requirement. The earlier 6–8 GB and laptop throughput claims referred to a model identity that did not match Llama 3.3 and should not guide a purchase.
Mistral Small 3 24B
Mistral’s release announcement specifies 24B parameters. Ollama lists the corresponding model under mistral-small:24b, with an approximately 14 GB default artifact. An artifact that occupies 14 GB does not leave adequate runtime and operating-system headroom on every 16 GB machine. The former 7B/50-tokens-per-second recommendation was not supported.
Qwen, Phi and DeepSeek Families
Use a complete model name when recording an evaluation. qwen2.5:14b and qwen2.5-coder:14b identify different choices. Phi-4 and Phi-4-mini are different checkpoints. An R1 distill inherits a smaller underlying architecture and must be evaluated in its own right.
A task-specific benchmark can help shortlist candidates, but it does not establish the best model for every coding, math or assistant workload. A visible reasoning trace also does not prove that the explanation is correct or faithfully reveals how the answer was produced.
Local Inference: Control, Privacy and Costs
Local execution can keep model inference on your own hardware. That property does not automatically extend to embeddings, web search, plugins, telemetry or a remotely hosted model selected through the same application. Ollama’s FAQ explicitly distinguishes its local and cloud features.
For sensitive data, document each component that can send a request off the machine and verify the configured path. Where offline operation is required, test the intended workflow after dependencies and weights are available, with external network access disabled. An application’s location alone is not proof of regulatory compliance.
Local inference also shifts costs. You may replace a hosted token bill with equipment, power, maintenance and administration. Whether that is cheaper depends on utilization, model quality and workload, rather than a universal monthly-token threshold.
Hardware Planning: Start with Weights, Then Add Runtime Memory
An approximate weight-only calculation is parameter count × bits per weight ÷ 8. Using rounded parameter counts and decimal GB gives:
| Size | Idealized 4-bit weights | 16-bit weights |
|---|---|---|
| 7B | 3.5 GB | 14 GB |
| 14B | 7 GB | 28 GB |
| 24B | 12 GB | 48 GB |
| 70B | 35 GB | 140 GB |
These are arithmetic estimates. Quantization metadata, mixed precision, KV cache, temporary buffers and other applications add to memory use. Actual downloaded artifacts can be larger than the idealized estimate. Longer context and concurrent requests can increase runtime requirements.
On Apple Silicon, the GPU is integrated and shares memory with the rest of the system. That is different from CPU-only inference. On a machine with a discrete GPU, offloading to system RAM can enable a larger model, but its speed must be measured. Neither setup guarantees the throughput previously claimed in this article.
Quantization Is a Tradeoff to Measure
Lower-bit weights generally reduce storage, but quality and speed effects depend on the model, quantization method, backend and task. Q4_K_M is one option to evaluate; it is not a guarantee of less than a fixed percentage of quality loss. More available memory also does not automatically make Q8 the best choice when context or concurrency matters more.
Runtime Selection and API Compatibility
Choose a runtime by operating-system support, model format, deployment needs and the controls your application requires. Verify those requirements against the current documentation of Ollama, LM Studio or llama.cpp. The earlier platform matrix and universal performance rankings have been removed because they were not adequately supported.
Ollama’s native API and its OpenAI-compatible endpoints are separate interfaces. The latter implements parts of the OpenAI API; it does not guarantee support for every feature of every client. See the compatibility documentation.
Local Versus Cloud: An Explicit Cost Example
Use the provider’s current rate for the exact model, input/output mix, caching and billing mode. The calculation is:
cloud token cost = input_tokens / 1,000,000 × input_rate
+ output_tokens / 1,000,000 × output_rate
For an illustrative, non-provider-specific rate of $2 per million input tokens and $10 per million output tokens:
| Monthly workload | Calculation | Token cost |
|---|---|---|
| 4M input + 1M output | 4 × $2 + 1 × $10 | $18 |
| 40M input + 10M output | 40 × $2 + 10 × $10 | $180 |
These are hypothetical examples, not current product quotations. Tool calls, storage and other charges are excluded. The previous claim that 50M tokens necessarily cost thousands of dollars per month did not follow from the rates shown.
For local operation, estimate monthly equipment allocation or rental, electricity, maintenance and any remote services. Keep cash expenditure separate from an accounting allocation of an already purchased machine. Compare equivalent task quality and usable capacity. A cheaper run that needs substantially more correction may cost more overall.
Getting Started Without Assuming a Hardware Fit
Choose a verified smaller checkpoint for an initial trial and inspect it before increasing size. This documentation-based example does not certify performance on your machine:
ollama pull qwen2.5-coder:7b
ollama show qwen2.5-coder:7b
ollama run qwen2.5-coder:7b
ollama ps
The exact tags are listed in the Qwen2.5-Coder catalog. In ollama ps, inspect whether the model is loaded on the CPU, GPU or a mixture. Save the configuration when comparing another candidate.
Use a handful of tasks drawn from your own work. Check factual answers against known references and code against tests. Record failures, memory use and elapsed time, then decide whether a larger checkpoint improves enough to justify its cost.
FAQ
Can I run Llama 3.3 as an 8B model?
The model covered here is 70B. An 8B model from another Llama release is a separate model and needs its own card and evaluation.
How much RAM is enough?
There is no universal minimum for a parameter class. Check the actual artifact and include context, runtime overhead, concurrency and operating-system needs.
Is Phi-4 definitely better than Llama 3.3 for math?
This article does not establish that ranking. The previous comparison used an incorrect Llama model identity and lacked a reproducible common evaluation setup.
Does running on a Mac mean CPU-only inference?
No. Apple Silicon systems include an integrated GPU. Verify which backend the runtime uses and how much shared memory remains available.
Does a local model guarantee no data leaves the machine?
No. Inspect the full application, selected endpoint and connected tools. Local weights alone do not establish that the surrounding workflow is offline.