Investigations
Two models fit in 48 GB; the speed does not
Both loaded in 26.78 GB of RAM, and generating together dropped each one to about 9 tokens a second.
7 minAI
Everything we publish on AI: the models we run on our own machines, what the tools send home, the benchmarks we measure ourselves and the claims we check against the code. Start with the investigations, then the guides.
Investigations
A 64K context costs 4.4 GB your model manager hides
Measured on a 48 GB Mac: 80 KB of RAM per token, and LM Studio reports 18.85 GB whether you load 8K or 64K.
8 min
Investigations
Your prompt benchmark is timing the cache
An M4 Pro reads prompts at 65 tokens a second; repeat the prompt and the cache reports 13,763, which is 182 times too fast.
8 min
Investigations
A 4-bit Nemotron is 12 percent 16-bit
NVIDIA's 22.46 GB Lightning GGUF holds 2.67 GB of BF16: the MTP head, which four publishers ship four ways.
8 min
Investigations
Your 4-bit vision model sees in 16-bit
An mmproj GGUF holds the whole vision encoder; four publishers ship it unquantised, and the one that compresses stops at 8.
7 min
Investigations
Hugging Face’s telemetry function is never called
An opt-out variable, a send function nothing uses, and the real trail: a user-agent header naming your Python and torch.
7 min
Investigations
The smallest Kimi K3 is 589 GB
Every shard summed: the smallest complete build is 589.5 GB, the family runs to 769, and two builds disagree by 58.
7 min
Investigations
LM Studio does not track you, but it routes every search
What the app sends, read from the wire.
8 min
Investigations
Ollama’s MLX engine has no 32 GB rule
The requirement is not in the source. Routing is by model format, and the only Darwin memory constant is 512 MiB.
7 min
Investigations
A GGUF filename does not say what is inside
Eight builds of one model audited: a file called UD-Q4_K_XL holds no Q4_K, and none had the silent experts people warn about.
9 min