Unofficial guide
Inkling 2 Mini requirements
Last updated
Inkling 2 Mini's official requirements are short. The model card says the checkpoint is about 20 GB in mixed NVFP4 and MXFP8, and that this is small enough for local inference on a DGX Spark, on Apple silicon with 32 GB or more of memory, or on other GPUs where 32 GB or more of VRAM is recommended.
This page is unofficial. It separates that recommendation from Unsloth's GGUF ranges, which are real published numbers and are not the same claim.
What Thinking Machines supports
| Machine | What they wrote |
|---|---|
| Apple silicon | 32 GB or more of memory. MacBooks, Mac minis, and Studios. Runtimes named: Ollama and Exo Labs. |
| NVIDIA DGX Spark | Listed as supported. The announcement's efficiency measurement used a DGX Spark with 128 GB, at MXFP8, which is more memory than the minimum story. |
| Other GPUs | "Possible, with 32 GB or more of VRAM recommended." Runtimes named: SGLang, vLLM, Unsloth, NVIDIA Dynamo, TokenSpeed. |
| Disk for the official file | About 20 GB for the checkpoint itself. |
"About 20 GB" is the footprint of the weights. It is not a measured working set with a long context and a vision tower resident. Plan for more than the file. The announcement's offline Mac demo used 48 GB, which is above the 32 GB line and is not a second minimum.
If you quantize
Unsloth's ranges are RAM plus VRAM, or unified memory. They say to use them as a starting point, and that longer contexts, images, and other applications use more. Full quant-by-quant sizes and KL numbers are on the GGUF page.
| What you load | Memory they associate with it |
|---|---|
| 1-bit GGUF | 9 GB |
| 2-bit GGUF | 10–11 GB |
| 3-bit GGUF | 13–15 GB |
| 4-bit GGUF (recommended file is 17.63 GB) | 17–18 GB |
| 8-bit GGUF, which they call lossless (file is 22.87 GB) | 24 GB |
| Official NVFP4 / MXFP8 checkpoint | About 20 GB on disk. 32 GB+ unified memory on Apple silicon, or 32 GB+ VRAM recommended elsewhere. |
| Vision projector mmproj-F16 | Add 639 MB to the GGUF download |
| DSpark drafter, Q8_0 | About 0.4 GB, optional |
Unsloth also writes that the model can run on 16 GB of RAM, or on 24 GB for full precision. Sixteen gigabytes lines up with the 3-bit band only if you are not also holding a long context. Twenty-four gigabytes lines up with their 8-bit column. Quote the table when you are choosing a file.
Disk
- Official checkpoint: about 20 GB, plus temporary space while the Hub download finishes.
- Ollama library build: listed at 20 GB.
- Unsloth GGUFs: 7.76 GB to 22.87 GB, before the projector and the drafter.
- llama.cpp binaries are small next to the weights. Keep the build's shared libraries beside
llama-server.
Context, images, and compute
- Maximum context is 1,048,576 tokens (
model_max_lengthin config.json). Ollama advertises 1M. Budget memory for the context you will actually send. - Images: any pixel image, best when each side is between 40 and 4096 pixels.
- The middle 12 layers run three times, so a short answer still pays for 48 layer visits. The model card says minimum reasoning effort does not turn that off.
- Text out only. The model does not generate images. Vision is an input.
Software by platform
| Platform | Documented stack |
|---|---|
| Apple silicon, simplest | Ollama. See the Ollama page and the Mac page. |
| Apple silicon, original checkpoint | MLX, as in the Hugging Face post. Exo Labs is named on the card without a command. |
| NVIDIA data-center or workstation | SGLang (the blog's example asks for SM103), vLLM, NVIDIA Dynamo, TokenSpeed. Recipes are linked from the download page. |
| Consumer GPU, less than 32 GB | Unsloth GGUF plus their llama.cpp build. Not the Ollama library tag, which is a 20 GB file documented for a 32 GB Mac or a DGX Spark. |
| CPU only | Unsloth documents --n-gpu-layers 0. They do not promise an interactive speed. |
| Fine-tuning | Tinker, Miles, SkyRL, and NVIDIA NeMo AutoModel, as named on the card and the announcement. No fine-tune command is reproduced here. |
Things that are not hardware requirements
- A network connection after the weights are local. The announcement's Mac demos ran in airplane mode.
- A 48 GB Mac. That was the demo laptop.
- Inkling 2 Small's machine. Small is a different model. See the comparison.
- An account with Thinking Machines, unless you are using Tinker instead of the weights.
If a load fails inside these numbers, the usual causes are a llama.cpp build without loop support, a context window left at 1 million, or a vision projector you did not budget for. Those are covered under errors and troubleshooting.
Common questions
How much VRAM does Inkling 2 Mini need?
Thinking Machines says the checkpoint's memory footprint is about 20 GB. They support Apple silicon with 32 GB or more of memory, one NVIDIA DGX Spark, and they recommend 32 GB or more of VRAM on other GPUs. They do not publish a card-by-card table. Unsloth's GGUF ranges start lower because those files are smaller quants: about 17–18 GB for 4-bit and 24 GB for their 8-bit quant, counting RAM plus VRAM or unified memory.
Does the 1 million token context fit in that memory?
The model supports 1,048,576 tokens. That is config.json's model_max_length, and it is the context Ollama prints. It is not the amount Unsloth allocates in their example, which sets --ctx-size 32768. Longer context uses more memory than the file size. If you cannot spare it, shorten the context before you drop to a smaller quant.
What do I have to install?
It depends on the machine. Apple silicon: Ollama or Exo Labs, as named on the model card, or MLX as shown by Hugging Face. NVIDIA: SGLang, vLLM, Unsloth, NVIDIA Dynamo, or TokenSpeed. GGUF: llama.cpp, with Unsloth's loop-aware build if you use their root files. The download page links the recipes the model card links.
Sources
- Inkling 2 Mini model card Hardware, license, architecture, evaluations, and limitations.
- Thinking Machines Lab, “Introducing Inkling 2 Mini” Announcement dated 20 November 2026.
- Unsloth, Inkling 2 Mini run guide GGUF sizes, memory ranges, sampler settings, and llama.cpp commands.
- Ollama library page for inkling-2-mini
- Inkling 2 Mini config.json Hidden size, layer count, loop bounds, experts, and context length.
- Hugging Face blog, “Welcome Inkling 2 Mini” Published 20 November 2026. Transformers, llama serve, SGLang, and MLX commands.