Field guide Inkling 2 Mini

Checked

Unofficial guide

Inkling 2 Mini requirements

Last updated

Inkling 2 Mini's official requirements are short. The model card says the checkpoint is about 20 GB in mixed NVFP4 and MXFP8, and that this is small enough for local inference on a DGX Spark, on Apple silicon with 32 GB or more of memory, or on other GPUs where 32 GB or more of VRAM is recommended.

This page is unofficial. It separates that recommendation from Unsloth's GGUF ranges, which are real published numbers and are not the same claim.

What Thinking Machines supports

From the model card's "Required hardware" section and the announcement's footprint sentence.
Machine What they wrote
Apple silicon 32 GB or more of memory. MacBooks, Mac minis, and Studios. Runtimes named: Ollama and Exo Labs.
NVIDIA DGX Spark Listed as supported. The announcement's efficiency measurement used a DGX Spark with 128 GB, at MXFP8, which is more memory than the minimum story.
Other GPUs "Possible, with 32 GB or more of VRAM recommended." Runtimes named: SGLang, vLLM, Unsloth, NVIDIA Dynamo, TokenSpeed.
Disk for the official file About 20 GB for the checkpoint itself.

"About 20 GB" is the footprint of the weights. It is not a measured working set with a long context and a vision tower resident. Plan for more than the file. The announcement's offline Mac demo used 48 GB, which is above the 32 GB line and is not a second minimum.

If you quantize

Unsloth's ranges are RAM plus VRAM, or unified memory. They say to use them as a starting point, and that longer contexts, images, and other applications use more. Full quant-by-quant sizes and KL numbers are on the GGUF page.

Unsloth's summary ranges, plus the official checkpoint on its own row so the two claims stay distinct.
What you load Memory they associate with it
1-bit GGUF 9 GB
2-bit GGUF 10–11 GB
3-bit GGUF 13–15 GB
4-bit GGUF (recommended file is 17.63 GB) 17–18 GB
8-bit GGUF, which they call lossless (file is 22.87 GB) 24 GB
Official NVFP4 / MXFP8 checkpoint About 20 GB on disk. 32 GB+ unified memory on Apple silicon, or 32 GB+ VRAM recommended elsewhere.
Vision projector mmproj-F16 Add 639 MB to the GGUF download
DSpark drafter, Q8_0 About 0.4 GB, optional

Unsloth also writes that the model can run on 16 GB of RAM, or on 24 GB for full precision. Sixteen gigabytes lines up with the 3-bit band only if you are not also holding a long context. Twenty-four gigabytes lines up with their 8-bit column. Quote the table when you are choosing a file.

Disk

Context, images, and compute

Software by platform

Names come from the model card unless the cell says otherwise.
Platform Documented stack
Apple silicon, simplest Ollama. See the Ollama page and the Mac page.
Apple silicon, original checkpoint MLX, as in the Hugging Face post. Exo Labs is named on the card without a command.
NVIDIA data-center or workstation SGLang (the blog's example asks for SM103), vLLM, NVIDIA Dynamo, TokenSpeed. Recipes are linked from the download page.
Consumer GPU, less than 32 GB Unsloth GGUF plus their llama.cpp build. Not the Ollama library tag, which is a 20 GB file documented for a 32 GB Mac or a DGX Spark.
CPU only Unsloth documents --n-gpu-layers 0. They do not promise an interactive speed.
Fine-tuning Tinker, Miles, SkyRL, and NVIDIA NeMo AutoModel, as named on the card and the announcement. No fine-tune command is reproduced here.

Things that are not hardware requirements

If a load fails inside these numbers, the usual causes are a llama.cpp build without loop support, a context window left at 1 million, or a vision projector you did not budget for. Those are covered under errors and troubleshooting.

Common questions

How much VRAM does Inkling 2 Mini need?

Thinking Machines says the checkpoint's memory footprint is about 20 GB. They support Apple silicon with 32 GB or more of memory, one NVIDIA DGX Spark, and they recommend 32 GB or more of VRAM on other GPUs. They do not publish a card-by-card table. Unsloth's GGUF ranges start lower because those files are smaller quants: about 17–18 GB for 4-bit and 24 GB for their 8-bit quant, counting RAM plus VRAM or unified memory.

Does the 1 million token context fit in that memory?

The model supports 1,048,576 tokens. That is config.json's model_max_length, and it is the context Ollama prints. It is not the amount Unsloth allocates in their example, which sets --ctx-size 32768. Longer context uses more memory than the file size. If you cannot spare it, shorten the context before you drop to a smaller quant.

What do I have to install?

It depends on the machine. Apple silicon: Ollama or Exo Labs, as named on the model card, or MLX as shown by Hugging Face. NVIDIA: SGLang, vLLM, Unsloth, NVIDIA Dynamo, or TokenSpeed. GGUF: llama.cpp, with Unsloth's loop-aware build if you use their root files. The download page links the recipes the model card links.

Sources

  1. Inkling 2 Mini model card Hardware, license, architecture, evaluations, and limitations.
  2. Thinking Machines Lab, “Introducing Inkling 2 Mini” Announcement dated 20 November 2026.
  3. Unsloth, Inkling 2 Mini run guide GGUF sizes, memory ranges, sampler settings, and llama.cpp commands.
  4. Ollama library page for inkling-2-mini
  5. Inkling 2 Mini config.json Hidden size, layer count, loop bounds, experts, and context length.
  6. Hugging Face blog, “Welcome Inkling 2 Mini” Published 20 November 2026. Transformers, llama serve, SGLang, and MLX commands.