Field guide Inkling 2 Mini

Checked

Unofficial guide

Inkling 2 Mini download

Last updated

The official Inkling 2 Mini download is one open-weights checkpoint on Hugging Face, repository thinkingmachines/Inkling-2-Mini. The model card says that file is about 20 GB, in mixed NVFP4 and MXFP8, and that the license is Apache 2.0.

This page is unofficial. It lists the repositories and commands that the model card, Unsloth, and the Hugging Face release post actually publish. Smaller GGUF files are conversions, not a second official release.

Official weights

Download from huggingface.co/thinkingmachines/Inkling-2-Mini. Thinking Machines does not print a shell command on the model card. The command below uses the Hugging Face Hub CLI, which is the same hf download tool Unsloth documents for the GGUF repo, pointed at the official repository id.

Hugging Face Hub CLI
pip install -U huggingface_hub
hf download thinkingmachines/Inkling-2-Mini --local-dir Inkling-2-Mini

Leave more free disk than 20 GB. The final folder is about that size, and the download needs working space. The checkpoint is the native precision the model was trained in: the announcement says 20 layers in NVFP4 and the last four in MXFP8, so you are not applying a second quant by downloading this repo.

Try it without a download

The announcement says Inkling 2 Mini is on Tinker for inference and customization. The playground is at tinker.thinkingmachines.ai/playground. The model card points custom work at the Tinker cookbook and the tml-renderers package. That path does not put weights on your disk.

GGUF downloads

Two GGUF families are documented, and they are not interchangeable.

Unsloth's download command for the 4-bit quant, the vision projector, and the drafter:

Unsloth GGUF download
pip install -U huggingface_hub
hf download unsloth/Inkling-2-Mini-GGUF \
    Inkling-2-Mini-UD-Q4_K_XL.gguf mmproj-F16.gguf dspark-Inkling-2-Mini-Q8_0.gguf \
    --local-dir Inkling-2-Mini-GGUF

How to serve those files, which llama.cpp build they need, and the full size table are on the GGUF and llama.cpp page. Drop mmproj-F16.gguf from the command if you will not send images. Drop the drafter if you do not want speculative decoding. Unsloth says the projector adds 639 MB and describes the drafter as 0.4 GB.

MLX downloads

The Hugging Face post says MLX can load the original thinkingmachines/Inkling-2-Mini checkpoint directly, keeping NVFP4 and MXFP8 without reconversion. You do not need a second download for that path. The Mac page has the mlx_lm generate command.

Community conversions, which are not Thinking Machines releases:

The 4-bit repo's Hugging Face header says "7B params". That is the Hub's label on the quantized files. The architecture Thinking Machines published is still 26.4 billion. Do not compare "7B" with Inkling 2 Small's 594B.

Ollama is a runtime, not a second weight license

ollama run inkling-2-mini pulls the library build. Ollama lists it at 20 GB, with a 1 million token context and text plus image input. Tags and the hardware they name are on the Ollama page. You do not also need the Hugging Face checkpoint if Ollama is the only way you will run the model.

The variable-loop checkpoint is a different download

thinkingmachines/TML-d24-varloop-research shares the architecture but was trained so the number of loop passes can change. The announcement says it was trained with one to five passes and primarily at three, and that it exists so researchers can study monitorability. The Hugging Face post says fewer passes are faster and may be lower quality. For a coding install, use thinkingmachines/Inkling-2-Mini, whose loop_config has is_varloop set to false and num_passes set to 3.

Serving the official checkpoint

The Hugging Face post loads the fixed-loop model with AutoModelForMultimodalLM. This is their example, shortened only by dropping the later tool-calling script. Reasoning effort is optional. The post says the default is high (0.9). Other named levels are none (0.0), minimal (0.1), low (0.2), medium (0.7), xhigh (0.99), and max (0.99). You can also pass a number from 0.0 to 0.99.

Transformers, from the Hugging Face blog
import torch
from transformers import AutoModelForMultimodalLM, AutoProcessor

model_id = "thinkingmachines/Inkling-2-Mini"

processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForMultimodalLM.from_pretrained(model_id)

messages = [{"role": "user", "content": [{"type": "text", "text": "Why is the sky blue?"}]}]

inputs = processor.apply_chat_template(
    messages, tokenize=True, add_generation_prompt=True, return_dict=True, return_tensors="pt",
).to(model.device)
input_len = inputs["input_ids"].shape[-1]

outputs = model.generate(**inputs, max_new_tokens=1024)
print(processor.decode(outputs[0][input_len:], skip_special_tokens=False))

For an image, the same post replaces the message content with an image part whose type is image and whose path is a local file, plus a text part. Tool calls in that post are passed through the tools argument of apply_chat_template, then read back with processor.parse_response.

SGLang, from the same post, has a hardware prerequisite they state up front: run it on SM103. The launch command they print:

SGLang, from the Hugging Face blog
export MODEL="thinkingmachines/Inkling-2-Mini"

python -m sglang.launch_server \
  --model-path $MODEL \
  --served-model-name inkling \
  --attention-backend fa4 --bf16-gemm-backend torch --dtype bfloat16 \
  --tp 1 --page-size 128 --grammar-backend xgrammar \
  --host 0.0.0.0 --port 30000

The model card also links a SGLang cookbook, a vLLM recipe, and an NVIDIA Dynamo recipe. This guide does not copy flags from those pages that were not in the sources re-read for this build. Follow those docs if you use those servers.

The post's chat example sends reasoning_effort on the JSON body. Multimodal serving adds --enable-multimodal. Speculative decoding in that post is --speculative-algorithm DSPARK when the draft weights are present.

License

Apache 2.0 is the license printed on the model card. The same card links the model acceptable use policy (the policy page says it was last updated 15 July 2026). If you build a product on the weights, that policy is part of what you agree to by downloading them. Read it. Do not treat this paragraph as a summary of every restriction.

Common questions

Where do I download Inkling 2 Mini?

The official weights are the Hugging Face repository thinkingmachines/Inkling-2-Mini. The model card describes that file as one checkpoint of about 20 GB in mixed NVFP4 and MXFP8. GGUF copies live in separate repositories from Unsloth and from ggml-org.

How big is the Inkling 2 Mini download?

The official checkpoint is about 20 GB. Unsloth's GGUF files run from 7.76 GB (UD-IQ1_S) to 22.87 GB (UD-Q8_K_XL). The vision projector they recommend, mmproj-F16.gguf, adds 639 MB, and the Q8_0 DSpark drafter is about 0.4 GB.

Can I use the Inkling 2 Mini weights commercially?

The model card lists the license as Apache 2.0 and also links Thinking Machines' model acceptable use policy. Read both before you ship a product. This guide is not legal advice.

Sources

  1. Inkling 2 Mini model card Hardware, license, architecture, evaluations, and limitations.
  2. thinkingmachines/Inkling-2-Mini on Hugging Face Official open weights.
  3. Unsloth, Inkling 2 Mini run guide GGUF sizes, memory ranges, sampler settings, and llama.cpp commands.
  4. unsloth/Inkling-2-Mini-GGUF
  5. ggml-org/Inkling-2-Mini-GGUF IQ4_NL, Q8_0, and BF16 GGUFs used by the Hugging Face llama serve example.
  6. Hugging Face blog, “Welcome Inkling 2 Mini” Published 20 November 2026. Transformers, llama serve, SGLang, and MLX commands.
  7. mlx-community/Inkling-2-Mini-4bit Community MLX build. Install line points at mlx-lm pull request 2086.
  8. mlx-community/Inkling-2-Mini-oQ4e-mtp Community oMLX quant. Requires oMLX 0.8.1 or later.
  9. thinkingmachines/TML-d24-varloop-research Research checkpoint with a variable number of loop passes. Not the default model.
  10. Tinker playground Inference in the browser. No local download.
  11. Thinking Machines model acceptable use policy
  12. Ollama library page for inkling-2-mini
  13. SGLang Inkling 2 Mini cookbook Linked from the model card.
  14. vLLM Inkling 2 Mini recipe Linked from the model card.