Unofficial guide
Run Inkling 2 Mini on Mac
Last updated
Thinking Machines supports Inkling 2 Mini on Apple silicon with 32 GB or more of unified memory: MacBook, Mac mini, and Mac Studio. The runtimes they name for that hardware are Ollama and Exo Labs. MLX and llama.cpp are documented by Hugging Face and Unsloth on top of that.
This guide is unofficial. The 48 GB laptop in the announcement is a demo, not the minimum.
What 32 GB is being asked to hold
The official checkpoint is about 20 GB in NVFP4 and MXFP8. Ollama's library build is listed at 20 GB. A 32 GB Mac has to hold that file, the runtime, the KV cache for whatever context you set, and the rest of macOS. The model card does not publish a measured free-memory number beyond "32 GB or more". Unsloth's note on their memory table is the practical one: longer contexts, image inputs, and other apps use more memory.
If the Mac has 16 GB or 24 GB, the official checkpoint is outside the supported line. Unsloth's GGUF table starts lower. Their 4-bit range is 17–18 GB of RAM plus VRAM, or unified memory, and their 8-bit "lossless" column is 24 GB. That table, and the command, are on the GGUF page. The page also says the model can be run on 16 GB of RAM, or 24 GB for full precision. The table is the more specific of the two sentences. Leave slack for context either way.
Path 1: Ollama
Install a current Ollama, then run the library command. The model card lists Ollama for Apple silicon, and the library page tells you the Mac needs 32 GB or more.
ollama run inkling-2-mini
The NVFP4 tag is marked MLX on Ollama's tags page: ollama run inkling-2-mini:26b-a5b-nvfp4. On the day this was checked it was the same 20 GB digest as latest. Details, the HTTP API, and the agent launch commands are on the Ollama page.
The model card also names Exo Labs for Apple silicon and does not include a command. This guide stops there rather than guessing an Exo invocation.
Path 2: MLX on the original checkpoint
The Hugging Face release post says they upstreamed MLX support, including images, and that MLX can run the original checkpoint in native NVFP4 and MXFP8 without reconverting it. Their CLI example, including the control tokens in the prompt, is:
mlx_lm generate --model thinkingmachines/Inkling-2-Mini --image image.jpg --prompt "Describe this image.<|end_message|><|message_user|><|content_image|><|unused_200054|>" -m 1000
Copy the prompt suffix as printed if you use that command. The post presents those tokens as part of the invocation. For a text-only try, the community 4-bit card shows a Python generate call without an image. The blog does not print a separate pip install line for this original-checkpoint command. The 4-bit community card does, and it pins a branch:
pip install -U git+https://github.com/ml-explore/mlx-lm.git@refs/pull/2086/head
That card says the conversion was tested with pull request 2086. It keeps expert weights in the original NVFP4 for layers 0–19 and MXFP8 for layers 20–23, puts attention, embeddings, and the output head in 4-bit, keeps the vision tower in bfloat16, and leaves the DSpark draft out. Image input, it says, goes through mlx_lm.server with images as image_url parts.
from mlx_lm import load, generate
model, tokenizer = load("mlx-community/Inkling-2-Mini-4bit")
prompt = "hello"
if tokenizer.chat_template is not None:
messages = [{"role": "user", "content": prompt}]
prompt = tokenizer.apply_chat_template(
messages, add_generation_prompt=True, return_dict=False,
)
response = generate(model, tokenizer, prompt=prompt, verbose=True) A second community file, Inkling-2-Mini-oQ4e-mtp, includes the draft and says it needs oMLX 0.8.1 or later. It is a community quant. The storage tables on that card are the source for its size, not a Thinking Machines measurement.
Path 3: llama.cpp on Apple Silicon
Use this when you want a GGUF quant, or when you want the Unsloth server with a vision projector and a drafter. Unsloth's Apple install, which unpacks a build they say supports the looped-layer layout and per-expert NVFP4 scales:
curl -L -o llama.tar.gz https://github.com/unslothai/llama.cpp/releases/download/b12588-mix-c0c3de8/llama-b12588-mix-c0c3de8-bin-macos-arm64.tar.gz
mkdir -p llama.cpp
tar -xzf llama.tar.gz -C llama.cpp --strip-components=1
./llama.cpp/llama-server --version
The server command, context flags, and the warning about mixing this build with upstream llama.cpp are on the GGUF page. Unsloth also ships a desktop app and documents brew install --cask unsloth plus unsloth run --model unsloth/Inkling-2-Mini-GGUF:UD-Q4_K_XL.
Intel Macs
The supported-hardware list is Apple silicon. Unsloth's llama.cpp section says that for Intel Macs you pick the matching file from their release page. That is a GGUF-on-CPU possibility they acknowledge, not a promise that the 20 GB NVFP4 Ollama or MLX build runs on Intel. Expect the official checkpoint's documented path to be Apple silicon only.
Images on a Mac
The model accepts images. For a useful result the model card wants each side between 40 and 4096 pixels. In llama.cpp, vision needs the --mmproj file. In the MLX blog command, vision is the --image flag plus the prompt they printed. Ollama's library page marks the model as image input and does not show a flag, so attach images the way your Ollama version documents for vision models, and confirm the tag you pulled is the vision build.
The announcement's offline demos, a flight simulator and a small town of 20 villagers, were recorded on a MacBook Pro (M5 Pro, 48 GB) in airplane mode. They show that the lab ran the model with no network. They are not a benchmark you can quote as frames per second or tokens per second. No such rate is printed next to the demo.
Which path to use
- 32 GB or more, and you want one command: Ollama.
- 32 GB or more, and you want the original checkpoint in MLX: the Hugging Face
mlx_lm generateline. - Under 32 GB, or you want to choose a quant: Unsloth GGUF, after you read the memory table.
Hardware numbers for non-Mac machines are on requirements. Load errors are on troubleshooting.
Common questions
Which Macs can run Inkling 2 Mini?
The model card supports Apple silicon devices with 32 GB or more of memory: MacBooks, Mac minis, and Studios. The Ollama library page says the same 32 GB floor. Intel Macs are not on that list.
Is 32 GB enough, or do I need the 48 GB demo machine?
32 GB is the supported minimum Thinking Machines prints. The announcement's offline flight-simulator demo used a MacBook Pro with an M5 Pro and 48 GB. That demo is not the requirements line. Context, images, and other apps still need headroom inside those 32 GB.
Should I use Ollama or MLX on a Mac?
Ollama is the one-command path and is named on the model card for Apple silicon. The Hugging Face post shows MLX loading the original NVFP4 and MXFP8 checkpoint, including an image. llama.cpp is the third path, via Unsloth's Apple Silicon build, and is the one that uses GGUF quants when 32 GB is tight.
Sources
- Inkling 2 Mini model card Hardware, license, architecture, evaluations, and limitations.
- Thinking Machines Lab, “Introducing Inkling 2 Mini” Announcement dated 20 November 2026.
- Ollama library page for inkling-2-mini
- Hugging Face blog, “Welcome Inkling 2 Mini” Published 20 November 2026. Transformers, llama serve, SGLang, and MLX commands.
- mlx-community/Inkling-2-Mini-4bit Community MLX build. Install line points at mlx-lm pull request 2086.
- mlx-community/Inkling-2-Mini-oQ4e-mtp Community oMLX quant. Requires oMLX 0.8.1 or later.
- Unsloth, Inkling 2 Mini run guide GGUF sizes, memory ranges, sampler settings, and llama.cpp commands.
- Inkling 2 Mini config.json Hidden size, layer count, loop bounds, experts, and context length.