Field guide Inkling 2 Mini

Checked

Unofficial guide

Run Inkling 2 Mini in the Cloud (No 32GB RAM Needed)

Last updated

The official Inkling 2 Mini checkpoint is about 20 GB, and Thinking Machines recommends 32 GB or more of memory on Apple silicon, or 32 GB or more of VRAM on other GPUs. If the laptop or Mac in front of you is under that line, you can rent a GPU for the hours you actually need, then stop it.

This page is unofficial. It is not a Thinking Machines service, and it does not add a benchmark. Memory numbers below are the ones already on the requirements page.

Why a rented GPU

A machine under 32 GB is not on the supported list for the native checkpoint. You still have two honest choices. One is a smaller GGUF on the hardware you already own. Unsloth's 4-bit range is 17–18 GB and their 3-bit range is 13–15 GB, with the quality cost in the GGUF table. The other is to run the same files on a rented card that does have the memory, and to turn that rental off when the session is over.

Renting does not change the model. Ollama's library build is still the 20 GB tag documented for a 32 GB Mac or a DGX Spark. llama.cpp still needs the loop-aware build if you use Unsloth's root GGUFs. The cloud machine is just where those commands run.

What to rent

Look at VRAM on the listing, not at a brand name.

Both rentals below are affiliate links. They are the only places on this site those URLs appear.

Vast.ai

Vast.aiAffiliate link is a marketplace of other people's GPUs. You rent one listing, use it, and end the rental in their console. The labels in that console change, so follow the buttons they show. The outline is the same.

  1. Open Vast.ai and sign in. Filter for a GPU with the VRAM you decided on above, and for enough disk for the file you will pull.
  2. Rent the instance. If the template list includes one whose description says Ollama or llama.cpp, that saves an install. Template names are not stable, so read the description. If none of them is that runtime, pick an ordinary Linux image with the GPU drivers and install the runtime yourself.
  3. Connect the way the instance page tells you to. That is usually an SSH command they print. This guide does not add flags they do not show.
  4. Pull a build this site already documents. On a machine with 32 GB or more of VRAM, ollama run inkling-2-mini is the library command from the Ollama page. The model card lists Ollama for Apple silicon, so treat that command on a rented NVIDIA card as a convenience, not as their named NVIDIA path. For a GGUF, use the llama.cpp steps: Unsloth's loop-aware build, then Inkling-2-Mini-UD-Q4_K_XL.gguf if you want their recommended quant. Drop --mmproj and the drafter if you are only chatting, so the download stays smaller.
  5. Use a short context. Unsloth's example is 32768 tokens, not the model's 1,048,576 maximum. A 1 million token window is the model maximum, not a memory budget.
  6. When you are finished, stop or destroy the instance in the Vast console. A running instance keeps billing. Check Vast's own billing note for whether a stopped instance still charges for disk before you leave a volume sitting there.

Novita AI

Novita AIAffiliate link offers GPU instances and an API. This guide has not confirmed an Inkling 2 Mini model id on that API, so it does not tell you to call one.

  1. For a GPU instance, pick the same VRAM class as above. A 24 GB-class card for a GGUF, or 32 GB or more if you want the official checkpoint under the model card's recommendation.
  2. Install or select Ollama or llama.cpp the same way as on Vast. Then use the commands on the Ollama and GGUF pages. Do not add a flag those pages do not already document.
  3. For the API, open their model list. If Inkling 2 Mini is there, follow their request format for that id. If it is not, use the instance. Do not guess the name.
  4. Stop the instance when the session is over. API calls bill on their terms, which are on their pricing page and are not repeated here. Prices vary.

What actually saves money

Local install paths stay where they were: download, Mac, and out-of-memory errors.

Common questions

Can I run Inkling 2 Mini in the cloud if my laptop has less than 32 GB?

Yes. The 32 GB figure is Thinking Machines' recommendation for a local Apple silicon machine, or for VRAM on other GPUs, not a rule about where the weights are allowed to run. Rent a GPU that meets the same memory guidance, run the model there, and stop the instance when you are finished.

Is a 24 GB GPU enough?

Thinking Machines recommends 32 GB or more of VRAM for the official checkpoint, which is about 20 GB on disk. Unsloth's 4-bit GGUF is a 17.63 GB file with a 17–18 GB memory range, and their 8-bit column is 24 GB. A 24 GB-class rental is a sensible fit for those GGUFs. For the native checkpoint, prefer a listing at or above the 32 GB recommendation. Read the VRAM on the listing. This site does not publish a price.

How do I stop paying?

A running instance keeps the rental going. Stop or destroy it in the provider's console when you are done. Check that provider's billing page for whether a stopped machine still charges for disk. Prices change, so the number on the listing at rental time is the one that applies.

Do Vast.ai or Novita host an official Inkling 2 Mini API?

This guide does not claim that either provider serves the model as a named API. On Vast.ai you rent a machine and run Ollama or llama.cpp yourself. On Novita, use a GPU instance the same way unless their console lists this model. If it does not, do not invent a model id.

Sources

  1. Inkling 2 Mini model card Hardware, license, architecture, evaluations, and limitations.
  2. Thinking Machines Lab, “Introducing Inkling 2 Mini” Announcement dated 20 November 2026.
  3. Unsloth, Inkling 2 Mini run guide GGUF sizes, memory ranges, sampler settings, and llama.cpp commands.
  4. Ollama library page for inkling-2-mini
  5. thinkingmachines/Inkling-2-Mini on Hugging Face Official open weights.