TL;DR
Thinking Machines Lab has released Inkling 2 Mini, a multimodal mixture-of-experts model the company says has about 26 billion total parameters and uses about 3 billion per token. The release includes support for several inference tools and a research variant that can use fewer passes through its repeated layers. The company reports strong benchmark results, but independent validation and practical hardware requirements remain unclear.
Thinking Machines Lab has released Inkling 2 Mini, a multimodal model built for local AI setups that the company describes as having about 26 billion total parameters and activating about 3 billion per token. The release also includes a research checkpoint with adjustable layer-loop passes and support for Transformers, llama.cpp, MLX and SGLang. The company says early benchmark results are competitive, though those results have not been independently established in the material provided.
Get business pricing on tech for your team
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
Inkling 2 Mini combines text and image input with a 1 million-token context window, according to the release. Its architecture has 24 physical decoder layers. The six layers at the start and end run once, while the 12 middle layers are reused in three passes. That gives the model 48 logical layer visits per token, while reusing weights in the middle section. The company says this approach raises capacity without storing a separate set of weights for every logical layer.
The model is a mixture of experts: at each layer visit, a routing system selects eight experts from a pool of 128. The release says each expert uses a SwiGLU design. Each of the three passes through the repeated middle layers starts with the same input from the first six layers; learned messages carry information between passes. This design still requires the model to execute all 48 visits, and each visit retains its own key-value and short-convolution state, according to the technical description.
Thinking Machines says it is distributing expert weights in NVFP4 and MXFP8 mixed-precision formats and including a DSpark drafter intended to speed generation with a small memory cost. The fixed-pass model is available through Transformers, llama.cpp and MLX, while SGLang offers day-one inference support for datacenter deployments. A separate research artifact, TML-d24-varloop-research, allows the number of passes to be changed; the company says fewer passes can speed inference, with a possible quality trade-off.
Inkling 2 Mini for Local AI
Thinking Machines Lab’s new multimodal model combines reused layers with expert routing. Its practical speed, hardware needs and benchmark results still need outside validation.
A loop reuses model layers
The design reuses middle-layer weights to build capacity, while routing selects part of the expert pool at each layer visit.
The first six and last six physical decoder layers run once.
The same 12 middle layers run in three passes, for 48 logical layer visits per token.
At each visit, the model selects eight experts from a pool of 128.
Options for inference
The release offers several runtimes and a research checkpoint with adjustable loop passes.
Multiple runtimes
Transformers, llama.cpp and MLX support the fixed-pass model; SGLang supports datacenter inference.
Fewer passes
The research checkpoint allows one or two passes. The company says this can be faster, with a possible quality trade-off.
Claims need outside checks
The company reports strong benchmark comparisons, but the supplied material does not include independent replication or a full score table.
- Leads in 32 of 48 benchmarks against named models.
- Reported tie with Qwen3.8-27B on Terminal-Bench 4.0.
- Trails on AutomationBench-AA v1.0.6.
- Complete memory requirements and device-specific speed.
- Independent image and long-context performance.
- Quality at reduced loop counts.
What local users should assess
Runtime availability is a starting point; actual usefulness depends on hardware and workload.
A Different Route to Local Capacity
Inkling 2 Mini targets people who want a model with substantial capacity on local hardware, where memory and inference speed can limit what is practical. Its repeated-layer design reuses weights, while expert routing activates only part of the expert pool for each token. Those choices may help balance model capacity and compute, but they do not by themselves establish how much memory a particular setup needs or how fast it will run.
The release also matters for users choosing among local models because it adds a multimodal option with a claimed million-token context and ports to several widely used runtimes. Actual utility will depend on image performance, context behavior, output quality and hardware compatibility. Those are questions benchmarks and hands-on use can help answer, but the supplied material does not provide independent measurements of them.
Products Worth Considering

Apple AirPods Pro

LG UltraGear 34G600A-B 34-Inch Curved WQHD Gaming Monitor

Samsung 32 Odyssey G5 QHD Gaming Monitor
As an affiliate, we earn on qualifying purchases.
How the Looping Layers Work
The repeated middle layers are the central architectural feature. Rather than giving each of 48 logical layer positions distinct weights, Inkling 2 Mini runs the same 12 physical layers three times. Cross-pass messages let later passes use information from earlier computations, while a learned gate controls how much information to carry forward. The release says the fixed model uses three passes; the research checkpoint can be configured for one or two.
Thinking Machines compares Mini with its Inkling 2 Small model. Both execute 48 layer visits per token, but Mini has a smaller hidden width—2,304 compared with 4,608—and routes among 128 experts per layer rather than 384. The supplied material says Mini is substantially smaller, but does not state a complete file size or a minimum hardware configuration. Earlier work, including architectures such as Nanbeige, has also explored looping, according to the release.
“Inkling 2 Mini is a mixture-of-experts model with 24 decoder layers, but the 12 middle layers run three times in a loop.”
— Thinking Machines Lab, in the supplied release material
Performance Still Needs Outside Checks
The release calls Inkling 2 Mini state of the art in its class on Artificial Analysis Intelligence Index v5 and says it leads in 32 of 48 benchmarks against Gemma 4 31B, Muse Glimmer, Qwen3.8-27B and Nemotron 3.5 Super VL. It also reports a lead on FrontierCode v2.0, a tie with Qwen3.8-27B on Terminal-Bench 4.0 and a trailing result on AutomationBench-AA v1.0.6. These are company-reported comparisons; the supplied material does not include the full score table or independent replication.
Other practical details remain unspecified here, including complete memory requirements, measured generation speeds across consumer devices, and how image and long-context performance hold up in independent evaluations. The research checkpoint’s quality at one or two passes is also not fully assessed by the company. The full model card may provide benchmark scores and additional technical detail, but it was not included in the source material supplied for this article.
Model Card and Independent Testing
The next useful evidence will come from the full model card, reproducible benchmark results and reports from users testing the available runtimes on clearly specified hardware. Those comparisons can show whether the claimed benchmark standing carries over across tasks and whether the mixed-precision weights and draft model provide practical speed or memory benefits.
For now, prospective users can assess the release through the listed Transformers, llama.cpp, MLX and SGLang support. The company says the variable-loop research checkpoint is available for experimentation, but its lower-pass settings should be treated as an option under evaluation rather than a demonstrated quality-equivalent mode.
Where I land
My assessment is that Inkling 2 Mini is a technically interesting release whose practical case is not settled by the announcement. Reusing middle-layer weights and offering several inference routes could make a capable multimodal model more accessible to local users. The reported benchmark spread, including a tie and a trailing result against Qwen3.8-27B on named coding tasks, is more useful than a broad claim of superiority, but it remains the company’s account of its own testing.
The strongest counterargument is that architectural novelty and a large context limit do not guarantee good performance on ordinary hardware or dependable results across real workloads. Memory use, speed, image handling and quality at reduced loop counts all need direct measurement. I would raise my assessment if independent evaluators reproduced the benchmark results and published device-specific speed and memory figures; I would lower it if those tests showed that the model’s requirements or quality trade-offs undermine the local-use case.
Key Questions
What is Inkling 2 Mini?
It is a multimodal mixture-of-experts model from Thinking Machines Lab. The company describes it as having about 26 billion total parameters, with about 3 billion active per token.
What does the layer loop do?
The model reuses its 12 middle physical decoder layers three times. This results in 48 logical layer visits per token, while sharing the weights for those repeated layers.
Can the model run locally?
Thinking Machines lists support for Transformers, llama.cpp and MLX, and says SGLang supports inference for datacenter deployments. The supplied material does not give a minimum hardware specification.
Are the benchmark claims independently confirmed?
Not in the source material provided. The reported comparisons come from Thinking Machines Lab, which also says more validation is needed.
Source: Hugging Face
