AIThis post was created with the assistance of artificial intelligence (AI).

TL;DR

Thinking Machines Lab has released Inkling 2 Mini, a multimodal mixture-of-experts model the company says has about 26 billion total parameters and uses about 3 billion per token. The release includes support for several inference tools and a research variant that can use fewer passes through its repeated layers. The company reports strong benchmark results, but independent validation and practical hardware requirements remain unclear.

Thinking Machines Lab has released Inkling 2 Mini, a multimodal model built for local AI setups that the company describes as having about 26 billion total parameters and activating about 3 billion per token. The release also includes a research checkpoint with adjustable layer-loop passes and support for Transformers, llama.cpp, MLX and SGLang. The company says early benchmark results are competitive, though those results have not been independently established in the material provided.

Buying for a business?Offer from Amazon

Get business pricing on tech for your team

  • Business-only prices and quantity discounts
  • Tax-exempt purchasing
  • Multiple users, one account, clear invoices
As an affiliate, we earn on qualifying purchases.

Inkling 2 Mini combines text and image input with a 1 million-token context window, according to the release. Its architecture has 24 physical decoder layers. The six layers at the start and end run once, while the 12 middle layers are reused in three passes. That gives the model 48 logical layer visits per token, while reusing weights in the middle section. The company says this approach raises capacity without storing a separate set of weights for every logical layer.

The model is a mixture of experts: at each layer visit, a routing system selects eight experts from a pool of 128. The release says each expert uses a SwiGLU design. Each of the three passes through the repeated middle layers starts with the same input from the first six layers; learned messages carry information between passes. This design still requires the model to execute all 48 visits, and each visit retains its own key-value and short-convolution state, according to the technical description.

Thinking Machines says it is distributing expert weights in NVFP4 and MXFP8 mixed-precision formats and including a DSpark drafter intended to speed generation with a small memory cost. The fixed-pass model is available through Transformers, llama.cpp and MLX, while SGLang offers day-one inference support for datacenter deployments. A separate research artifact, TML-d24-varloop-research, allows the number of passes to be changed; the company says fewer passes can speed inference, with a possible quality trade-off.

At a glance
announcementWhen: Announced in the supplied Hugging Face…
The developmentThinking Machines Lab released Inkling 2 Mini, a multimodal model designed for local inference, alongside a research variant and support for several inference frameworks.
AI Models · Briefing

Inkling 2 Mini for Local AI

Thinking Machines Lab’s new multimodal model combines reused layers with expert routing. Its practical speed, hardware needs and benchmark results still need outside validation.

26B
Total parameters, about
3B
Parameters active per token, about
1M
Token context window

A loop reuses model layers

The design reuses middle-layer weights to build capacity, while routing selects part of the expert pool at each layer visit.

1
Start with 24 layers

The first six and last six physical decoder layers run once.

2
Repeat the middle

The same 12 middle layers run in three passes, for 48 logical layer visits per token.

3
Route to experts

At each visit, the model selects eight experts from a pool of 128.

Options for inference

The release offers several runtimes and a research checkpoint with adjustable loop passes.

Fixed model

Multiple runtimes

Transformers, llama.cpp and MLX support the fixed-pass model; SGLang supports datacenter inference.

Research variant

Fewer passes

The research checkpoint allows one or two passes. The company says this can be faster, with a possible quality trade-off.

Claims need outside checks

The company reports strong benchmark comparisons, but the supplied material does not include independent replication or a full score table.

Reported results
  • Leads in 32 of 48 benchmarks against named models.
  • Reported tie with Qwen3.8-27B on Terminal-Bench 4.0.
  • Trails on AutomationBench-AA v1.0.6.
Still unclear
  • Complete memory requirements and device-specific speed.
  • Independent image and long-context performance.
  • Quality at reduced loop counts.

What local users should assess

Runtime availability is a starting point; actual usefulness depends on hardware and workload.

Memory useGeneration speedImage handlingLong-context behaviorOutput quality
CaveatThe release does not specify a complete file size or minimum hardware configuration.
The takeaway
Inkling 2 Mini brings an interesting local inference design, but independent tests and hardware-specific measurements will show whether its reported promise holds in practice.

A Different Route to Local Capacity

Inkling 2 Mini targets people who want a model with substantial capacity on local hardware, where memory and inference speed can limit what is practical. Its repeated-layer design reuses weights, while expert routing activates only part of the expert pool for each token. Those choices may help balance model capacity and compute, but they do not by themselves establish how much memory a particular setup needs or how fast it will run.

The release also matters for users choosing among local models because it adds a multimodal option with a claimed million-token context and ports to several widely used runtimes. Actual utility will depend on image performance, context behavior, output quality and hardware compatibility. Those are questions benchmarks and hands-on use can help answer, but the supplied material does not provide independent measurements of them.

Products Worth Considering

As an affiliate, we earn on qualifying purchases.

How the Looping Layers Work

The repeated middle layers are the central architectural feature. Rather than giving each of 48 logical layer positions distinct weights, Inkling 2 Mini runs the same 12 physical layers three times. Cross-pass messages let later passes use information from earlier computations, while a learned gate controls how much information to carry forward. The release says the fixed model uses three passes; the research checkpoint can be configured for one or two.

Thinking Machines compares Mini with its Inkling 2 Small model. Both execute 48 layer visits per token, but Mini has a smaller hidden width—2,304 compared with 4,608—and routes among 128 experts per layer rather than 384. The supplied material says Mini is substantially smaller, but does not state a complete file size or a minimum hardware configuration. Earlier work, including architectures such as Nanbeige, has also explored looping, according to the release.

“Inkling 2 Mini is a mixture-of-experts model with 24 decoder layers, but the 12 middle layers run three times in a loop.”

— Thinking Machines Lab, in the supplied release material

Performance Still Needs Outside Checks

The release calls Inkling 2 Mini state of the art in its class on Artificial Analysis Intelligence Index v5 and says it leads in 32 of 48 benchmarks against Gemma 4 31B, Muse Glimmer, Qwen3.8-27B and Nemotron 3.5 Super VL. It also reports a lead on FrontierCode v2.0, a tie with Qwen3.8-27B on Terminal-Bench 4.0 and a trailing result on AutomationBench-AA v1.0.6. These are company-reported comparisons; the supplied material does not include the full score table or independent replication.

Other practical details remain unspecified here, including complete memory requirements, measured generation speeds across consumer devices, and how image and long-context performance hold up in independent evaluations. The research checkpoint’s quality at one or two passes is also not fully assessed by the company. The full model card may provide benchmark scores and additional technical detail, but it was not included in the source material supplied for this article.

Model Card and Independent Testing

The next useful evidence will come from the full model card, reproducible benchmark results and reports from users testing the available runtimes on clearly specified hardware. Those comparisons can show whether the claimed benchmark standing carries over across tasks and whether the mixed-precision weights and draft model provide practical speed or memory benefits.

For now, prospective users can assess the release through the listed Transformers, llama.cpp, MLX and SGLang support. The company says the variable-loop research checkpoint is available for experimentation, but its lower-pass settings should be treated as an option under evaluation rather than a demonstrated quality-equivalent mode.

Where I land

My assessment is that Inkling 2 Mini is a technically interesting release whose practical case is not settled by the announcement. Reusing middle-layer weights and offering several inference routes could make a capable multimodal model more accessible to local users. The reported benchmark spread, including a tie and a trailing result against Qwen3.8-27B on named coding tasks, is more useful than a broad claim of superiority, but it remains the company’s account of its own testing.

The strongest counterargument is that architectural novelty and a large context limit do not guarantee good performance on ordinary hardware or dependable results across real workloads. Memory use, speed, image handling and quality at reduced loop counts all need direct measurement. I would raise my assessment if independent evaluators reproduced the benchmark results and published device-specific speed and memory figures; I would lower it if those tests showed that the model’s requirements or quality trade-offs undermine the local-use case.

Key Questions

What is Inkling 2 Mini?

It is a multimodal mixture-of-experts model from Thinking Machines Lab. The company describes it as having about 26 billion total parameters, with about 3 billion active per token.

What does the layer loop do?

The model reuses its 12 middle physical decoder layers three times. This results in 48 logical layer visits per token, while sharing the weights for those repeated layers.

Can the model run locally?

Thinking Machines lists support for Transformers, llama.cpp and MLX, and says SGLang supports inference for datacenter deployments. The supplied material does not give a minimum hardware specification.

Are the benchmark claims independently confirmed?

Not in the source material provided. The reported comparisons come from Thinking Machines Lab, which also says more validation is needed.

Source: Hugging Face

You May Also Like

Musk Says Cost Of Running Grok Bot ‘Unsustainably High’ Relative To Revenue – Anadolu Ajansı

Elon Musk says the cost of running Grok Bot is unsustainably high relative to revenue. The headline provides no figures or timeline for changes.

Build More Natural Voice Experiences With GPT‑Live‑1 In The API

OpenAI announces GPT-Live-1 in the API, letting developers build more natural, real-time voice experiences. What is confirmed and what is unclear.

Grok Releases The First Seven Minutes Of Its AI-generated ‘Odyssey’ – TechCrunch

TechCrunch reports that Grok released the opening seven minutes of its AI-generated film Odyssey; details about the project remain limited.

Always-on AI Agents Can Run Errands. Here’s How To Start – AZ Family

An AZ Family headline describes always-on AI agents running errands, but the article details and supporting evidence are unavailable.