TL;DR

Buying for a business?Offer from Amazon

Get business pricing on tech for your team

• Business-only prices and quantity discounts
• Tax-exempt purchasing
• Multiple users, one account, clear invoices

Create a free Business account →Free to register

As an affiliate, we earn on qualifying purchases.Hide

Hugging Face has introduced a public Space that shows whether selected Transformers models run on AWS Trainium2 and provides scripts to reproduce the tests. Its PASS results cover short generations and basic output checks; they do not establish throughput or performance for longer runs.

Hugging Face has launched a model support Space that lets users check whether selected Transformers models run on AWS Trainium2, Amazon’s AI accelerator. The tool reports test outcomes and provides the script and software versions used, giving developers a way to inspect compatibility before trying a model on Trainium.

The Space presents a curated selection of popular Hub models, chosen using downloads and trending scores. Users can also search the Hub’s Trending or Downloads rankings by task and model name. Those rankings include models that have not been tested. The interface distinguishes models that were intentionally left out, usually because they were not prioritized for testing, from cases marked as incompatible.

For each tested model, the Space lists the result, Trainium core count, wall time including compilation, and peak device memory, complementing system-level profiling on AWS Trainium. Users can select the versions of the Neuron compiler, neuronx-cc, and Transformers associated with a result. A model page also lists the tested Neuron software stack, additional packages and a run.py script that can be used to reproduce the run.

Hugging Face says the script loads a model with Transformers’ from_pretrained, shards it when needed, and generates using a static cache and torch.compile with the Neuron backend, an approach related to running Hugging Face models on AWS Trainium. On a Trainium2 instance, users install the listed packages and run the script with Python. For sharded models, the script starts its own distributed ranks, so users do not need to invoke torchrun separately. The Space also provides a command to copy compiled graphs from a public cache, avoiding compilation on a first run.

A Practical Trainium Compatibility Check

The lookup addresses a practical hurdle for teams considering Trainium: support can depend on a model, software version and execution setup, so broad claims about framework compatibility do not establish that a specific checkpoint will run. The per-model results and runnable scripts make it easier to identify a tested configuration and reproduce it.

The dataset also gives developers a record they can query outside the interface. Hugging Face publishes it as aws-neuron/torch-neuronx-model-support, with Parquet files organized by compiler and library versions. Each row includes the model result, failure error where applicable, stack versions and script. That detail may help developers distinguish a model compatibility issue from a change in the software stack.

The findings have a defined scope. A PASS indicates that the tested model completed a short run and passed the stated output check; it does not show how quickly the model serves users, how much it costs to run, or how it performs at production scale. Those measures matter to infrastructure decisions, so the Space is a starting point for evaluation rather than a benchmark ranking.

How Hugging Face Tests Models

The Space builds on Hugging Face’s earlier introduction of TorchNeuron, its name for using the native PyTorch backend for AWS Trainium through the torch-neuronx package. In that setup, Transformers uses its modeling code on Trainium. The support Space takes the next step by documenting which of a limited set of models Hugging Face has tried and how to repeat those runs.

Tests run on a Trainium2, or trn2, instance. Each model is assigned the smallest number of Trainium cores that can hold its weights, up to four Trainium2 devices, or 16 cores. Most models use tensor parallelism. Mixture-of-experts models without a tensor-parallel plan use expert parallelism alone.

The test generates 32 tokens. On one core, the output tokens are compared with a CPU run of the same checkpoint. For sharded models, a small judge model checks whether the generated text is fluent English. At announcement time, tests cover Transformers models for text generation and image-text-to-text tasks. Hugging Face says it plans to add Diffusers, sentence-transformer models and other Transformers tasks.

““Which ones actually do?””

— Hugging Face

What a PASS Does Not Show

The announcement does not give a complete count of tested models or a summary of how many passed, failed or were excluded. It also does not provide throughput measurements, long-generation results, cost comparisons or production workload data. A brief compatibility run cannot answer those questions.

Coverage is limited to models that fit across at most 16 Trainium2 cores, and the published checks differ by configuration: single-core outputs are compared with CPU output, while sharded runs receive a fluency check from a small judge model. The announcement does not state that every run has the same validation method. Users should inspect the dataset card and the row for the exact model and versions before drawing conclusions.

The results are tied to the listed versions of Transformers and neuronx-cc. The announcement does not establish whether a model will keep working after those versions change, or whether untested models are incompatible. In many cases, Hugging Face says, a model may simply not have been prioritized.

More Tasks and Models Ahead

Hugging Face says it plans to broaden testing to Diffusers, sentence-transformer models and additional Transformers tasks. No release schedule or target coverage was given. For now, developers can search the Space, inspect the listed stack and script, and query the public dataset for a model’s result across available version pairs.

Anyone reproducing a result will need access to a compatible Trainium2 instance and the software stack named in that result. A first run compiles model graphs; users can use the supplied command to retrieve cached graphs built with the corresponding compiler version. The next useful evidence would include wider model coverage and measurements of throughput and longer generations, alongside the compatibility checks already published.

Source: Hugging Face

Where I land

I see the Space as a useful compatibility record because it pairs model-level results with version details and scripts that developers can inspect. That is more actionable than a general statement that a framework supports an accelerator, especially when teams need to identify whether a particular model and stack combination runs.

The strongest counterargument is that a short test with a limited set of models does not tell infrastructure teams whether Trainium will meet their performance or cost needs. I would give the results more weight in deployment decisions if the dataset added broader coverage, repeatable throughput measurements and longer-generation tests across named software versions. Until then, I read PASS as evidence of a bounded compatibility check, not proof of production readiness.

Key Questions

What does the Trainium support Space show?

It reports whether a tested model ran on Trainium2, along with its core count, wall time including compilation, peak device memory, software versions and a script for reproducing the run.

Does PASS mean a model is fast on Trainium?

No. The test generates 32 tokens and checks the output. Hugging Face says a PASS does not report throughput or performance on longer generations.

Are all models on the Hugging Face Hub tested?

No. The current selection is limited to Transformers models for text generation and image-text-to-text tasks that fit on up to four Trainium2 devices. Ranking views may show models that have not been tested.

Can developers run the tests themselves?

The Space provides a run.py script, package details and the tested software stack. Reproducing a test requires a compatible Trainium2 instance; the script handles distributed ranks for sharded models.

Source: Hugging Face