TorchNeuron
AWS's Apache 2.0 native PyTorch backend for Trainium2 and Trainium3: .to('neuron'), eager, torch.compile, and NKI kernels.
Overview
TorchNeuron is an open-source native PyTorch backend for AWS Trainium, going live as a public beta on October 20, 2026 at PyTorch Conference North America under Apache 2.0. Trainium registers as a standard PyTorch device through the PrivateUse1 extension point, so code runs with model.to('neuron') in eager mode (plus an experimental adaptive eager mode) or torch.compile(backend='neuron') on Trainium2 and Trainium3. It supports DDP, FSDP2, tensor and pipeline parallelism, DTensor, torch.profiler with Neuron Explorer, and custom kernels through the Neuron Kernel Interface. Deepgram reports about 60% more input audio seconds per dollar than its GPU baselines.
This is a sourced directory entry. Features and capabilities are reported in the linked sources; this entry does not document independent Aymorphic testing.
Key features
- Standard device switch: .to('neuron'); torch.compile backend 'neuron'
- Eager mode plus experimental adaptive eager mode that reduces launch cost and fuses ops
- DDP, FSDP2, SimpleFSDP, tensor and pipeline parallelism, DTensor/DeviceMesh, collectives over EFA
- Custom kernels via NKI and the open-source NKI-lib (FlashAttention, MoE blockwise kernels)
- Hugging Face Transformers v5, Diffusers, TRL, TorchTitan, and vLLM Neuron plugins
- FP8 via TorchAO is in private beta; FP16, BF16, FP32 supported
Use cases
- Training or serving PyTorch and Hugging Face models on Trainium without Neuron-specific rewrites
- Cost-optimized audio, video, and LLM inference on AWS silicon
Pricing
Software is free and open source (Apache 2.0); Trainium instance usage is billed through AWS. Public beta; install with pip install torch-neuronx.
Pricing reflects the source record and may change. Confirm current terms with the provider.
Dates and sources
- Released
- 20 Oct 2026
- Record updated
- 23 Oct 2026
- Record last checked
- 23 Oct 2026