FlyDSL (PyTorch backend for AMD MI350)
AMD's FlyDSL kernel language is now an optional PyTorch backend on MI350-series GPUs, with up to 3.01x end-to-end vLLM gains.
Overview
FlyDSL (github.com/ROCm/FlyDSL) is a Python-embedded kernel DSL for AMD GPUs that lowers through MLIR. A PyTorch blog post dated October 21, 2026 says PyTorch users on AMD MI350-series (gfx950) GPUs can now use it through existing APIs for dense and grouped GEMM, MXFP8/MXFP4 scaled GEMM, RMSNorm and TopK, and for fused attention through AOTriton 0.14.2b or later. On MI355X, reported kernel geometric-mean speedups are 1.10x on 15 BF16 GEMM shapes versus the faster of ATen and Triton, 1.58x (MXFP8) and 1.68x (MXFP4) versus ATen, and 4.82x on small-K TopK. End-to-end vLLM whole-request speedups reach 1.13x BF16, 2.04x MXFP8 and 3.01x MXFP4, depending on model and concurrency.
This is a sourced directory entry. Features and capabilities are reported in the linked sources; this entry does not document independent Aymorphic testing.
Key features
- Optional backend: TorchInductor benchmarks FlyDSL beside ATen and Triton and caches the fastest
- Eager dispatch for eligible RMSNorm and TopK calls; unsupported inputs fall back to ATen
- FlyDSL attention kernels ship in AOTriton 0.14.2b+, also for RDNA4 (gfx1201)
- Install with pip install flydsl>=0.3.0,<0.4 on a ROCm PyTorch nightly
- Planned: MI450 support, MXFP8 grouped GEMM, FlexAttention backend
Use cases
- Speeding up MXFP4 and MXFP8 LLM inference on MI355X
- Mixture-of-experts grouped GEMM with uneven expert loads (reported 1.20x over Triton across 24 cases)
Pricing
Free and open source; license not stated in the PyTorch post.
Pricing reflects the source record and may change. Confirm current terms with the provider.
Dates and sources
- Released
- 21 Oct 2026
- Record updated
- 27 Oct 2026
- Record last checked
- 27 Oct 2026