TorchTitan (with TitanRL)

PyTorch-native pre- and post-training platform; the October 2026 update adds TitanRL, Kimi K3, DeepSeek-V4 and Muse Spark Open recipes.

PyTorch · Data, training, and fine-tuning platforms (datasets, labeling, RAG/vector DBs, fine-tuning-as-a-service) · open-source

Published in Feed

Overview

TorchTitan is PyTorch's native platform for large-scale pre- and post-training; its October 21, 2026 update post adds TitanRL, which keeps one model definition from pretraining through post-training inside vLLM, with asynchronous agentic RL rollouts and a batch-invariant mode that gives bitwise-identical trainer and generator log probabilities in supported on-policy setups. New full-stack model support covers Muse Spark Open, Muse Glimmer, Qwen3-VL, Qwen3.5, Kimi K2.7, Kimi K3, and DeepSeek-V4. Reported results include DeepSeek V3 671B at 5,324 tokens/sec/GPU on 256 GB300 GPUs and a torch_checkpointing library that cut average checkpoint write time from 384.4s to 13.1s on 512 H100s.

This is a sourced directory entry. Features and capabilities are reported in the linked sources; this entry does not document independent Aymorphic testing.

Key features

  • TitanRL: RL with the same model code used in pretraining and vLLM; async rollouts for long-context agentic tasks
  • PyTorch-native 5D parallelism recipes for Muse Spark Open, Kimi K3, DeepSeek-V4, Qwen3.5 and others
  • DeepSeek V3 671B+MTP: 5,324.21 TPS/GPU on 256 GPUs with MXFP8; matches Megatron loss on the MLPerf benchmark
  • FlexShard: up to 79% faster Muon optimizer step on DeepSeekV3 at 256 H100s; up to 20% faster Llama 3 405B steps with blockwise FP8 all-gathers
  • torch_checkpointing: trainer blocking time 841.2ms to 34.1ms, write time 384.4s to 13.1s at 512 H100s
  • Integrated as a training backend in veRL and Miles

Use cases

  • Pretraining and fine-tuning new open-weight MoE architectures in native PyTorch
  • Agentic reinforcement learning on code and terminal tasks
  • Large-scale training (16k+ ranks) with low-overhead checkpointing

Pricing

Free and open source (github.com/pytorch/torchtitan); license not stated in the update post.

Pricing reflects the source record and may change. Confirm current terms with the provider.

Dates and sources

Released
Not listed
Record updated
23 Oct 2026
Record last checked
23 Oct 2026

Explore the Feed

Other entries filed under data, training, and fine-tuning platforms (datasets, labeling, rag/vector dbs, fine-tuning-as-a-service); inclusion does not imply equivalent features.

  • DigitalOcean Agent Data Services

    DigitalOcean's managed data layer for agents: ingestion flows, a catalog with vectors, and SQL or vector query, in private preview.

  • pplx-embed-v2-late

    Perplexity's late-interaction multimodal embedding models (0.6B and 9B) with 128-dim per-token vectors, open on Hugging Face.

  • EmbeddingGemma 2

    Open-weight 740M-parameter multimodal embedding model built on Gemma 4 that encodes text, images, audio, and video for on-device search.

Browse the Feed