Olmo-core 3

Ai2's open, redesigned mixture-of-experts training stack, benchmarked past one trillion total parameters at 2.7x the throughput of its prior FSDP implementation.

Ai2 · Data, training, and fine-tuning platforms (datasets, labeling, RAG/vector DBs, fine-tuning-as-a-service) · infrastructure

Published in Feed

Overview

The Allen Institute for AI (Ai2) released Olmo-core 3 on October 1, 2026, an upgrade to its open LLM development framework built around a redesigned mixture-of-experts (MoE) training system. It switches to a distributed-data-parallel approach that keeps experts resident on GPUs and routes data to them, avoiding the repeated weight-gathering of the prior FSDP-based system. Ai2 benchmarked the new stack at over one trillion total parameters, with successful scaling tests up to 2.38 trillion total parameters, roughly 2.7x the throughput of the earlier FSDP implementation, and a further 21% gain from using MXFP8 over BF16. The release ships with a technical report, an interactive demo, and open code so researchers can train their own MoEs and experiment with routing and parallelism.

This is a sourced directory entry. Features and capabilities are reported in the linked sources; this entry does not document independent Aymorphic testing.

Key features

  • Redesigned MoE training system using distributed data parallelism that keeps experts resident on GPU
  • Benchmarked at over 1 trillion total parameters, with scaling tests to 2.38 trillion total parameters
  • ~2.7x throughput improvement over the prior FSDP-based implementation
  • 21% additional throughput gain from MXFP8 over BF16
  • Open code, technical report, and interactive demo released alongside the stack

Use cases

  • Training large mixture-of-experts language models at trillion-parameter scale
  • Researching MoE routing, parallelism strategies, and hardware adaptation
  • Building the next generation of Olmo models on open infrastructure

Pricing

Free and open-source; compute cost is whatever hardware the user runs it on.

Pricing reflects the source record and may change. Confirm current terms with the provider.

Dates and sources

Released
1 Oct 2026
Record updated
3 Oct 2026
Record last checked
3 Oct 2026

Explore the Feed

Other entries filed under data, training, and fine-tuning platforms (datasets, labeling, rag/vector dbs, fine-tuning-as-a-service); inclusion does not imply equivalent features.

  • DigitalOcean Agent Data Services

    DigitalOcean's managed data layer for agents: ingestion flows, a catalog with vectors, and SQL or vector query, in private preview.

  • pplx-embed-v2-late

    Perplexity's late-interaction multimodal embedding models (0.6B and 9B) with 128-dim per-token vectors, open on Hugging Face.

  • EmbeddingGemma 2

    Open-weight 740M-parameter multimodal embedding model built on Gemma 4 that encodes text, images, audio, and video for on-device search.

Browse the Feed