llama.cpp v0.5.0

The v0.5.0 release of the widely used C/C++ LLM inference engine, adding new model support and backend performance work.

llama.cpp · Infrastructure and inference/serving · infrastructure

Published in Feed

This entry is marked for review. Check the original sources before relying on its details.

Overview

llama.cpp shipped v0.5.0, following v0.4.0 (September 4, 2026) and v0.4.1 (September 14, 2026) under the project's recently adopted semantic-versioning scheme (the project moved from continuous build-tagged releases to numbered semver releases starting August 25, 2026). The release focuses on backend performance and correctness, broader model coverage, and more robust server/router operation: CUDA conv2d acceleration via implicit GEMM, Metal MoE and SSM_CONV fusion, multi-address HTTP binding for servers, new model support (HRM-Text/DFM Mimir 1B, MiMo-V2.6 conversion, HunyuanOCR with DFlash, extended Nemotron and Qwen4 handling), an updated ggml v0.25.0 with expanded hyper-connection and flash-attention support, and function-call outputs that now support image input.

This is a sourced directory entry. Features and capabilities are reported in the linked sources; this entry does not document independent Aymorphic testing.

Key features

  • CUDA conv2d acceleration via implicit GEMM; Metal MoE and SSM_CONV fusion
  • Multi-address HTTP binding for servers (comma-separated TCP addresses and UNIX sockets)
  • New model support: HRM-Text (DFM Mimir 1B), MiMo-V2.6 conversion, HunyuanOCR with DFlash, extended Nemotron/Qwen4 handling
  • Updated ggml to v0.25.0 with expanded hyper-connection and flash-attention support
  • Function-call outputs now support image input
  • LoRA loading from open file pointers

Use cases

  • Local and server-side LLM/VLM inference in C/C++ with minimal dependencies
  • Cross-hardware deployment (CPU, Apple Silicon, CUDA, HIP, MUSA, Vulkan, SYCL)
  • Running newly supported model families (MiMo-V2.6, HunyuanOCR, Nemotron, Qwen4) locally

Pricing

Free and open source under the MIT license.

Pricing reflects the source record and may change. Confirm current terms with the provider.

Dates and sources

Released
23 Sept 2026
Record updated
25 Sept 2026
Record last checked
25 Sept 2026

Explore the Feed

Other entries filed under infrastructure and inference/serving; inclusion does not imply equivalent features.

  • NVIDIA PAIR (Personal AI Router)

    NVIDIA's free, open-source personal AI router that spreads local model workloads across PCs on your network; out of beta Oct 13, 2026.

  • Agent37

    YC F26 startup selling low-cost agent sandboxes (gVisor or full VM) with prebuilt harnesses for Claude Code, Hermes and opencode.

  • Microsoft-Decision-1

    Microsoft's Qwen3.5-9B-based decision-scoring model returns calibrated option probabilities in one pass, at $0.042 per million input tokens.

Browse the Feed