SageMaker AI GPU partitioning for real-time endpoints
SageMaker real-time endpoints can split one NVIDIA GPU into up to seven isolated MIG partitions shared by several models.
Overview
AWS posted on October 30, 2026 that Amazon SageMaker AI real-time endpoints support GPU partitioning with NVIDIA Multi-Instance GPU (MIG), dividing one GPU into as many as seven isolated partitions, each with its own memory and compute, so multiple models share one accelerator. It applies only to endpoints hosting inference components; models are still invoked by component name, and the partition layout is chosen when the endpoint is created. It is available on P4d, P5, G7 and G7e instances in seven regions (N. Virginia, Ohio, Oregon, Singapore, Tokyo, Frankfurt, Ireland). Docs note that partitioned endpoints do not publish GPU, CPU or memory utilization metrics, and models on one instance still share host CPU, memory, storage and networking.
This is a sourced directory entry. Features and capabilities are reported in the linked sources; this entry does not document independent Aymorphic testing.
Key features
- Up to seven hardware-isolated MIG partitions per GPU
- Inference-component endpoints only; no new API operations or IAM actions
- Instances: P4d, P5, G7, G7e
- Regions: us-east-1, us-east-2, us-west-2, ap-southeast-1, ap-northeast-1, eu-central-1, eu-west-1
- Layout fixed at endpoint creation
Use cases
- Hosting several small or mid-size models on one GPU instead of one instance each
- Stopping a busy model from consuming memory or throughput of neighbors
Pricing
unknown; billed at the underlying instance rate, no separate partitioning fee stated in the announcement.
Pricing reflects the source record and may change. Confirm current terms with the provider.
Dates and sources
- Released
- 30 Oct 2026
- Record updated
- 8 Nov 2026
- Record last checked
- 8 Nov 2026