Open-source checkpoint and restore system cuts cold-start times by 8x to 36x across three inference engines.

NVIDIA published Snapshot, an open-source checkpoint and restore component of its DSX AI Factory Platform that targets GPU workloads on Kubernetes. The system captures a fully initialized workload, including CPU and GPU memory, and restores that state on compatible infrastructure so inference capacity comes online in seconds rather than minutes.

Engineers originally built Snapshot for NVIDIA Dynamo inference workers, and the release extends checkpoint and restore operations to other inference engines, schedulers, controllers, and platforms. Benchmarks on NVIDIA B200 GPUs with VAST NFS storage measured restores across three inference engines, with GPU state restore taking 18 to 29 seconds and CRIU-based CPU process restore taking 4.2 to 10.8 seconds.

Restoring from a Snapshot artifact brings replicas online in 3.6 to 42.7 seconds, an 8x to 36x speedup over cold starts that run for minutes. Snapshot handles multi-GPU workloads through tensor parallelism, data and expert parallelism, and decode context parallelism for large mixture-of-experts models, with checkpoints reaching 1 TB across eight GPUs.

Checkpoint creation alone takes 15 seconds to 5.5 minutes depending on model size, and combined checkpoint plus restore time beat a single cold start in every tested configuration. NVIDIA publishes installation guides in the Snapshot repository and worker restore steps in the Dynamo user guide. Infrastructure provisioning and container image loading fall outside the measured restore path.

What matters

• NVIDIA open-sourced Snapshot, a Kubernetes tool that checkpoints and restores full CPU and GPU memory state.
• Restores finish in 3.6 to 42.7 seconds, giving operators 8x to 36x faster recovery than cold starts.
• Watch whether other inference engines adopt the checkpoint and restore operations Snapshot exposes.

Why it matters

Watch whether other inference engines adopt the checkpoint and restore operations Snapshot exposes.

This GenAI News article was prepared in original wording using reporting and materials published by NVIDIA Developer Blog. Source reference: https://developer.nvidia.com/blog/start-and-restore-gpu-workloads-in-seconds-with-nvidia-snapshot/.

Drafted by the GenAI News review pipeline.