Salesforce uses the new SchedulingConfig parameter to meet two-AZ compliance and cut GPU costs by 8x.
AWS added a SchedulingConfig parameter to the CreateInferenceComponent API, giving customers explicit control over how inference components distribute across instances and Availability Zones. Salesforce adopted the feature to make Agentforce highly available across two AZs while maintaining the 8x GPU cost savings from shared inference components.
Salesforce faced an availability gap in the default SageMaker placement algorithm, which optimizes each deployment in isolation without balancing models across AZs. The new parameter introduces AvailabilityZoneBalance and PlacementStrategy sub-options, where SPREAD isolates fault domains and BINPACK improves utilization.
This change lets operators keep cost-efficient co-hosted models and still satisfy strict resilience bars. Teams running multi-AZ endpoints should pair IC placement with instance-level spread to prevent a single failure from taking down an entire model. The compliance win also makes SageMaker viable for more regulated workloads.
Salesforce has demonstrated a pattern for combining cost efficiency with high availability, and other enterprises can apply the same API controls to their own deployments. The SchedulingConfig parameter is now the reference point for aligning inference placement with resilience requirements. Teams should validate their failover plans across AZs before relying on the new defaults.
What matters
- AWS added SchedulingConfig to the inference component API for cross-AZ placement.
- Builders can now enforce two-AZ fault isolation for shared GPU inference endpoints.
- Watch for IC placement controls to expand across other SageMaker endpoint types.
Why it matters
Watch for IC placement controls to expand across other SageMaker endpoint types.
This GenAI News article was prepared in original wording using reporting and materials published by AWS Machine Learning Blog. Source reference: https://aws.amazon.com/blogs/machine-learning/spreading-the-load-how-salesforce-met-multi-az-ha-with-sagemaker-inference-components/.
Drafted by the GenAI News review pipeline.