在 SageMaker HyperPod 上使用 Ray 实现可扩展且具有弹性的分布式 AI.pdf

1、 2025,Amazon Web Services,Inc.or its affiliates.All rights reserved.2025,Amazon Web Services,Inc.or its affiliates.All rights reserved.A I M 3 6 9Scalable and resilient distributed AI with Ray on SageMaker HyperPodShreyas AdiyodiHe/himFlorian GauterHe/himMark VinciguerraHe/him 2025,Amazon Web Servic

2、es,Inc.or its affiliates.All rights reserved.2025,Amazon Web Services,Inc.or its affiliates.All rights reserved.Please use this QR Code throughout the session to participate 2025,Amazon Web Services,Inc.or its affiliates.All rights reserved.AgendaCommon distributed AI training&inference challengesWh

3、at is SageMaker HyperPodWhat is RaySageMaker HyperPod and Ray ArchitectureDemoDiscussion 2025,Amazon Web Services,Inc.or its affiliates.All rights reserved.Foundation model compute demand outpaces FM development efficiency innovations5Data source:Epoch(2025)LoRA and other parameter-efficientfine-tun

4、ing techniquesFlash Attention 2.0 foroptimized GPU memory accessMixture of Experts architecturefor efficient layer activationDisaggregated serving forhigher per-GPU throughput GRPO for streamlinedreinforcement learningMARKET ADOPTION MILESTONESOCT 20,2022MAY 5,2023NOV 21,2023JUN 8,2024DEC 25,2024JUL

5、 13,2025100 BILLION10 BILLION1 BILLION100 MILLIONDisclosure required at 100 billion petaFLOP under the Executive Order Grok-3Claude 3.7 SonnetDeepSeek-R1DeepSeek-V3Amazon Nova ProGPT-4oGrok-1Llama 2-70B 2025,Amazon Web Services,Inc.or its affiliates.All rights reserved.Unique challenges to manage ha

6、rdware resources efficiently for training at scaleFault-tolerant strategies for distributed training NetworkingCluster provisioning&management Large-scale data handling 2025,Amazon Web Services,Inc.or its affiliates.All rights reserved.Unique challenges for inference workloads at scaleResource Utili