使用 Lambda SnapStart 构建高性能推理 API.pdf

1、 2025,Amazon Web Services,Inc.or its affiliates.All rights reserved.2025,Amazon Web Services,Inc.or its affiliates.All rights reserved.C N S 3 6 8Build performant inference APIs with Lambda SnapStartHarold SunHe/HimSenior Software Engineer,AWS LambdaAyush KulkarniHe/HimSenior Product Manager,AWS Lam

2、bda 2025,Amazon Web Services,Inc.or its affiliates.All rights reserved.About us-Harold Sun-Senior Software Engineer,AWS Lambda-Area of focus:Developer Tooling-Lambda Web Adapter,Remote Debugging-Ayush Kulkarni-Senior Product Manager,AWS Lambda-Area of focus:Core compute capabilities-SnapStart,Firecr

3、acker virtualization 2025,Amazon Web Services,Inc.or its affiliates.All rights reserved.Session Agenda01 Inference with AWS Lambda A:Use cases B:Considerations02 Architectural components A:Packaging:1GB+models and libraries B:Cold-start:Rapid startup times C:Streaming responses interactively 2025,Am

4、azon Web Services,Inc.or its affiliates.All rights reserved.2025,Amazon Web Services,Inc.or its affiliates.All rights reserved.Inference with AWS Lambda 2025,Amazon Web Services,Inc.or its affiliates.All rights reserved.Todays subject:Amazon BedrockAWS LambdaWeb EndpointAWS LambdaWeb EndpointHosted

5、ML model 2025,Amazon Web Services,Inc.or its affiliates.All rights reserved.Todays subject:Amazon BedrockAWS LambdaWeb EndpointAWS LambdaWeb EndpointHosted ML model 2025,Amazon Web Services,Inc.or its affiliates.All rights reserved.Use CasesCPU-basedProprietary ML modelsMemory SizeUnpredictable traf

6、fic 2025,Amazon Web Services,Inc.or its affiliates.All rights reserved.Natural fit with AWS LambdaResponsive scalingFully-managed operationsFamiliar DevEx1000 new instances/10 secondsScale-to-zero when idleManaged runtimes Integrations with 200+AWS servicesRapid development cycleNative support for p