1、 2025,Amazon Web Services,Inc.or its affiliates.All rights reserved.2025,Amazon Web Services,Inc.or its affiliates.All rights reserved.S A S 3 0 8-REvaluating and testing multi-tenant agentsChris DorringtonPrincipal Cloud ArchitectAmazon Web ServicesDaniel WirjoSenior Solutions ArchitectAmazon Web S
2、ervices 2025,Amazon Web Services,Inc.or its affiliates.All rights reserved.AgendaIntroductionExample agent with Bedrock AgentCoreGenAIOps lifecycleResources and Key Takeaways 2025,Amazon Web Services,Inc.or its affiliates.All rights reserved.2025,Amazon Web Services,Inc.or its affiliates.All rights
3、reserved.Example Scenario:Customer support agent with tiered access to tools 2025,Amazon Web Services,Inc.or its affiliates.All rights reserved.Golf Store UserPremium TierAWS CloudHardware Store UserBasic TierAgentCore RuntimeCustomer SupportAgentTool 1:get_product_info()Tool 2:get_return_policy()To
4、ol 4:process_return()Amazon BedrockAgentCore GatewayAmazon CognitoLambda TargetsTool 3:cancel_order()Tool 3:get_current_orders()Tool 5:log_feedback()Tenant PromptAgentCore IdentityAuthenticateAgentCore MemoryTaskDeveloperOffline evaluationsTenant-aware Golden DatasetIsolationTestingEnd-to-EndMetrics
5、Component MetricsAgentCore ObservabilityMonitor tenant behaviorCapture user feedbackHuman annotationOnline batch evaluationsTenant Telemetry AttributesObservability and Online Evals 2025,Amazon Web Services,Inc.or its affiliates.All rights reserved.SaaS GenAIOps LifecycleDesignPre-ProductionProducti
6、onDesign agentic system for tenant-awareness from identity to observabilityDesign for tenant isolation upfrontUse Behaviour Driven Design(BDD)to identify tenant/tier behaviourEvaluate agent accuracy with golden dataset Test for security and tenant isolation Monitor tenant behavior including token us