评估和测试多租户代理 [重复].pdf

1、 2025,Amazon Web Services,Inc.or its affiliates.All rights reserved.2025,Amazon Web Services,Inc.or its affiliates.All rights reserved.S A S 3 0 8-REvaluating and testing multi-tenant agentsChris DorringtonPrincipal Cloud ArchitectAmazon Web ServicesDaniel WirjoSenior Solutions ArchitectAmazon Web S

2、ervices 2025,Amazon Web Services,Inc.or its affiliates.All rights reserved.AgendaIntroductionExample agent with Bedrock AgentCoreGenAIOps lifecycleResources and Key Takeaways 2025,Amazon Web Services,Inc.or its affiliates.All rights reserved.2025,Amazon Web Services,Inc.or its affiliates.All rights

3、reserved.Example Scenario:Customer support agent with tiered access to tools 2025,Amazon Web Services,Inc.or its affiliates.All rights reserved.Golf Store UserPremium TierAWS CloudHardware Store UserBasic TierAgentCore RuntimeCustomer SupportAgentTool 1:get_product_info()Tool 2:get_return_policy()To

4、ol 4:process_return()Amazon BedrockAgentCore GatewayAmazon CognitoLambda TargetsTool 3:cancel_order()Tool 3:get_current_orders()Tool 5:log_feedback()Tenant PromptAgentCore IdentityAuthenticateAgentCore MemoryTaskDeveloperOffline evaluationsTenant-aware Golden DatasetIsolationTestingEnd-to-EndMetrics

5、Component MetricsAgentCore ObservabilityMonitor tenant behaviorCapture user feedbackHuman annotationOnline batch evaluationsTenant Telemetry AttributesObservability and Online Evals 2025,Amazon Web Services,Inc.or its affiliates.All rights reserved.SaaS GenAIOps LifecycleDesignPre-ProductionProducti

6、onDesign agentic system for tenant-awareness from identity to observabilityDesign for tenant isolation upfrontUse Behaviour Driven Design(BDD)to identify tenant/tier behaviourEvaluate agent accuracy with golden dataset Test for security and tenant isolation Monitor tenant behavior including token us