幕后揭秘：金融服务领域生成式人工智能的可观测性 [重复].pdf

1、 2025,Amazon Web Services,Inc.or its affiliates.All rights reserved.2025,Amazon Web Services,Inc.or its affiliates.All rights reserved.I N D 3 4 1-RBehind the curtain:Generative AI Observability in Financial ServicesJiwon Yeom(she/her)Senior Solutions ArchitectAmazon Web ServicesKihyeon Myung(he/him

2、)Sr.Applied AI ArchitectAmazon Web Services 2025,Amazon Web Services,Inc.or its affiliates.All rights reserved.Before we even beginQuality assessmentRelevance scoring,factuality checks,toxicity detectionPass/fail decisions,quality scores,performance metricsUnderstanding and tracking overall system b

3、ehavior and stateResponse time trends,error rates,usage patterns,system resourcesDashboards,alerts,trend analysisEvaluationObservability 2025,Amazon Web Services,Inc.or its affiliates.All rights reserved.Building Investment Research AssistantMarket Research AutomationFund Performance Analytics Risk

4、Metric Computation Regulatory Impact AssessmentComplex Data and Query RequirementsSensitive data and decisionsNear Real-Time requirements:Market regime shifts require prompt adaptation monitoringHigh-Stakes Decision MakingCritical Failure Modes:Hallucination,Grounding failure,Output volatilityAnalys

5、t insights directly influence in assets under managementRegulation and ComplianceMitigate vulnerabilities prompt attacks,etc.Explainability and auditability at the heart 2025,Amazon Web Services,Inc.or its affiliates.All rights reserved.Risk ModelingUrgencySeverityPrompt injection,Jailbreak,Tool abu

6、seSystem performance degradation Lack of intelligence,minor failure patternsREAL TIMEDefense&ObservabilityNEAR REAL TIMEObservabilityBatchEvaluations 2025,Amazon Web Services,Inc.or its affiliates.All rights reserved.Key takeaway Model your