简化大规模智能体人工智能的部署.pdf

1、Tim ZhouOct 14th,2025Simplifying the Deployment of Agentic AI at ScaleCopyright 2023 Accton Technology Corporation.All rights reserved.Inference10%Training90%2023Inference90%Training10%203010%10%InferenceInference90%90%InferenceInferenceEnterprise GenAI accelerated Retooling workflows,automate cross

2、-functional processes,contextual reasoning across departments Data privacy and data sovereignty will be critical for Enterprise Hybrid training in cloud and Inference onprem Enterprise-Grade Infrastructure will be required Power real-time decision-making without latency bottlenecksSource:Jeff Clarke

3、,COO,Dell Technologies-DTW24Source:https:/ organizations piloting Gen AI75%of organizations piloting Gen AI.Only 9%are deploying.1On-prem inference demands dynamic and flexible GPU solutionsdemands dynamic and flexible GPU solutions83%of All Data is On-PremGenAI is retooling Enterprise50%of This Dat

4、a is Generated at the Edge Enterprise demands AI to deliver tangible business outcomesTAM$100BEnterprise GenAIEnterprise GenAIAgentic AI in the Enterprise:From Chatbots to Autonomous WorkflowsBeyond Q&A:multi-agent planners with tool-use,RAG,and cross-function task orchestrationDepartmental on-prem

5、inference(HR,Finance,R&D,Ops)for latency,privacy,sovereigntyLong-context LLMs(e.g.,120K-token prompts)TB-scale KV-cache;strict TTFT/tokens-per-sec SLAsMulti-tenancy:per-tenant isolation,quotas,and policy guardrails across shared GPU poolsGovernance&audit:per-tenant usage,cost,and SLO tracking to ena

6、ble chargeback/showbackDesign goal:empower business workflows while hiding infra complexity behind templates&APIsEnterprise Reality:What We See in the FieldOn-premise inference drivers(incl Jurisdiction)On-premise inference drivers(incl Jurisdiction)Multi-tenancy RequirementsMulti-tenancy Requiremen