MangoBoost 全栈 AI 基础设施解决方案：MLPerf 推理、训练、存储案例研究.pdf

1、MangoBoostOCP Global Summit 2025MangoBoost Full-Stack AI Infrastructure Solutions MLPerf Inference,Training,Storage Case StudiesWebsiteContactwww.mangoboost.iocontactmangoboost.ioEriko Nurvitadhi,PhD,MBAChief Product Officer&Co-Founder 2025 MangoBoost,Inc.All rights reserved.Do Not distribute withou

2、t permission.2AI Era and Key BottlenecksPART I.2025 MangoBoost,Inc.All rights reserved.Do Not distribute without permission.The AI Era Calls for Extreme HW/SW Engineeringneeds more interconnectionsGenerative AI Boom“DeepSeeks modular,distributed AI training will likely drive demand for efficient net

3、working solutions”-Source:ForbesGPU clusterGPU serverGPU serverGPU serverIntra-server networkIntra-server NetworkInter-server network(through external switches)3 2025 MangoBoost,Inc.All rights reserved.Do Not distribute without permission.4Complicated AI SW StackApplication APIs(Python,REST,OpenAI,W

4、ebSocket,etc.)AI MicroservicesDeployment(Inference Serving)Data Analytics(RAG,etc.)Fine-tuning(LoRA,etc.)TrainingInfrastructure Management(Kubernetes,admin UI,monitoring,etc.)Multi-Node Scaling&Optimized Kernels(collective communication,auto-scaling,etc.)Problem#1:Extremely Difficult Software Engine

5、ering 2025 MangoBoost,Inc.All rights reserved.Do Not distribute without permission.Problem#2:Extremely Difficult Hardware(Network)EngineeringScaling of Peak Hardware FLOPS and Interconnect BandwidthIncreasing Inter-node communications Source:AI and Memory Wall“AI applications are bottlenecked by com

6、munication across/to AI accelerators,rather than compute.”-Amir Gholami,International Computer Science Institute,UC Berkeley 5 2025 MangoBoost,Inc.All rights reserved.Do Not distribute without permission.Traditional Data Center:CPU runs heavy data center I/O tasksCPU runs heavy data center I/O tasks