利用NeuraScale高基数交换机实现AI扩展.pdf

1、Saurabh Gayen,Chief Solutions ArchitectBaya SystemsAI Scaling with NeuraScale High-Radix SwitchesNeed for AI scale:Data Center example3Scale-Up(vertical scaling)Increases server performance for larger workloads Supports larger modelsScale-outXPUXPUXPUXPUXPUXPUXPUXPUUltra EthernetUALinkUltra Ethernet

2、Scale-Out NetworkXPUXPUXPUXPUXPUXPUXPUXPUUltra EthernetUALinkServerServerScale-upScale-Out(horizontal scaling)Increases the number of servers to process more workloads in parallel Does not improve model performanceScale-Out is Being OverusedScale Out Fabric Scale-Out is used a lot,but it mostly only

3、 helps with model-level parallelism.Scale-Out does not help with supporting larger AI models.This is what is slowing AI model improvement!More Scale-Up is needed for large AI ModelsSwitchSwitchSwitchSwitchGPUGPUGPUGPUGPUGPUGPUGPUSwitchSwitchSwitchSwitchSwitchSwitchGPUGPUGPUGPUGPUGPUGPUGPUSwitchSwitc

4、hExample:Nvidia DGX-2 supporting 16 GPUs Scale-Up uses a fully non-blocking network with low-latency links and switches.The entire Pod acts as one large GPU using tensor-level parallelism.Next generation needs larger pods.Higher Scale-Up demands high-radix switches.High-Radix Switches enable higher

5、Scale-UpTodays systems:72-Port Switches(S)72-GPU Scale Up SystemsNext gen requires:144-Port Switches(S+)144-GPU Scale Up SystemsNext gen+1 requires:288-Port Switches(S+)288-GPU Scale Up SystemsNVL72 Rack72 GPUs18 NVSwitchesNVL36NVL36NVL36NVL36NVL72NVL72NVL72NVL72NVL72NVL72NVL36NVL36NVL36NVL36NVL36NV

6、L36NVL36NVL36Liquid-Cooled SystemsAir-Cooled SystemsNVL36 Rack36 GPUs18 NVSwitchesNVL36 Rack36 GPUs18 NVSwitchesSS+S+S+S+SSS+S+S+S+S+S+S+S+S+S+S+S+S+S+NeuraScale enables High-Radix SwitchesSwitchSwitchSwitchSwitchGPUGPUGPUGPUGPUGPUGPUGPUSwitchSwitchSwitchSwitchSwitchSwitchGPUGPUGPUGPUGPUGPUGPUGPUSwi