主题演讲 - 使用 P4 网卡实现弹性横向扩展 GPU 互连.pdf

1、Using P4 NICs for resilient scale-outGPU InterconnectParameters(Log Scale)2014201620182020202220232025AmoebaNetB557 millionResNet5026 millionBERT-large330 millionChatGPT-41.8 trillionGPT-3175 billionPaLM540 billionGPT-21.5 billion2x per yearImage and speech recognition models14x per yearLanguage+rec

2、ommender models2024Llama 4 Behemoth 2 trillion48,000 100,000 1,000,000 20 x GPU Cluster ScaleInfinibandEthernet(RoCEv2)Ethernet(UEC)The need for Scale-out AI fabricTraits of Scale-out GPU Interconnect-UECNIC:Packet Spray+Out-of-Order handlingSwitch:ECMP NICsNICsMultipathingUtilizing full bisectional

3、 bandwidthCongestion ControlHandle lossy networksLoss identificationSelective acknowledgement and retransmissionTCOMaximize compute by eliminating inefficiencies Network Switch Fabric(Non-Blocking)GPUNICGPUNICGPUNICGPUNICLink down,Link errors(optics/cable)LinkSwitch hardware failure,Switch software

4、failureSwitchNIC hardware failure,NIC software failureNICGPU hardware failure,GPU software failureGPULink down,Link errors(optics/cable)LinkThe challenge of AI fabric scale100K AI Cluster Key ComponentsComponentsQuantityCommentsGPU100KTypical GPUBack End NIC100K1:1(GPU:NIC)GPU Servers13K8 GPUs/serve

5、rNetwork Switches1.2K512x100GbE portsOptical Cables600K+2-tier designTransceivers600K+QSFPRacks1.6K64 GPUs/rack*About 78%of unexpected interruptions were attributed to confirmed or suspected hardware issues.*https:/arxiv.org/pdf/2407.21783(The Llama 3 Herd of Models)Infrastructure resiliency is not

6、optionalGPUNICGPUNICGPUNICGPUNICNetwork Switch Fabric(Non-Blocking)Switch hardware failure,Switch software failureSwitchNIC hardware failure,NIC software failureNICGPU hardware failure,GPU software failureGPUFailure ScenarioTypeWhy P4 Modular System ArchitectureAny Ethernet Network SwitchSoftware De