扩展型人工智能网络方案——UALink、SUE 和 NVLink 对比.pdf

1、What Matters in Scale-up Fabrics?Sharada Yeluri,AVP of EngineeringAstera LabsConnecting GPUs and AI AcceleratorsNETWORKINGAgendaScale-Up FundamentalsFabric RequirementsScale-up TechnologiesOpen Ecosystem ComparisonSummary&ActionScale-up FundamentalsPurpose-built connectivityDesigned for GPUs and AI

2、acceleratorsRack-level scaleConnects XPUs within a single rack or serverHigher bandwidth612x more than scale-out networksNeeded for high bandwidth collective communicationScale-up FundamentalsCollective CommunicationXPUs exchange partial results from tensor/pipeline or sequence parallel operationsTi

3、ght coupling-limited opportunities to hide communication latenciesLoad/Store(LS)SemanticsNo SW overhead Aggregates memory across all XPUsLS exactly identical remote or local memoryLow Latency/Low JitterLonger latency-XPU threads stallsFor inference,XPU stalls have multiplicative effect for reasoning

4、 modelsCompiler scheduling relies on stable and deterministic communication latencies.Low jitter is important Larger radix enables single-stage fabric lower latencyM switches,each N x N connect N number of XPUs to the fabricusing M switching planesExample in diagram:N=8 and M=4Deployment example:N=2

5、88,M=8Scale-up FundamentalsHigh RadixFabric packet drops are fatal to the XPU process threadsLossless fabric with hop-by-hop credit-based flow control(link as well as application level)Reliable links with FEC and link-level retries are a mustNon-blocking fabric to eliminate deadlocks:use different V

6、Cs/traffic classes for requests and responsesScale-up FundamentalsCells to be TransmittedLossless+Non-Blocking FabricFabric RequirementsRequirementPurposeUltra-Low Latency&JitterMinimize stalls;predictable communicationMemory OrderingPreserve consistency;avoid reordering256B aligned memory must not