基于工作负载的人工智能应用概念和硬件优化.pdf

1、AI Application Concept and Hardware Optimization by WorkloadIntel DCSE APJAI Application Concept and Hardware Optimization by WorkloadGerry JuanAI System ArchitectCustomer and Systems EngineeringSERVERAbstractInference tasks vary widely in complexity,data size,latency requirements,and parallelism,an

2、d each workload type interacts differently with CPU capabilities.Understanding this relationship allows for more effective hardware selection and optimization strategies tailored to specific use cases.Key Learning Areas-AI Model Architecture-Types of Inference Workloads-Quantization:Balancing Accura

3、cy and Efficiency-Data Throughput and Bandwidth-Benchmarking Inference Performance-Frameworks and Libraries Impact PerformanceTraining and Inferencehttps:/www.goohttps:/ is the process of teaching a machine learning model to learn patterns and relationships from a dataset.InferenceInference is the p

4、rocess of using a trained machine learning model to make predictions or decisions on new,unseen data.https:/wandb.ai/byyoung3/ml-news/reports/DeepSeek-V3-Training-671-Billion-Parameters-with-a-6-Million-dollar-Budget-VmlldzoxMDczNTI2NgDense ModelsAI ReasoningAI DistillationMixture of Experts(MoE)Mod

5、elsTypes of AI ModelsStorage to GPU/CPUBPU to GPU,GPU to CPU,CPU to GPUGPU Scale UP(Top Board,PCIe SwitchAI Workload Data ThroughputCPU Cores,Bus Bandwidth and ThroughputRemarkDescriptionConfigurationCPU Core,2 x 96=192 CoresThreads,192 x 2=384 ThreadsThreads2pcs GNR-AP 96CCPUDDR-6400 32Gx24=768G,1D

6、PCMemoryTensor Core,2 x 456=912 CoresMax Threads,2 x 233472=466,948 ThreadsThreads2pcs Nvidia H100 350WGPUResNet50 Workload Performance Analysis62.0722.0121.3423.0126.3236.82140.22280.1244.3444.8545.4445.0445.3345.8248.5746.070102030405060050100150200250300151