DCC 过去走过的路在哪里？接下来 DCC 又应该探索哪些领域？.pdf

1、Gary Grider,LANLAnupam Thakur,MicrosoftPankaj Mehra,Ohio State UniversityNader Salessi,MaxLinearJason Molgaard,Solidigm-ModeratorData Centric Compute:Past,Present,and FuturePanel DiscussionGary GriderSenior Director for Computing TechnologiesLos Alamos National LaboratoryAnupam ThakurDirector,Strate

2、gic Planning and ArchitectureMicrosoft AzurePankaj MehraResearch ProfessorComputer ScienceThe Ohio State Univ.CEO,Elephance Memory,Inc.Nader SalessiStorage ExecutiveComputational Storage VeteranMaxLinearJason MolgaardPrincipal Storage Solutions Architect,SolidigmPanel ModeratorWhat is the past,prese

3、nt,and future of Data Centric Compute(Near Data Compute)Opening Question for Each Panelist The past data centric is exemplified by(data parallel and compute near*)DataParallel Vector-FP4/8/16 has been done before TMC CM-1/2/5C*/DPFortran-CUDA,Madcap-AI written math-code(IBM Stretch 1961)Massive Grap

4、hs-PIM,IBM A2A,Cray XMT,HPE SD/SGI Origin NUMA-link The current is dominated by really just new versions of the aboveGPU/TPU,NVLINK,CXL-Apache pushdown analyticsApache pushdown analytics The”hopefully”near futureCXL+FamFS+PNFS(orchestration/LocalIO/mmap)revisit remote threads?Compute near memory(whe

5、re Amdahl lets you PLEASE use some standards!)Standardization of compute near storage(Apache/other)Object,PNFS,Standardize GPU-direct/pushdown block(layer violation)LANL has prototyped all three(SK hynix,Hammerspace,and Seagate)Gary Grider AI workloads are memory intensive with growing model size an

6、d complexity.Memory wall creates bottlenecks and limits performance scaling.Techniques like Quantization,Flash attention,GQA have reduced memory footprint and traffic.As the models scale to trillions of parameters and context lengths grow,memory bandwidth/capacity issues remain.A