LANL 计算存储：下推的故事.pdf

1、Gary GriderLA-UR-_LANL Computational Storage:The Pushdown StoriesSimulations:100k-nM cores,state 100TB-1PB per time step,state 5-100 variables(distributed arrays(columnar)and distributed structs(row)-64bit float.Simulations are 1-10k time steps typicallyCurrent state:read 100tb/1pb time steps or som

2、e subset and let the supercomputer find the parts we wantDealing with this kind of gravity is hard/expensive soNeed to compress but its high entropy sparse floatsNeed to protect but N copies is not feasibleNeed to index/characterize but its too big to sortWant to use standards/standard facing soluti

3、ons,no vendor lock-in,flexible solution so we can track economic changes Supercomputer idle while checkpointing but has little free memory:there is a compute resource available while writing stateDesire to run analytics on smaller machines than simulation,we dont tie up simulation resources for anal

4、ysisLess power,faster time to insight,BackgroundComputational Storage What and Why?Data GravityData Agnostic OffloadsServer memory BW does not allow many passes over streaming dataData Aware OffloadsAnalytics is often multiple orders of magnitude less reading than writingYou just have a hard time fi

5、nding what you are looking for(filter/index/histogram/etc.)Can we add metadata/indexing/ordering to data as it is written with almost no overhead and reap huge wins on read(time,hdwr resources,energy)For ScienceParticle methods-“Ordered”row-based analytics(KV)Grid methods-columnar-based analytics Pu

6、shdown what mostly reduction or nearest neighborCompression/Encoding/erasure(Eideticom,Maxlinear,Intel,Aeon)Block list(Seagate Kinetic and ZFS(library to get block lists from file)KV range(SK hynix KVCSD)Object list(SK hynix OBCSD,Versity,AirMettle,NeuroBlade)