HC2022.Nodai.Menon.pdf

1、Harsh Menon(nod.ai)Code Generation in MLIROverviewMotivationCode generation in LLVM/MLIRDialects(Linalg,Vector,GPU,etc.)Walkthrough of code generation using IREE/SHARKCPU and GPU code generationTargeting custom acceleratorsAuto-tuningConclusionAcknowledgementsReferencesMotivationDeep learning has be

2、come extremely pervasive spanning domains ranging from autonomous cars,natural language processing to medical imagingLarge models are achieving state of the art results(such as transformers in NLP)Unfortunately,these large models take many months and millions of dollars to train on existing hardware

3、On edge-based systems,inference dominates with latency being a key metricNew hardware vendors have risen to the occasion with custom accelerators that address some of these concernsBut as the number of models and hardware combinations explode,need a strong compiler infrastructure that can provide pe

4、rformance gains and is easily re-targetable to new hardware MotivationLLVM provides a strong compiler infrastructure that already scales to various hardware targets and can be used to fill the gap But LLVM IR is too low-level and many opportunities for optimization are missed if we start at that lev

5、el of abstractionMLIR provides compiler infrastructure to handle varying levels of abstraction and provides a way to progressively lower to LLVM IR,leveraging best of both worldsIn machine learning(ML),neural networks are defined in Python-based frameworks such as Tensorflow,PyTorch and JAXMLIR help

6、s progressively lower computation graphs from their pythonic representation to LLVM IRCode generation in LLVMFinal phase in compiler pipelineMust make effective use of available resources while preserving program semanticsDeals with problems such as instruction selection,register allocation and assi