利用 Andes 矩阵乘法 (AMM) 和 RISC-V 向量 (RVV) 扩展加速 AI 模型：从 CNN 到 LLM.pdf

1、Pei-Hsiang Hung,Chung-Hua Yen,I-Wei Wu.Andes TechnologyAccelerating AI Models with Andes Matrix Multiplication (AMM)from CNN to LLM1TakingMainstreamSubject to change without notice copyright 2025 Andes TechnologyOutline Introduction to Andes Matrix Multiplication(AMM)Illustrations of AMM Scalability

2、.AMM Code Generation in IREE for LLM deployment Choosing Optimal Tiling Size Generating VLEN-Agnostic code Handling LLM Prefill and Decode Stages.Performance Estimates Conclusion2TakingMainstreamSubject to change without notice copyright 2025 Andes TechnologyAMM Introduction The Andes Matrix Multipl

3、ication(AMM)is being designed to optimize tiledmatrix multiplication.M Ntile=AMM(MK tile,KN tile)Key features:The tiles are stored in the RVV vector registers 2D load and store instructions facilitatedata movement between memory and vector registers.The scalability across RVV VLEN,LMUL and SEW.Under

4、standing Tiling Size M,N and K:(fully tiled cases)1.Mis always 2.2.N equals VLEN/64.3.Kis determined by LMUL and SEW.KNMNKM3TakingMainstreamSubject to change without notice copyright 2025 Andes TechnologyVLEN-Scalable Design VLEN(Vector Length)depends on the specific VPU implementation Conditions fo

5、r the illustration below:F32*F32-F32 LMUL 1VLENMNKLMULSEW124812822I8-I32816326425624F16-F3248163251228F32-F322481610242164TakingMainstreamSubject to change without notice copyright 2025 Andes TechnologyLMUL-Scalable Design LMUL(Vector Length Multiplier)AMM supports the integer LMUL values.Fractional

6、 LMULs are not supported;boundary control is used instead.Conditions for the illustration below:F32 *F32-F32 VLEN 128VLENMNKLMULSEW124812822I8-I32816326425624F16-F3248163251228F32-F322481610242165TakingMainstreamSubject to change without notice copyright 2025 Andes TechnologySEW-Scalable Design SEW(