1、Snapdragon and Qualcomm branded products are products of Qualcomm Technologies,Inc.and/or its subsidiaries.September 24,2024How AI is learning tosee,hear,and interactIntegratingsenses:Roland MemisevicSenior Director of Engineering at Qualcomm AI ResearchJoint workwith Sunny Panchal,Apratim Bhattacha

2、ryya,Guillaume Berger,Antoine Mercier,Reza Pourreza,Sanjay Haresh,and othersSnapdragon and Qualcomm branded products are products of Qualcomm Technologies,Inc.and/or its subsidiaries.2Agenda Key concept:streaming architecture Importance of datasets for end-to-end training Efficient human-AI interact

3、ion and video-based reasoning Improving streaming video LLMs using auxiliary tasks Q&A23Generative AIcapabilitiescontinue toincreaseMODALITY AND USE CASECAPABILITY AND KPIAgentsExecute multi-step tasks with reasoning autonomously to achieve a goalVideo&3DGenerating content for a richerand more reali

4、stic experienceVoice UIVoice is a natural and intuitiveinterface for conversationLarge multimodal modelsUtilizing more sensinginput modalities to betterunderstand the worldPersonalizationFine-tuned models customized to consumers,enterprises,or industries(e.g.,LoRA)Longer context windowAllows in-dept

5、h conversationsHigher resolutionProcess higher fidelity imagesfor better accuracyLoRA:low-rank adaptation4LM:Language vision modelAI acceleration on the QualcommHexagon NPU of the Snapdragon 8Gen 3 Mobile ProcessorQualcomm AI Engine directfor improved performance andminimized memory spillage Knowled

6、ge distillation for pruning andremoving of attention blocks,resultingin accurate model with improvedperformance and power efficiencyDesigning an efficient diffusionmodel through knowledgedistillation for high accuracyFull-stack AI optimizationfor LMsRuns completelyon the deviceSignificantly reducesr