Skip2-LoRA：一种适用于低成本边缘设备的轻量级设备内 DNN 微调方法.pdf

1、Skip2-LoRA:A Lightweight On-device DNN Fine-tuning Method for Low-cost Edge Devices1Hiroki Matsutani,Masaaki Kondo,Kazuki Sunaga(Keio Univ),Radu Marculescu(UT Austin)TinyML:Applications Machine learning tasks in real environmentsFactory,building,robot,mobility,security,surveillance,BuildingFactoryRo

2、bot(UAV)SurveillanceWheelchairWeather2Electrical SafetyHuman RecognitionAn example:Equipment monitoring Anomaly detection on air-conditioning systemsAnomaly detection results are transmitted to a cloud server and then visualized at the cloud side3On-device finetuning for IoT devices Motivation for n

3、eural network training at edge sideAddressing the gap between pretrained model and deployed environment by updating the model on-device41 Mineto Tsukada et al.,A Neural Network-Based On-device Learning Anomaly Detector for Edge Devices,IEEE Trans.on Computers(2020).2 Kazuki Sunaga et al.,Addressing

4、Gap between Training Data and Deployed Environment by On-Device Learning,IEEE Micro(2023).SensorSensorSensorSensorSensor data may change due to noises or location-dependent factors1,2On-device finetuning for IoT devices 2D visualization results of 6-class human activity recognition dataset(30 human

5、subjects)Samples obtained from the same human subject are plotted with the same colorSamples from the same human subject form clusters(e.g.,Walking,Walking upstairs,Walking downstairs,Laying)51 Jorge Reyes-Ortiz et al.,Human Activity Recognition Using Smartphones,UCI Machine Learning Repository(2012

6、).2 Hiroki Matsutani et al.,A Tiny Supervised ODL Core with Auto Data Pruning for Human Activity Recognition,IEEE BSN24.12On-device finetuning for IoT devices 2D visualization results of 6-class human activity recognition dataset(30 human subjects)Problem:A pre-trained model that has been optimized