朱思语-基于视频生成模型Hallo和Champ的数字人构建.pdf

1、基于视频生成模型Hallo和Champ的数字人构建朱思语|复旦大学朱思语复旦大学教授复旦大学人工智能创新与产业研究院研究员、长聘教授及博士生导师，研究方向为视频与三维生成模型。在CVPR、ICCV、ECCV、PAMI等国际会议和期刊上发表论文60余篇。博士期间，他联合创立了3D视觉公司Altizure，后被苹果公司收购。2017年至2023年，他担任阿里云人工智能实验室总监。他曾任ICCV/AAAI领域主席/程序委员会成员，荣获中国计算机学会杰出工程师奖。目 录CONTENTSI.数字人的产业背景II.数字人的技术难点III.数字人的整体方案IV.数字人的技术实践V.总结与展望数字人：生成式应

2、用爆发数字人：生成式应用爆发数字人：生成式应用爆发数字人：生成式应用爆发数字人：主流技术方案VAE:maximize variational lower boundInputOutput生成式模型的定义人工智能的关键概念Video Diffusion models 视频生成模型的快速发展Video Auto-regressive modelsLatent Space DiffusionDiffusion through Transformer Controllable video generation is still chanllenging.Can video generation res

3、tore the 3D physical world?Sora:世界模拟器？2025/5/2614 Appearance Geometry Motion&DynamicsThe Bottleneck of Scaling Law Hard to really model the physical word.Failure case in appearance and geometry.The Bottleneck of Scaling Law2025/5/2616 Hard to really model the physical word!Failure case in motion.Con

4、trollable video generation is still chanllenging.Can video generation restore the 3D physical world?Sora:世界模拟器？2025/5/2617 Appearance Geometry Motion&Dynamics Appearance and corresponding lighting.MCMat:Multiview-Consistent and Physically Accurate PBR Material Generation.外观：纹理和材质 Appearance and corr

5、esponding lighting.MCMat:Multiview-Consistent and Physically Accurate PBR Material Generation.外观：纹理和材质2025/5/2619 Directly Generate Dynamic 3D?Static 3D Generation limited to small.VideoMV:Consistent Multi-View Generation Based on Large Video Generative Model.几何：三维形状2025/5/2620 Directly Generate Dyn

6、amic 3D?Static 3D Generation limited to small.VideoMV:Consistent Multi-View Generation Based on Large Video Generative Model.几何：三维形状 Directly Generate Dynamic 3D?Not to mention 4D generation.ECCV 2024 STAG4D:Spatial-Temporal Anchored Generative 4D Gaussians运动和动画 Directly Generate