前沿趋势论坛-多模态机器翻译前沿趋势-中国科学院计算技术研究所.pdf

1、多模态机器翻译前沿趋势!#$%&()*+,1!n#$#$%&%&n()&)&n*+*+%&%&2#$%&n!#$n%&!()*+!#$%&()*+,-./&()*+,0)1)*+,n!#$%#$&!2345670)1)*+,3#$()*+n()*+,()*+,-.-.!/!/-.-.89:;?A!#$%&()*+,-./012n!#$%&()&*+,-.,*/0,1234567,89:;?,ABCnDEFGH,EFIJKLMNOPQRSEFTNUVWOXPYZGHF=n_%aEFVWbc89defghijkhi4!#$%&()*+,-./0123456/!#$%!#$%!#!#%&$%&%()

2、*+,#$(),+n()*()*+,-.+,-.!/-.!/-.)BCD:;1EF)BG*HIJn!#$%&()*+,-./01234-567897:;KLCD:;2MKNO)BIJAPKNO*HIJn(VQ-GAN+,-!1297&-?ABCDE9751 Ann Lee,Peng-Jen Chen,Changhan Wang,Jiatao Gu,Sravya Popuri,Xutai Ma,Adam Polyak,Yossi Adi,Qing He,Yun Tang,Juan Pino,and Wei-Ning Hsu.2022.Direct Speech-to-Speech Transla

3、tion With Discrete Units.In Proceedings of ACL 2022.2 Alexandre Dfossez,Jade Copet,Gabriel Synnaeve,Yossi Adi.2022.High Fidelity Neural Audio Compression.arXiv 2022.76/8912233564(a):;?76/A23578798198263469918796512233564(b)BC?D6/!#$%&#$!#$-.#$%-.#$%!,0,1,1%&%/0-.12%-.12%!$,&0,1,1%&%345%6789:;1 Seaml

4、ess Communication.2023.SeamlessM4T:Massively Multilingual&Multimodal Machine Translation.arXiv 2023.2 Yu-An Chung,Yu Zhang,Wei Han,Chung-Cheng Chiu,James Qin,Ruoming Pang,and Yonghui Wu.2021.W2v-BERT:Combining Contrastive Learning and Masked Language Modeling for Self-Supervised Speech Pre-Training.

5、In Proceedings of ASRU 2021.3 NLLB Team.2022.No Language Left Behind:Scaling Human-Centered Machine Translation.arXiv 2022.-.#$/01%&)*+1.0101234567#$234567#$&!#$%)*QRSGTS+,&AUVWXYXY)*+,&!2)*QRSG+,QRSZ:_aSeamlessM4T1A!w2v-BERT2G NLLB3+,&6:E76/FG76/FGD6/:E76/FG76/FGD6/(a)HIJK(b)LIJK(c)SeamlessM4TConfo

6、rmer!#$%Transformer&#$%Transformer&($%)*+,%!#$%&%&()!%&()-.#$/01%&),+2.8989:567#$&:567#$&!2)*QRS+)*bcS+/)1&1)*QRSWhisper QRSde)*#$%&)*bcSf_KgQ-Former2gCTC3hij/)1&EFmnoEpiq!#$%&()I ate pizza yesterday-.#$/01%&)2+3.;#&#&klmfn#$%)*&o bcS o#$%pq&(!2rsUVAtnpq&(0)1uvwx0)1)*y0)1pq(+,z|QRSYA#$%)*&wav2vec 2.