关于建立基准的教程用于使用公共数据评估商业大型语言模型.pdf

1、IMF IMF|Statistics1Benchmarking Commercial Large Language ModelsNOVEMBER 21,2024Jan BatznerWeizenbaum Institute,German Internet InstituteTechnical University Munich,Grad.Center CITIMF IMF|StatisticsIMF IMF|Statistics3Research on Research on Large Language Models Large Language Models“Since large lan

2、guage models,or LLMs,started to appear in 2017,the share of AI content in patent applications related to algorithmictrading has risen from 19 percent in 2017 to over 50 percent eachyear since 2020,suggesting a wave of innovation is coming in thisarea.”(IMF Blog,2024)Nassira Abbas,Charles Cohen,Dirk

3、Jan Grolleman,Benjamin Mosk(2024):Artificial Intelligence Can Make Markets More Efficientand More Volatile.International Monetary Fund(IMF)Blog.EconPhenomenonIMF IMF|Statistics4Research on Research on Large Language Models Large Language Models“Since large language models,or LLMs,started to appear i

4、n 2017,the share of AI content in patent applications related to algorithmic trading has risen from 19 percent in 2017 to over 50 percent each year since 2020,suggesting a wave of innovation is coming in this area.”(IMF Blog,2024)Nassira Abbas,Charles Cohen,Dirk Jan Grolleman,Benjamin Mosk(2024):Art

5、ificial Intelligence Can Make Markets More Efficientand More Volatile.International Monetary Fund(IMF)Blog.EconNLPAnthropic.Claude 3 Models on B IMF|Statistics5Language Modeling:What is a LM?Language Modeling:What is a LM?probability distribution over sequences of words LMs are generative models:Wha

6、t we understand as LLM are Autoregressive(AR)language models:Yann Dubois(2024):CS229 Building Large Language Models(LLMs).Stanford Online Lecture.Recorded Lecture.Slide 5(Credit:Y.Dubois).LMIMF IMF|Statistics6ReinforReinforcecement Learning by Human Feedback(RLHF)ment Learning by Human Feedback(RLHF