1、Why Language Models HallucinateAdam Tauman KalaiOpenAIOfir NachumOpenAISantosh S.VempalaGeorgia TechEdwin ZhangOpenAISeptember 4,2025AbstractLike students facing hard exam questions,large language models sometimes guess whenuncertain,producing plausible yet incorrect statements instead of admitting

2、uncertainty.Such“hallucinations”persist even in state-of-the-art systems and undermine trust.We argue thatlanguage models hallucinate because the training and evaluation procedures reward guessing overacknowledging uncertainty,and we analyze the statistical causes of hallucinations in the moderntrai

3、ning pipeline.Hallucinations need not be mysteriousthey originate simply as errors in binaryclassification.If incorrect statements cannot be distinguished from facts,then hallucinationsin pretrained language models will arise through natural statistical pressures.We then arguethat hallucinations per

4、sist due to the way most evaluations are gradedlanguage models areoptimized to be good test-takers,and guessing when uncertain improves test performance.This“epidemic”of penalizing uncertain responses can only be addressed through a socio-technicalmitigation:modifying the scoring of existing benchma

5、rks that are misaligned but dominateleaderboards,rather than introducing additional hallucination evaluations.This change maysteer the field toward more trustworthy AI systems.1IntroductionLanguage models are known to produce overconfident,plausible falsehoods,which diminish theirutility.This error

6、mode is known as“hallucination,”though it differs fundamentally from thehuman perceptual experience.Despite significant progress,hallucinations continue to plague thefield,and are still present in the latest models(OpenAI,2025a).Consider the prompt:What is Adam Tauman Kalais birthday?If you know,jus