于超-RLinf：面向具身智能的可扩展与自适应大规模强化学习框架.pdf

1、RLinf:A Scalable,Flexible and Large-Scale Reinforcement Learning Framework for Embodied IntelligenceChao Yu(于超)目录01Motivation02System-level Design03Algorithm-level Design01MotivationEmbodied IntelligenceBuildingmodelsthatcanpasstheembodiedembodiedTuringTuringtesttestwill providearoadmapforthe next g

2、eneration of AI.Yann LecunRoboticRobotic FoundationFoundation ModelModelleverage the general ability of large models and the special features of robot-specific data 4321Perception AIGenerative AIAgentic AIPhysical AIPhysical AIReinforcement LearningFutureNowPastevolutionevolutionHuman Data-driven Pr

3、e-training/Supervised Fine-TuningReward-driven Reinforcement LearningReinforcement Learning Pre-training?RL Framework for Large Reasoning Model2024TRL2025OpenRLHFNVIDIA Aligner字节VeRL蚂蚁 AReaL阿里 ROLL华为 AsyncFlow智谱 slimeRL framework for VLA?None!Robotic Robotic FoundationFoundationModelModelCerebrumCer

4、ebrum+CerebellumCerebellumCerebrumCerebrumLargeLargeReasoningReasoningModelModelRoboBrainRoboBrainOpenVLAOpenVLAPiPi-0.50.5RL framework for VLA?None!Robotic Robotic FoundationFoundationModelModelMultiMulti-step Decisionstep DecisionOneOne-step Decisionstep DecisionLargeLargeReasoningReasoningModelMo

5、delRL framework for VLA?None!Robotic Robotic FoundationFoundationModelModelGPUGPU-based Simulatorbased SimulatorDatasetDatasetLargeLargeReasoningReasoningModelModelNVIDIA Isaac SimNVIDIA Isaac SimGENESISGENESISManiSkillManiSkillRealistic 3D RenderingRealistic 3D RenderingEfficient Physical Simulatio

6、n Efficient Physical Simulation 02System-levelDesignRLinf:An Open-source RL Framework for Embodied Intelligence1 Chao Yu et al.,RLinf:Flexible and Efficient Large-scale Reinforcement Learning via Macro-to-Micro Flow Transformation.https:/arxiv.org/abs/2509.15965 Typical reasoning RL workflow Typical