Title: CosmoCore Affective Dream-Replay Reinforcement Learning for Code Generation Show Chinese title

Title: CosmoCore 情感梦境重放强化学习用于代码生成

Abstract: We introduce CosmoCore, a neuroscience-inspired reinforcement learning (RL) architecture that integrates affective signals to enhance code generation in large language models (LLMs). Motivated by human and animal learning where embarrassment from mistakes drives rapid correction, as observed in training a puppy to avoid repeating errors after a single scolding CosmoCore tags code generation trajectories with valence and surprise using a lightweight multi-layer perceptron (MLP). High-negative valence (cringe) episodes, such as buggy code outputs, are prioritized in a Dream Queue for five-fold replay during off-policy updates, while low-surprise successes are pruned to prevent overconfidence and buffer bloat. Evaluated on code generation benchmarks like HumanEval and BigCodeBench, alongside simulations with a custom data pipeline environment, CosmoCore reduces hallucinated code (e.g., syntax errors or logical bugs) by 48\% and accelerates self-correction by 45\%. Local experiments using Hugging Face models in a PySpark environment validate these gains, with code snippets provided for replication. Ablations confirm valence tagging boosts curiosity in exploration, and pruning mitigates inefficiency. This framework extends RL from human feedback (RLHF) for more emotionally aware code assistants, with applications in IDEs and data pipelines. Code and the custom mini-world simulation are released. Show Chinese abstract

Abstract: 我们引入了CosmoCore，这是一种受神经科学启发的强化学习（RL）架构，它整合了情感信号以增强大型语言模型（LLMs）中的代码生成。 受到人类和动物学习的启发，在训练小狗避免在一次责骂后重复错误时观察到的尴尬感驱动快速修正，CosmoCore使用轻量级多层感知机（MLP）对代码生成轨迹进行情感和惊喜度标记。 高负向情感（尴尬）事件，如存在错误的代码输出，在离策略更新期间被优先放入梦境队列中进行五倍重放，而低惊喜的成功则被修剪以防止过度自信和缓冲区溢出。 在代码生成基准测试如HumanEval和BigCodeBench上，以及使用自定义数据管道环境的模拟中进行评估，CosmoCore将幻觉代码（例如语法错误或逻辑错误）减少了48%，并加速了自我修正45%。 在PySpark环境中使用Hugging Face模型进行的本地实验验证了这些改进，并提供了代码片段以供复制。 消融实验确认情感标记增强了探索中的好奇心，而修剪减少了效率低下。 该框架扩展了从人类反馈中学习的强化学习（RLHF），以实现更具情感意识的代码助手，应用于集成开发环境（IDEs）和数据管道。 代码和自定义的小世界模拟已发布。