Title: Towards New Benchmark for AI Alignment & Sentiment Analysis in Socially Important Issues: A Comparative Study of Human and LLMs in the Context of AGI Show Chinese title

Title: 面向人工智能对齐与社会重要问题情感分析的新基准：在通用人工智能背景下的人类与大语言模型的比较研究

Abstract: As general-purpose artificial intelligence systems become increasingly integrated into society and are used for information seeking, content generation, problem solving, textual analysis, coding, and running processes, it is crucial to assess their long-term impact on humans. This research explores the sentiment of large language models (LLMs) and humans toward artificial general intelligence (AGI) using a Likert-scale survey. Seven LLMs, including GPT-4 and Bard, were analyzed and compared with sentiment data from three independent human sample populations. Temporal variations in sentiment were also evaluated over three consecutive days. The results show a diversity in sentiment scores among LLMs, ranging from 3.32 to 4.12 out of 5. GPT-4 recorded the most positive sentiment toward AGI, while Bard leaned toward a neutral sentiment. In contrast, the human samples showed a lower average sentiment of 2.97. The analysis outlines potential conflicts of interest and biases in the sentiment formation of LLMs, and indicates that LLMs could subtly influence societal perceptions. To address the need for regulatory oversight and culturally grounded assessments of AI systems, we introduce the Societal AI Alignment and Sentiment Benchmark (SAAS-AI), which leverages multidimensional prompts and empirically validated societal value frameworks to evaluate language model outputs across temporal, model, and multilingual axes. This benchmark is designed to guide policymakers and AI agencies, including within frameworks such as the EU AI Act, by providing robust, actionable insights into AI alignment with human values, public sentiment, and ethical norms at both national and international levels. Future research should further refine the operationalization of the SAAS-AI benchmark and systematically evaluate its effectiveness through comprehensive empirical testing. Show Chinese abstract

Abstract: 随着通用人工智能系统日益融入社会，并被用于信息检索、内容生成、问题解决、文本分析、编程和运行流程，评估其对人类的长期影响至关重要。 这项研究使用李克特量表调查，探讨了大型语言模型（LLMs）和人类对通用人工智能（AGI）的情感。 分析并比较了包括GPT-4和Bard在内的七种LLMs与三个独立人类样本群体的情感数据。 还评估了三天连续时间内的感情变化。 结果表明，LLMs的情感评分存在多样性，范围从5分中的3.32到4.12。 GPT-4对AGI表现出最积极的情感，而Bard则倾向于中立情感。 相反，人类样本的平均情感较低，为2.97。 分析指出了LLMs情感形成中的潜在利益冲突和偏见，并表明LLMs可能会微妙地影响社会认知。 为了应对监管监督和基于文化背景的AI系统评估需求，我们引入了社会AI对齐和情感基准（SAAS-AI），该基准利用多维提示和经验验证的社会价值框架，以评估语言模型输出在时间、模型和多语言轴上的表现。 该基准旨在通过提供关于AI与人类价值观、公众情绪和伦理规范在国家和国际层面的稳健、可操作的见解，指导政策制定者和AI机构，包括在欧盟AI法案等框架内。 未来的研究应进一步完善SAAS-AI基准的操作化，并通过全面的实证测试系统地评估其有效性。