标题： 从群体概率判断中推断命题的真理性的集体推理 显示英文标题

标题： Collective inference of the truth of propositions from crowd probability judgments

摘要： 每天，我们都会对命题的概率进行判断。 当我们表达分级的自信（例如：“我90%确定”），我们就让他人能够判断我们的判断应被赋予多大的权重。 理想情况下，人们应该分享他们的判断，以集体达成更准确的结论。 用于集体推理的点对点工具可以帮助在社交网络上消除虚假信息并放大可靠信息，从而改善民主讨论。 然而，个体在精确校准的概率判断方面未能达到理想的水平，群体动态可能会放大错误并加剧观点极化。 在这里，我们将认知科学、结构化专家判断和众包的见解结合起来，从人类概率判断中推断命题的真实性。 在一个在线实验中，376名参与者对1200个我们有真实情况的常识性陈述进行了概率判断（共451,200次评分）。 通过多数投票聚合二元判断已经表现出“群体智慧”——相对于个体判断，集体推理的准确性更高。 然而，使用连续概率评分并考虑个体准确性和校准度可以显著提高集体推理。 同伴判断行为可以用概率模型来描述，每个同伴的准确性和误校准的个体参数可以与命题概率一起推断出来。 这种无监督方法可以通过依赖真实标签的监督方法加以补充，以学习实现良好校准的集体推理的模型。 我们引入的算法可以使协作小组和在线社区汇集他们的分布式智能，共同判断命题的概率，并具有良好的不确定性意识。 显示英文摘要

摘要： Every day, we judge the probability of propositions. When we communicate graded confidence (e.g. "I am 90% sure"), we enable others to gauge how much weight to attach to our judgment. Ideally, people should share their judgments to reach more accurate conclusions collectively. Peer-to-peer tools for collective inference could help debunk disinformation and amplify reliable information on social networks, improving democratic discourse. However, individuals fall short of the ideal of well-calibrated probability judgments, and group dynamics can amplify errors and polarize opinions. Here, we connect insights from cognitive science, structured expert judgment, and crowdsourcing to infer the truth of propositions from human probability judgments. In an online experiment, 376 participants judged the probability of each of 1,200 general-knowledge claims for which we have ground truth (451,200 ratings). Aggregating binary judgments by majority vote already exhibits the "wisdom of the crowd"--the superior accuracy of collective inferences relative to individual inferences. However, using continuous probability ratings and accounting for individual accuracy and calibration significantly improves collective inferences. Peer judgment behavior can be modeled probabilistically, and individual parameters capturing each peer's accuracy and miscalibration can be inferred jointly with the claim probabilities. This unsupervised approach can be complemented by supervised methods relying on truth labels to learn models that achieve well-calibrated collective inference. The algorithms we introduce can empower groups of collaborators and online communities to pool their distributed intelligence and jointly judge the probability of propositions with a well-calibrated sense of uncertainty.