标题： 大陪审团：一种动态质量标准的协作机器学习模型评估协议 显示英文标题

标题： GrandJury: A Collaborative Machine Learning Model Evaluation Protocol for Dynamic Quality Rubrics

摘要： 生成式机器学习模型已成为现代系统的核心，推动了创意写作、摘要生成、多跳推理和上下文感知对话等应用。 这些模型支撑着大规模人工智能助手、工作流自动化和自主决策。 在这些领域中，可接受的响应很少是绝对或静态的，而是多样且高度依赖于上下文的。 然而，标准的评估制度仍然依赖于静态的基准测试，鼓励优化以获得排行榜分数，而不是与动态用户需求或不断变化的现实保持一致。 GrandJury引入了一种正式的评估协议，结合时间衰减聚合、完整的可追溯性，以及对动态、透明任务评分标准的归属支持和多评估者的人类判断。 这些元素共同实现了多元化的、可问责的评估，能够捕捉不断演变的共识并揭示分歧。 我们提供了一个开源实现（grandjury PyPI包）和一个公开的大型语言模型（LLM）推理输出集合，以说明需求和方法。 GrandJury为人工智能从业者在评估没有绝对真实数据的机器学习输出时提供了一种新范式。 显示英文摘要

摘要： Generative Machine Learning models have become central to modern systems, powering applications in creative writing, summarization, multi-hop reasoning, and context-aware dialogue. These models underpin large-scale AI assistants, workflow automation, and autonomous decision-making. In such domains, acceptable response is rarely absolute or static, but plural and highly context-dependent. Yet standard evaluation regimes still rely on static, benchmark-style tests, incentivizing optimization toward leaderboard scores rather than alignment with dynamic user needs or evolving realities. GrandJury introduces a formal evaluation protocol combining time-decayed aggregation, complete traceability, with the support of dynamic, transparent task rubric attribution, and multi-rater human judgment. Together, these elements enable pluralistic, accountable evaluation that captures evolving consensus and surfaces disagreement. We provide an open-source implementation (grandjury PyPI package) and a public collection of Large Language Model (LLM) inference outputs to illustrate the need and method. GrandJury provides a new paradigm for AI practitioners when evaluating machine learning outputs without absolute ground truth.