标题： 大学生在开放性任务中手动生成和机器生成评分的特征 显示英文标题

标题： Characteristics of hand and machine-assigned scores to college students' answers to open-ended tasks

摘要： 高等教育中的学习评估是政策制定者、教育工作者、家长和学生关注的重要问题。 适当进行评估可能需要在评估系统中包括构造性回答测试。 我们研究了在大规模使用开放性测量方法时，评分成本和其他问题（例如处理时间与读者间一致性）是否可以通过机器评分来解决。 对来自14所学院的1359名学生的分析发现，两名人类评分者在对三种类型开放性问题的回答评分上高度一致。 这些评分者给出的分数也与计算机评分高度一致。 机器评分与SAT分数、大学成绩及其他指标的相关性与人工评分的相关性相当。 机器评分并未扩大不同种族/民族或性别群体之间的平均分数差异。 我们的研究结果表明，通过提供一种快速、准确且经济的评分方式，机器评分可以促进在大规模测试项目中使用开放性问题。 显示英文摘要

摘要： Assessment of learning in higher education is a critical concern to policy makers, educators, parents, and students. And, doing so appropriately is likely to require including constructed response tests in the assessment system. We examined whether scoring costs and other concerns with using open-end measures on a large scale (e.g., turnaround time and inter-reader consistency) could be addressed by machine grading the answers. Analyses with 1359 students from 14 colleges found that two human readers agreed highly with each other in the scores they assigned to the answers to three types of open-ended questions. These reader assigned scores also agreed highly with those assigned by a computer. The correlations of the machine-assigned scores with SAT scores, college grades, and other measures were comparable to the correlations of these variables with the hand-assigned scores. Machine scoring did not widen differences in mean scores between racial/ethnic or gender groups. Our findings demonstrated that machine scoring can facilitate the use of open-ended questions in large-scale testing programs by providing a fast, accurate, and economical way to grade responses.