标题： 去偏可能不可靠：减轻评估去偏推荐中的偏差问题 显示英文标题

标题： Debias Can be Unreliable: Mitigating Bias Issue in Evaluating Debiasing Recommendation

摘要： 最近的工作通过为推荐模型配备去偏方法，显著提高了推荐模型的性能。 由于完全暴露的数据集不可用，大多数现有方法将随机暴露的数据集作为评估去偏模型的代理，使用传统的评估方案来表示推荐性能。 然而，在本研究中，我们揭示了传统评估方案不适用于随机暴露的数据集，导致使用随机暴露数据集获得的Recall性能与使用完全暴露数据集获得的Recall性能之间存在不一致。 这种不一致表明之前去偏技术的实验结论可能存在不可靠性，并呼吁使用随机暴露数据集进行无偏的Recall评估。 为了弥补这一差距，我们提出了无偏Recall评估（URE）方案，该方案调整了随机暴露数据集的使用方式，以无偏地估计在完全暴露数据集上的真实Recall性能。 我们提供了理论证据来证明URE的合理性，并在现实世界的数据集上进行了广泛的实验以验证其有效性。 显示英文摘要

摘要： Recent work has improved recommendation models remarkably by equipping them with debiasing methods. Due to the unavailability of fully-exposed datasets, most existing approaches resort to randomly-exposed datasets as a proxy for evaluating debiased models, employing traditional evaluation scheme to represent the recommendation performance. However, in this study, we reveal that traditional evaluation scheme is not suitable for randomly-exposed datasets, leading to inconsistency between the Recall performance obtained using randomly-exposed datasets and that obtained using fully-exposed datasets. Such inconsistency indicates the potential unreliability of experiment conclusions on previous debiasing techniques and calls for unbiased Recall evaluation using randomly-exposed datasets. To bridge the gap, we propose the Unbiased Recall Evaluation (URE) scheme, which adjusts the utilization of randomly-exposed datasets to unbiasedly estimate the true Recall performance on fully-exposed datasets. We provide theoretical evidence to demonstrate the rationality of URE and perform extensive experiments on real-world datasets to validate its soundness.