标题： 传感器：一种基于机器学习的在线注释工具，用于从社交媒体应用的用户评论中发现隐私问题 显示英文标题

标题： SENSOR: An ML-Enhanced Online Annotation Tool to Uncover Privacy Concerns from User Reviews in Social-Media Applications

摘要： 社交媒体应用的广泛使用引发了重要的隐私担忧，这在用户评论中经常被提及。 这些评论还为开发者提供了宝贵的见解，以通过解决这些问题并引入更好的功能来改进应用。 然而，评论的数量庞大且内容复杂，使得开发者手动识别和优先处理与隐私相关的担忧变得具有挑战性。 先前的研究开发了软件工具，利用机器学习自动将用户评论分类为与隐私相关、与隐私无关、错误报告、功能请求等。 值得注意的是，目前缺乏对将评论具体分类为与隐私相关的功能请求、与隐私相关的错误报告或与隐私无关的评论的关注。 本文介绍了SENtinel SORt (SENSOR)，这是一种自动化在线标注工具，旨在帮助开发者对用户评论进行标注和分类。 为了实现此类评论的自动化标注，本文引入了标注模型GRACE（基于GRU的注意力与CBOW嵌入），该模型使用门控循环单元（GRU）结合连续词袋（CBOW）和注意力机制。 大约16000条来自Google Play商店七款流行社交媒体应用的用户评论被分析，包括Instagram、Facebook、WhatsApp、Snapchat、X（前身为Twitter）、Facebook Lite和Line。 两名标注者手动标记了这些评论，达到了Cohen's Kappa值0.87，确保了用于训练机器学习模型的高一致性标注数据集。 在测试的模型中，尽管存在类别不平衡，GRACE表现出最佳性能（宏F1分数：0.9434，宏ROC-AUC：0.9934，准确率：95.10%）。 SENSOR显示出显著的潜力，可以帮助开发者从用户评论中提取并处理与隐私相关的功能请求或错误报告，从而提升用户隐私和信任。 显示英文摘要

摘要： The widespread use of social media applications has raised significant privacy concerns, often highlighted in user reviews. These reviews also provide developers with valuable insights into improving apps by addressing issues and introducing better features. However, the sheer volume and nuanced nature of reviews make manual identification and prioritization of privacy-related concerns challenging for developers. Previous studies have developed software utilities to automatically classify user reviews as privacy-relevant, privacy-irrelevant, bug reports, feature requests, etc., using machine learning. Notably, there is a lack of focus on classifying reviews specifically as privacy-related feature requests, privacy-related bug reports, or privacy-irrelevant. This paper introduces SENtinel SORt (SENSOR), an automated online annotation tool designed to help developers annotate and classify user reviews into these categories. For automating the annotation of such reviews, this paper introduces the annotation model, GRACE (GRU-based Attention with CBOW Embedding), using Gated Recurrent Units (GRU) with Continuous Bag of Words (CBOW) and Attention mechanism. Approximately 16000 user reviews from seven popular social media apps on Google Play Store, including Instagram, Facebook, WhatsApp, Snapchat, X (formerly Twitter), Facebook Lite, and Line were analyzed. Two annotators manually labelled the reviews, achieving a Cohen's Kappa value of 0.87, ensuring a labeled dataset with high inter-rater agreement for training machine learning models. Among the models tested, GRACE demonstrated the best performance (macro F1-score: 0.9434, macro ROC-AUC: 0.9934, and accuracy: 95.10%) despite class imbalance. SENSOR demonstrates significant potential to assist developers with extracting and addressing privacy-related feature requests or bug reports from user reviews, enhancing user privacy and trust.