Title: Fine-grained Generalization Analysis of Structured Output Prediction Show Chinese title

Title: 结构化输出预测的细粒度泛化分析

Abstract: In machine learning we often encounter structured output prediction problems (SOPPs), i.e. problems where the output space admits a rich internal structure. Application domains where SOPPs naturally occur include natural language processing, speech recognition, and computer vision. Typical SOPPs have an extremely large label set, which grows exponentially as a function of the size of the output. Existing generalization analysis implies generalization bounds with at least a square-root dependency on the cardinality $d$ of the label set, which can be vacuous in practice. In this paper, we significantly improve the state of the art by developing novel high-probability bounds with a logarithmic dependency on $d$. Moreover, we leverage the lens of algorithmic stability to develop generalization bounds in expectation without any dependency on $d$. Our results therefore build a solid theoretical foundation for learning in large-scale SOPPs. Furthermore, we extend our results to learning with weakly dependent data. Show Chinese abstract

Abstract: 在机器学习中，我们经常遇到结构化输出预测问题 (SOPPs)，即输出空间具有丰富内部结构的问题。SOPPs 自然出现的应用领域包括自然语言处理、语音识别和计算机视觉。典型的 SOPPs 具有极其庞大的标签集，该标签集的大小随着输出规模呈指数增长。现有的泛化分析表明，泛化界至少依赖于标签集基数 $d$ 的平方根，这在实际应用中可能没有意义。本文中，我们通过开发具有对数依赖性于 $d$ 的高概率界，显著改进了当前的技术水平。此外，我们利用算法稳定性视角，在没有任何依赖于 $d$ 的情况下，发展出期望下的泛化界。因此，我们的成果为大规模 SOPPs 学习奠定了坚实的理论基础。此外，我们将研究扩展到弱相关数据的学习中。