标题： SoftAdapt：多部分损失函数的神经网络自适应损失权重技术 显示英文标题

标题： SoftAdapt: Techniques for Adaptive Loss Weighting of Neural Networks with Multi-Part Loss Functions

摘要： 自适应损失函数的构建是一个活跃的研究领域，并且近年来随着深度学习的成功而广受欢迎。 然而，现有的自适应损失函数框架通常会遇到收敛速度慢以及对损失组件权重选择不佳的问题。 传统上，多部分损失函数的各个组成部分要么被赋予相等的权重，要么通过启发式方法确定权重，这些方法可以产生接近最优（或次优）的结果。 为了解决这个问题，我们提出了一组方法，称为 SoftAdapt，该方法基于组成损失的实时性能统计动态调整多部分损失函数的权重。 SoftAdapt 在数学上直观易懂，计算效率高且易于实现。 本文介绍了 SoftAdapt 的数学公式和伪代码，同时还展示了我们将该方法应用于图像重建（稀疏自动编码器）和合成数据生成（内省变分自动编码器）时的结果。 显示英文摘要

摘要： Adaptive loss function formulation is an active area of research and has gained a great deal of popularity in recent years, following the success of deep learning. However, existing frameworks of adaptive loss functions often suffer from slow convergence and poor choice of weights for the loss components. Traditionally, the elements of a multi-part loss function are weighted equally or their weights are determined through heuristic approaches that yield near-optimal (or sub-optimal) results. To address this problem, we propose a family of methods, called SoftAdapt, that dynamically change function weights for multi-part loss functions based on live performance statistics of the component losses. SoftAdapt is mathematically intuitive, computationally efficient and straightforward to implement. In this paper, we present the mathematical formulation and pseudocode for SoftAdapt, along with results from applying our methods to image reconstruction (Sparse Autoencoders) and synthetic data generation (Introspective Variational Autoencoders).