标题： 基于严重性原理的随机优化算法性能分析的新排名方案 显示英文标题

标题： A Novel Ranking Scheme for the Performance Analysis of Stochastic Optimization Algorithms using the Principles of Severity

摘要： 随机优化算法已被成功应用于多个领域以找到最优解。 由于集成系统的复杂性不断增加，正在提出新的随机算法，这使得对算法性能分析的任务变得极其重要。 在本文中，我们提供了一种新的排序方案，用于对多个单目标优化问题中的算法进行排序。 使用基于稳健引导的假设检验程序来比较算法的结果，该程序基于严重性原则。 类似于足球联赛的计分方案，我们提出了算法之间的两两比较，如同联赛竞争一样。 每个算法积累分数，并有一个性能指标，用以衡量其与其他算法相比表现的好坏，类似于足球联赛计分系统中的净胜球指标。 净胜球性能指标不仅可以作为平局决胜的依据，还可以用来获得每个算法的定量性能。 所提出的排序方案的关键创新之处在于，它考虑了每个算法的性能，同时考虑到实现的性能改进幅度及其实际相关性，并且没有任何分布假设。 所提出的排序方案与经典假设检验进行了比较，分析结果表明结果相当，而我们提出的排序方案展示了诸多额外的优势。 显示英文摘要

摘要： Stochastic optimization algorithms have been successfully applied in several domains to find optimal solutions. Because of the ever-growing complexity of the integrated systems, novel stochastic algorithms are being proposed, which makes the task of the performance analysis of the algorithms extremely important. In this paper, we provide a novel ranking scheme to rank the algorithms over multiple single-objective optimization problems. The results of the algorithms are compared using a robust bootstrapping-based hypothesis testing procedure that is based on the principles of severity. Analogous to the football league scoring scheme, we propose pairwise comparison of algorithms as in league competition. Each algorithm accumulates points and a performance metric of how good or bad it performed against other algorithms analogous to goal differences metric in football league scoring system. The goal differences performance metric can not only be used as a tie-breaker but also be used to obtain a quantitative performance of each algorithm. The key novelty of the proposed ranking scheme is that it takes into account the performance of each algorithm considering the magnitude of the achieved performance improvement along with its practical relevance and does not have any distributional assumptions. The proposed ranking scheme is compared to classical hypothesis testing and the analysis of the results shows that the results are comparable and our proposed ranking showcases many additional benefits.