标题： 关于使用少量随机位的随机舍入 显示英文标题

标题： On Stochastic Rounding with Few Random Bits

摘要： 大规模数值计算越来越多地使用低精度浮点格式和混合精度算术，这些可以通过随机舍入技术得到增强，即根据中间高精度值距离两个舍入候选值的距离，随机将其向上或向下舍入。 随机舍入除了需要高精度输入值外，还需要随机位的来源。 由于提供高质量的随机位会带来额外的计算成本，因此在保持给定计算或计算域中随机舍入的期望特性的同时，尽量减少所需的位数是有意义的。 本文研究了几种少量位随机舍入的实现方式，并展示了几种自然实现方式如何有时会在舍入过程中引入显著偏差，而在无限位、无限精度的情况下检查这些实现时则不存在这些问题。 文章探讨了这些偏差在机器学习实例中的影响，从而为从业者在开发或采用低精度浮点时应了解的另一类配置参数打开了新的视角。 代码可在http://github.com/graphcore-research/arith25-stochastic-rounding获取。 显示英文摘要

摘要： Large-scale numerical computations make increasing use of low-precision (LP) floating point formats and mixed precision arithmetic, which can be enhanced by the technique of stochastic rounding (SR), that is, rounding an intermediate high-precision value up or down randomly as a function of the value's distance to the two rounding candidates. Stochastic rounding requires, in addition to the high-precision input value, a source of random bits. As the provision of high-quality random bits is an additional computational cost, it is of interest to require as few bits as possible while maintaining the desirable properties of SR in a given computation, or computational domain. This paper examines a number of possible implementations of few-bit stochastic rounding (FBSR), and shows how several natural implementations can introduce sometimes significant bias into the rounding process, which are not present in the case of infinite-bit, infinite-precision examinations of these implementations. The paper explores the impact of these biases in machine learning examples, and hence opens another class of configuration parameters of which practitioners should be aware when developing or adopting low-precision floating point. Code is available at http://github.com/graphcore-research/arith25-stochastic-rounding.