标题： TriADA：用于加速三维离散变换的大规模并行三线性矩阵-张量乘加算法和设备架构 显示英文标题

标题： TriADA: Massively Parallel Trilinear Matrix-by-Tensor Multiply-Add Algorithm and Device Architecture for the Acceleration of 3D Discrete Transformations

摘要： 多线性变换在高性能计算（HPC）和人工智能（AI）工作负载中至关重要，其中数据以张量形式表示。 然而，它们的高计算和内存需求随着维度的增加而增加，常常减缓关键任务。 此外，通过增加并行处理单元的数量来扩展计算会显著增加能耗，限制了广泛采用，尤其是在稀疏数据的情况下，这在HPC和AI应用中很常见。 本文介绍了三线性算法和同构算法设备架构（TriADA），以解决这些挑战，并具有以下创新：（1）一种大规模并行、低秩算法，用于计算一组三线性（3D）离散正交变换（3D-DXTs），这是更一般的三模式矩阵乘张量（3D-GEMT）的特殊情况；（2）一种基于外积的新GEMM内核，具有解耦流式活动内存，专门设计用于加速3D-GEMT操作；（3）一种与所提出算法同构的，完全分布的三维网格互连处理单元或单元网络，具有无坐标、数据驱动的局部处理活动，与问题规模无关；（4）一种弹性稀疏外积（ESOP）方法，避免了具有零值操作数的不必要的计算和通信操作，从而提高了能效、计算精度和稳定性。 TriADA能够在时间步数线性的情况下执行各种三线性变换，具有超立方体算术复杂度。 TriADA的大规模并行、可扩展和节能架构非常适合加速多线性张量操作，这是AI和HPC工作负载中最耗资源的部分。 显示英文摘要

摘要： Multilinear transformations are key in high-performance computing (HPC) and artificial intelligence (AI) workloads, where data is represented as tensors. However, their high computational and memory demands, which grow with dimensionality, often slow down critical tasks. Moreover, scaling computation by enlarging the number of parallel processing units substantially increases energy consumption, limiting widespread adoption, especially for sparse data, which is common in HPC and AI applications. This paper introduces the Trilinear Algorithm and isomorphic to algorithm Device Architecture (TriADA) to address these challenges with the following innovations: (1) a massively parallel, low-rank algorithm for computing a family of trilinear (3D) discrete orthogonal transformations (3D-DXTs), which is a special case of the more general 3-mode matrix-by-tensor multiplication (3D-GEMT); (2) a new outer-product-based GEMM kernel with decoupled streaming active memory, specially designed to accelerate 3D-GEMT operation; (3) an isomorphic to the proposed algorithm, fully distributed 3D network of mesh interconnected processing elements or cells with a coordinate-free, data-driven local processing activity, which is independent of problem size; (4) an elastic sparse outer-product (ESOP) method that avoids unnecessary computing and communication operations with zero-valued operands, thereby enhancing energy efficiency, computational accuracy, and stability. TriADA is capable of performing a variety of trilinear transformations with hypercubic arithmetic complexity in a linear number of time-steps. The massively parallel, scalable, and energy-efficient architecture of TriADA is ideal for accelerating multilinear tensor operations, which are the most demanding parts of AI and HPC workloads.