标题： 概率确定性有限自动机和递归网络，重访 显示英文标题

标题： Probabilistic Deterministic Finite Automata and Recurrent Networks, Revisited

摘要： reservoir 计算机 (RCs) 和递归神经网络 (RNNs) 在理论上可以模仿任何有限状态自动机，一些研究人员证明了这一点在实践中同样成立。 我们测试了广义线性模型、RCs 和长短期记忆 (LSTM) RNN 架构预测由大量概率确定性有限状态自动机 (PDFAs) 生成的随机过程的能力。 PDFAs 提供了一个极好的性能基准，因为它们可以系统地枚举，所生成过程的随机性和相关结构是完全已知的，并且其最优的记忆限制预测器很容易计算。 不出所料，LSTMs 的表现优于 RCs，RCs 又优于广义线性模型。 令人惊讶的是，每种方法在训练后可能最多有 50% 的预测准确性低于最大值，在优化时则通常会低于最大预测准确性的 5%，尽管之前可用的方法使用数量级更少的数据就能达到最大预测准确性。 因此，尽管 RCs 和 RNNs 具有表示的普适性，但在处理简单刺激时，使用它们可能会导致令人惊讶的预测差距。 由此得出结论，推断“因果状态”或“预测状态表示”的方法有着重要且未被充分重视的作用。 显示英文摘要

摘要： Reservoir computers (RCs) and recurrent neural networks (RNNs) can mimic any finite-state automaton in theory, and some workers demonstrated that this can hold in practice. We test the capability of generalized linear models, RCs, and Long Short-Term Memory (LSTM) RNN architectures to predict the stochastic processes generated by a large suite of probabilistic deterministic finite-state automata (PDFA). PDFAs provide an excellent performance benchmark in that they can be systematically enumerated, the randomness and correlation structure of their generated processes are exactly known, and their optimal memory-limited predictors are easily computed. Unsurprisingly, LSTMs outperform RCs, which outperform generalized linear models. Surprisingly, each of these methods can fall short of the maximal predictive accuracy by as much as 50% after training and, when optimized, tend to fall short of the maximal predictive accuracy by ~5%, even though previously available methods achieve maximal predictive accuracy with orders-of-magnitude less data. Thus, despite the representational universality of RCs and RNNs, using them can engender a surprising predictive gap for simple stimuli. One concludes that there is an important and underappreciated role for methods that infer "causal states" or "predictive state representations".