统计理论
查看 最近的 文章
显示 2025年06月06日， 星期五 新的列表
- [1] arXiv:2506.04656 [中文pdf, pdf, html, 其他]
-
标题： 金融市场的极值依赖性分类通过自助推断标题： Classification of Extremal Dependence in Financial Markets via Bootstrap Inference主题： 统计理论 (math.ST)
准确识别多元重尾数据中的极值相依结构是一项基础但具有挑战性的任务，尤其是在金融应用中。 根据最近提出的一种基于自助法的检验程序，我们将该方法应用于2024年美国大选前一段时间内的美国标普500指数和中国A股股票的绝对对数收益率。 结果显示，与表现出不同特征且具有更紧密的极值相依模式的中国经济相比，美国经济中存在更多孤立的相依资产聚类现象。 跨市场分析表明，在材料、日常消费品和可选消费品等行业中存在较强的极值关联性，这凸显了该检验程序在大规模经验研究中的有效性。
Accurately identifying the extremal dependence structure in multivariate heavy-tailed data is a fundamental yet challenging task, particularly in financial applications. Following a recently proposed bootstrap-based testing procedure, we apply the methodology to absolute log returns of U.S. S&P 500 and Chinese A-share stocks over a time period well before the U.S. election in 2024. The procedure reveals more isolated clustering of dependent assets in the U.S. economy compared with China which exhibits different characteristics and a more interconnected pattern of extremal dependence. Cross-market analysis identifies strong extremal linkages in sectors such as materials, consumer staples and consumer discretionary, highlighting the effectiveness of the testing procedure for large-scale empirical applications.
- [2] arXiv:2506.04825 [中文pdf, pdf, html, 其他]
-
标题： 极端方向依赖的维数约简标题： A dimension reduction for extreme types of directed dependence评论： 14页，7幅图主题： 统计理论 (math.ST)
近年来，人们引入了各种新型的依赖性度量方法，能够刻画不同类型的单向依赖关系，因此也能描述多个预测变量 $\mathbf{X} = (X_1, \dots, X_p)$, $p \in \mathbb{N}$ 如何影响响应变量 $Y$。这包括 $Y$ 对 $\mathbf{X}$ 的完全依赖以及 $\mathbf{X}$ 和 $Y$ 之间的独立性，还包括一些不太为人所知的概念，例如零可解释性、随机可比性和完全分离。 某些此类度量以马尔可夫积$(Y,Y')$的形式表示，其中$Y'$是在给定$\mathbf{X}$条件下$Y$的条件独立副本。 这种维度约简原则允许这些度量通过[4]中引入的强大的基于最近邻的估计原理来估计。 为了更深入地理解这一维度约简原则，本文旨在将通常以随机向量$(\mathbf{X},Y)$表述的有向依赖的极端变体转化为马尔可夫积$(Y,Y')$。
In recent years, a variety of novel measures of dependence have been introduced being capable of characterizing diverse types of directed dependence, hence diverse types of how a number of predictor variables $\mathbf{X} = (X_1, \dots, X_p)$, $p \in \mathbb{N}$, may affect a response variable $Y$. This includes perfect dependence of $Y$ on $\mathbf{X}$ and independence between $\mathbf{X}$ and $Y$, but also less well-known concepts such as zero-explainability, stochastic comparability and complete separation. Certain such measures offer a representation in terms of the Markov product $(Y,Y')$, with $Y'$ being a conditionally independent copy of $Y$ given $\mathbf{X}$. This dimension reduction principle allows these measures to be estimated via the powerful nearest neighbor based estimation principle introduced in [4]. To achieve a deeper insight into the dimension reduction principle, this paper aims at translating the extreme variants of directed dependence, typically formulated in terms of the random vector $(\mathbf{X},Y)$, into the Markov product $(Y,Y')$.
- [3] arXiv:2506.04878 [中文pdf, pdf, html, 其他]
-
标题： kTULA：一种具有改进的超线性对数梯度KL界的Langevin抽样算法标题： kTULA: A Langevin sampling algorithm with improved KL bounds under super-linear log-gradients主题： 统计理论 (math.ST) ; 机器学习 (cs.LG) ; 概率 (math.PR) ; 机器学习 (stat.ML)
受深度学习应用的启发，其中全局Lipschitz连续性条件通常不满足，我们研究了从具有超线性增长的对数梯度的分布中采样的问题。我们提出了一种基于改进型 Langevin 动力学的新算法，称为kTULA，以解决上述采样问题，并为其性能提供了理论保证。更具体地说，我们在 Kullback-Leibler (KL) 散度中建立了非渐近收敛界，其最佳收敛率等于 $2-\overline{\epsilon}$, $\overline{\epsilon}>0$，这显著改进了现有文献中的相关结果。这使我们能够在 Wasserstein-2 距离中获得改进的非渐近误差界，该界可以进一步用于推导出kTULA解决相关优化问题的非渐近保证。为了展示kTULA的适用性，我们将所提出的算法应用于从高维双井势分布中采样的问题以及涉及神经网络的优化问题。我们表明，我们的主要结果可以用于提供kTULA性能的理论保证。
Motivated by applications in deep learning, where the global Lipschitz continuity condition is often not satisfied, we examine the problem of sampling from distributions with super-linearly growing log-gradients. We propose a novel tamed Langevin dynamics-based algorithm, called kTULA, to solve the aforementioned sampling problem, and provide a theoretical guarantee for its performance. More precisely, we establish a non-asymptotic convergence bound in Kullback-Leibler (KL) divergence with the best-known rate of convergence equal to $2-\overline{\epsilon}$, $\overline{\epsilon}>0$, which significantly improves relevant results in existing literature. This enables us to obtain an improved non-asymptotic error bound in Wasserstein-2 distance, which can be used to further derive a non-asymptotic guarantee for kTULA to solve the associated optimization problems. To illustrate the applicability of kTULA, we apply the proposed algorithm to the problem of sampling from a high-dimensional double-well potential distribution and to an optimization problem involving a neural network. We show that our main results can be used to provide theoretical guarantees for the performance of kTULA.
- [4] arXiv:2506.05112 [中文pdf, pdf, html, 其他]
-
标题： 在Donsker定理的边缘：多尺度扫描统计量的渐近性质标题： At the edge of Donsker's Theorem: Asymptotics of multiscale scan statistics评论： 41页，4幅图主题： 统计理论 (math.ST) ; 方法论 (stat.ME)
对于函数的非参数推断，多尺度检验程序解决了带宽选择的需求，并在广泛的备选假设下实现了渐近最优的检测性能。 然而，临界值强烈依赖于噪声分布，我们论证了现有方法要么在统计上不可行，要么渐近次优。 为了解决这一方法论上的挑战，我们通过弱收敛论证展示了如何通过弱化加性多尺度惩罚项为乘法权重来构建可行的多尺度检验。 这种新的理论基础保留了多尺度检验的最优检测特性，并通过定制的自助法扩展了它们在非平稳非线性时间序列中的适用性。 信号发现的推断、回归函数的拟合优度检验以及多重变点检测被详细研究，我们应用新方法分析了伊比利亚半岛2025年4月的大停电。 我们的方法得益于在Hölder空间中具有临界连续模的新颖泛函中心极限定理，在这里由于缺乏紧性，Donsker定理不成立。 概率上，我们发现了分布的上支集处才成立的一种新颖的阈值弱收敛形式。
For nonparametric inference about a function, multiscale testing procedures resolve the need for bandwidth selection and achieve asymptotically optimal detection performance against a broad range of alternatives. However, critical values strongly depend on the noise distribution, and we argue that existing methods are either statistically infeasible, or asymptotically sub-optimal. To address this methodological challenge, we show how to develop a feasible multiscale test via weak convergence arguments, by replacing the additive multiscale penalty with a multiplicative weighting. This new theoretical foundation preserves the optimal detection properties of multiscale tests and extends their applicability to nonstationary nonlinear time series via a tailored bootstrap scheme. Inference for signal discovery, goodness-of-fit testing of regression functions, and multiple changepoint detection is studied in detail, and we apply the new methodology to analyze the April 2025 power blackout on the Iberian peninsula. Our methodology is enabled by a novel functional central limit in H\"older spaces with critical modulus of continuity, where Donsker's theorem fails to hold due to lack of tightness. Probabilistically, we discover a novel form of thresholded weak convergence that holds only in the upper support of the distribution.
- [5] arXiv:2506.05113 [中文pdf, pdf, html, 其他]
-
标题： 二维X射线CT的统计微局部分析标题： Statistical microlocal analysis in two-dimensional X-ray CT评论： 27页，13幅图主题： 统计理论 (math.ST) ; 泛函分析 (math.FA)
在许多成像应用中，评估原始物体边缘在由测量数据重构的图像中的清晰程度是非常重要的。例如，原始物体为$f$，重构图像为$f^\text{rec}$，测量数据为$g$。 本文研究了二维 X 射线计算机断层扫描（CT）中的图像重构问题。 设 $f$ 为描述被扫描物体的函数，$g=Rf + \eta$ 为在 $\mathbb{R}^2$ 中由噪声 $\eta$ 污染并通过步长 $\sim\epsilon$ 采样的Radon变换数据。传统微局部分析在连续且无噪声的数据（当 $\eta = 0$ 时）情况下提供了基于扫描几何的边缘可检测性条件，但未考虑噪声和有限采样步长的影响。 我们开发了一种称为SMA（\emph{统计微局部分析}）的新技术，它使用统计假设检验框架来确定可以从$f^\text{rec}$检测到$f$图像边缘（奇异性）的可能性，并通过检验的统计功效量化边缘可检测性。 我们的方法基于我们在\cite{AKW2024_1}中提出的一种理论，该理论在$\eta \neq 0$时提供了在局部$O(\epsilon)$大小邻域内对$f^\text{rec}$的描述。 我们推导出一个统计检验方法，用于在已知$\eta$的幅度和数据采样步长的情况下检测边缘的存在性和方向性。 利用检验的零分布性质，我们量化了边缘幅度和方向性的不确定性。 通过模拟验证我们的理论，结果显示预测与实验观测之间有很强的一致性。 我们的工作不仅具有实际价值，而且具有理论价值。 SMA 是经典微局部分析理论的自然延伸，它在与数据兼容的尽可能高的分辨率下考虑了实际测量缺陷，如噪声和有限步长。
In many imaging applications it is important to assess how well the edges of the original object, $f$, are resolved in an image, $f^\text{rec}$, reconstructed from the measured data, $g$. In this paper we consider the case of image reconstruction in 2D X-ray Computed Tomography (CT). Let $f$ be a function describing the object being scanned, and $g=Rf + \eta$ be the Radon transform data in $\mathbb{R}^2$ corrupted by noise, $\eta$, and sampled with step size $\sim\epsilon$. Conventional microlocal analysis provides conditions for edge detectability based on the scanner geometry in the case of continuous, noiseless data (when $\eta = 0$), but does not account for noise and finite sampling step size. We develop a novel technique called \emph{Statistical Microlocal Analysis} (SMA), which uses a statistical hypothesis testing framework to determine if an image edge (singularity) of $f$ is detectable from $f^\text{rec}$, and we quantify edge detectability using the statistical power of the test. Our approach is based on the theory we developed in \cite{AKW2024_1}, which provides a characterization of $f^\text{rec}$ in local $O(\epsilon)$-size neighborhoods when $\eta \neq 0$. We derive a statistical test for the presence and direction of an edge microlocally given the magnitude of $\eta$ and data sampling step size. Using the properties of the null distribution of the test, we quantify the uncertainty of the edge magnitude and direction. We validate our theory using simulations, which show strong agreement between our predictions and experimental observations. Our work is not only of practical value, but of theoretical value as well. SMA is a natural extension of classical microlocal analysis theory which accounts for practical measurement imperfections, such as noise and finite step size, at the highest possible resolution compatible with the data.
新提交 (展示 5 之 5 条目 )
- [6] arXiv:2506.04441 (交叉列表自 stat.ME) [中文pdf, pdf, html, 其他]
-
标题： 关于球面Dirichlet分布：修正与结果标题： On the Spherical Dirichlet Distribution: Corrections and Results评论： 25页，2个图。本次投稿修正并扩展了一篇先前发表在《统计分布与应用期刊》上的开放获取文章。期刊参考： 《Journal of Statistical Distributions and Applications》7, 6 (2020)主题： 方法论 (stat.ME) ; 统计理论 (math.ST)
本文纠正了Guardiola（2020，统计分布与应用期刊）中的一个技术错误，呈现了更新的推导，并对球面Dirichlet分布的性质进行了扩展讨论。如今，数据挖掘和基因表达处于现代数据分析的前沿。在此，我们介绍了一种适用于这些领域的新型概率分布。本文发展了提议的球面-Dirichlet分布，旨在适合位于超球正象限的向量，因为这种情况在这些领域的数据中经常出现，从而避免不必要的概率质量。提出了所提议分布的基本性质，包括归一化常数和矩。还探讨了与其他分布的关系。基于经典推断统计方法（如矩法估计和最大似然估计）得到了估计器。开发了两个应用：第一个使用模拟数据，第二个使用真实的文本挖掘实例。这两个例子均使用提议的球面-Dirichlet分布进行拟合，并讨论了其结果。
This note corrects a technical error in Guardiola (2020, Journal of Statistical Distributions and Applications), presents updated derivations, and offers an extended discussion of the properties of the spherical Dirichlet distribution. Today, data mining and gene expressions are at the forefront of modern data analysis. Here we introduce a novel probability distribution that is applicable in these fields. This paper develops the proposed Spherical-Dirichlet Distribution designed to fit vectors located at the positive orthant of the hypersphere, as it is often the case for data in these fields, avoiding unnecessary probability mass. Basic properties of the proposed distribution, including normalizing constants and moments are developed. Relationships with other distributions are also explored. Estimators based on classical inferential statistics, such as method of moments and maximum likelihood estimators are obtained. Two applications are developed: the first one uses simulated data, and the second uses a real text mining example. Both examples are fitted using the proposed Spherical-Dirichlet Distribution and their results are discussed.
- [7] arXiv:2506.04445 (交叉列表自 stat.ME) [中文pdf, pdf, html, 其他]
-
标题： 指数寿命分布下步加应力实验的稳健估计标题： Robust Estimation in Step-Stress Experiments under Exponential Lifetime Distributions评论： 20页（不含附录），4幅图，6张表格主题： 方法论 (stat.ME) ; 统计理论 (math.ST)
许多现代产品表现出极高的可靠性，往往导致故障时间很长。因此，在正常工作条件下进行实验可能需要不切实际的长时间才能获得足够的失效数据以进行可靠的统计推断。作为替代方案，加速寿命试验（ALTs）被用来诱导更早的失效，从而减少测试时间。在步进应力实验中，确定了一种加速产品退化的应力因子，并系统性地增加该因子以引发早期失效。应力水平在预定的时间点增加，并在此期间内保持恒定。在增加应力水平下观察到的失效数据经过统计分析后，结果被外推到正常工作条件。经典的估计方法，如基于最大似然估计量（MLE）的方法，虽然以其高效性而闻名，但在存在异常值数据时缺乏鲁棒性。在这项工作中，提出了最小密度幂散度估计量（MDPDEs）作为一种稳健的替代方案，展示了效率和稳健性之间的令人满意的折衷。基于混合分布的MDPDE被开发出来，并在指数寿命假设下推导出其理论性质，包括模型参数渐近分布的表达式。通过模拟研究评估了所提出方法的良好性能，并使用真实数据证明了其适用性。
Many modern products exhibit high reliability, often resulting in long times to failure. Consequently, conducting experiments under normal operating conditions may require an impractically long duration to obtain sufficient failure data for reliable statistical inference. As an alternative, accelerated life tests (ALTs) are employed to induce earlier failures and thereby reduce testing time. In step-stress experiments a stress factor that accelerates product degradation is identified and systematically increased to provoke early failures. The stress level is increased at predetermined time points and maintained constant between these intervals. Failure data observed under increased levels of stress is statistically analyzed, and results are then extrapolate to normal operating conditions. Classical estimation methods such analysis rely on the maximum likelihood estimator (MLE) which is know to be very efficient, but lack robustness in the presence of outlying data. In this work, Minimum Density Power Divergence Estimators (MDPDEs) are proposed as a robust alternative, demonstrating an appealing compromise between efficiency and robustness. The MDPDE based on mixed distributions is developed, and its theoretical properties, including the expression for the asymptotic distribution of the model parameters, are derived under exponential lifetime assumptions. The good performance of the proposed method is evaluated through simulation studies, and its applicability is demonstrated using real data.
- [8] arXiv:2506.05116 (交叉列表自 stat.ME) [中文pdf, pdf, 其他]
-
标题： 虚假因子困境：重尾椭圆因子模型中的稳健推断标题： The Spurious Factor Dilemma: Robust Inference in Heavy-Tailed Elliptical Factor Models主题： 方法论 (stat.ME) ; 计量经济学 (econ.EM) ; 统计理论 (math.ST)
因子模型是分析高维数据的重要工具，尤其是在经济学和金融学领域。 然而，标准的确定因子数量的方法在数据表现出重尾随机性时，往往高估真实因子的数量，将由噪声引起的异常值误认为是真实的因子。 本文在椭圆因子模型（EFM）框架内解决了这一挑战，该模型能够同时处理现实数据中常见的重尾性和潜在非线性依赖性。 我们从理论上和实证上证明了重尾噪声会产生虚假的特征值，这些特征值会模拟真实的因子信号。 为了解决这个问题，我们提出了一种基于波动放大算法的新方法。 我们表明，在扰动放大的情况下，与真实因子相关的特征值相比，由重尾效应产生的虚假特征值表现出显著更大的波动（渐近趋于不稳定）。 这种差异行为使得我们可以识别和检测出真正的因子和虚假因子。 我们基于此原理开发了一种正式的检验程序，并将其应用于准确选择重尾EFM中公共因子数量的问题。 仿真研究和真实数据分析验证了我们的方法相对于现有方法的有效性，特别是在重尾性明显的场景下。
Factor models are essential tools for analyzing high-dimensional data, particularly in economics and finance. However, standard methods for determining the number of factors often overestimate the true number when data exhibit heavy-tailed randomness, misinterpreting noise-induced outliers as genuine factors. This paper addresses this challenge within the framework of Elliptical Factor Models (EFM), which accommodate both heavy tails and potential non-linear dependencies common in real-world data. We demonstrate theoretically and empirically that heavy-tailed noise generates spurious eigenvalues that mimic true factor signals. To distinguish these, we propose a novel methodology based on a fluctuation magnification algorithm. We show that under magnifying perturbations, the eigenvalues associated with real factors exhibit significantly less fluctuation (stabilizing asymptotically) compared to spurious eigenvalues arising from heavy-tailed effects. This differential behavior allows the identification and detection of the true and spurious factors. We develop a formal testing procedure based on this principle and apply it to the problem of accurately selecting the number of common factors in heavy-tailed EFMs. Simulation studies and real data analysis confirm the effectiveness of our approach compared to existing methods, particularly in scenarios with pronounced heavy-tailedness.
- [9] arXiv:2506.05188 (交叉列表自 cs.CL) [中文pdf, pdf, html, 其他]
-
标题： 反事实推理：情境中涌现的分析标题： Counterfactual reasoning: an analysis of in-context emergence主题： 计算与语言 (cs.CL) ; 人工智能 (cs.AI) ; 机器学习 (cs.LG) ; 统计理论 (math.ST)
大规模神经语言模型（LMs）在即时学习方面表现出色：能够在无需参数更新的情况下即时学习和推理输入上下文。 本文研究了语言模型中的即时反事实推理，即预测假设情景下变化的后果。 我们专注于研究一个定义明确的合成设置：需要从实际观察中推断和复制上下文噪声的线性回归任务，其中准确预测依赖于从事实观测中推断并复制上下文噪声。 我们表明，语言模型在这种受控设置下能够进行反事实推理，并提供见解表明，广泛函数类别的反事实推理可以归结为对即时上下文观察的变换；我们发现自注意力机制、模型深度以及预训练中的数据多样性推动了Transformer模型的表现。 更有趣的是，我们的发现超越了回归任务，并表明Transformer可以在序列数据上执行噪声推断，为反事实故事生成提供了初步证据。 我们的代码可在https://github.com/moXmiller/counterfactual-reasoning.git 获取。
Large-scale neural language models (LMs) exhibit remarkable performance in in-context learning: the ability to learn and reason the input context on the fly without parameter update. This work studies in-context counterfactual reasoning in language models, that is, to predict the consequences of changes under hypothetical scenarios. We focus on studying a well-defined synthetic setup: a linear regression task that requires noise abduction, where accurate prediction is based on inferring and copying the contextual noise from factual observations. We show that language models are capable of counterfactual reasoning in this controlled setup and provide insights that counterfactual reasoning for a broad class of functions can be reduced to a transformation on in-context observations; we find self-attention, model depth, and data diversity in pre-training drive performance in Transformers. More interestingly, our findings extend beyond regression tasks and show that Transformers can perform noise abduction on sequential data, providing preliminary evidence on the potential for counterfactual story generation. Our code is available under https://github.com/moXmiller/counterfactual-reasoning.git .
- [10] arXiv:2506.05200 (交叉列表自 cs.LG) [中文pdf, pdf, html, 其他]
-
标题： Transformer 遇到提示学习：一种通用近似理论标题： Transformers Meet In-Context Learning: A Universal Approximation Theory主题： 机器学习 (cs.LG) ; 统计理论 (math.ST) ; 机器学习 (stat.ML)
现代大型语言模型具备即时学习的能力，即在推理时仅使用提示中的少量输入-输出示例即可执行新任务，而无需微调或参数更新。 我们发展了一种通用近似理论，以更好地理解变换器如何实现即时学习。 对于任何函数类（每个代表一个不同的任务），我们展示了如何构建一个变换器，该变换器在没有任何进一步权重更新的情况下，仅通过几个即时上下文示例即可进行可靠的预测。 与最近的许多文献将变换器视为算法近似器的做法不同——即构造变换器来模拟优化算法的迭代过程作为近似学习问题解的一种方式——我们的工作采取了根本不同的方法，基于通用函数近似。 这种方法提供的近似保证不受所近似优化算法有效性的影响，因此远远超出了凸问题和线性函数类。 我们的构建揭示了变换器如何能够同时学习通用表示并动态适应即时上下文示例。
Modern large language models are capable of in-context learning, the ability to perform new tasks at inference time using only a handful of input-output examples in the prompt, without any fine-tuning or parameter updates. We develop a universal approximation theory to better understand how transformers enable in-context learning. For any class of functions (each representing a distinct task), we demonstrate how to construct a transformer that, without any further weight updates, can perform reliable prediction given only a few in-context examples. In contrast to much of the recent literature that frames transformers as algorithm approximators -- i.e., constructing transformers to emulate the iterations of optimization algorithms as a means to approximate solutions of learning problems -- our work adopts a fundamentally different approach rooted in universal function approximation. This alternative approach offers approximation guarantees that are not constrained by the effectiveness of the optimization algorithms being approximated, thereby extending far beyond convex problems and linear function classes. Our construction sheds light on how transformers can simultaneously learn general-purpose representations and adapt dynamically to in-context examples.
交叉提交 (展示 5 之 5 条目 )
- [11] arXiv:2401.00510 (替换) [中文pdf, pdf, html, 其他]
-
标题： 关于Whittle-Matérn过程在闭黎曼流形上的光滑度估计标题： Smoothness Estimation for Whittle-Matérn Processes on Closed Riemannian Manifolds期刊参考： 随机过程及其应用 189:104685, 2025主题： 统计理论 (math.ST)
Matérn 核函数族在空间统计、函数逼近以及机器学习中的高斯过程方法中经常被使用。它们之所以受欢迎的一个原因是存在一个平滑参数，该参数可以控制例如克里金最优误差界和高斯过程回归后验收缩率等性质。 在闭黎曼流形上，我们证明了当数据来源于高斯过程的点评估时，可以通过最大化高斯似然函数来一致估计平滑参数，并且出人意料的是，即使数据来源于非高斯过程的评估时也可以做到这一点。观测该过程的点无需具有任何特定的空间结构，只需满足准均匀性即可。 我们的方法基于索博列夫空间尺度的逼近理论结果。此外，我们通过使用角谷定理将与 Matérn 核相关的著名等价测度现象推广到非高斯情形。
The family of Mat\'ern kernels are often used in spatial statistics, function approximation and Gaussian process methods in machine learning. One reason for their popularity is the presence of a smoothness parameter that controls, for example, optimal error bounds for kriging and posterior contraction rates in Gaussian process regression. On closed Riemannian manifolds, we show that the smoothness parameter can be consistently estimated from the maximizer(s) of the Gaussian likelihood when the underlying data are from point evaluations of a Gaussian process and, perhaps surprisingly, even when the data comprise evaluations of a non-Gaussian process. The points at which the process is observed need not have any particular spatial structure beyond quasi-uniformity. Our methods are based on results from approximation theory for the Sobolev scale of Hilbert spaces. Moreover, we generalize a well-known equivalence of measures phenomenon related to Mat\'ern kernels to the non-Gaussian case by using Kakutani's theorem.
- [12] arXiv:2502.06765 (替换) [中文pdf, pdf, html, 其他]
-
标题： 所有的模型都是错误的吗？ 无分布假设下经验模型证伪的基本限制标题： Are all models wrong? Fundamental limits in distribution-free empirical model falsification评论： 39页，1幅图主题： 统计理论 (math.ST) ; 机器学习 (cs.LG) ; 机器学习 (stat.ML)
在统计学和机器学习中，当我们用可用数据训练一个拟合模型时，通常希望确保我们正在寻找的模型类中至少包含一个准确的模型——也就是说，我们希望确保模型类风险（该类中任何模型所能达到的最低可能风险）有一个上界。 然而，建立模型类风险的下界同样具有重要意义，例如，这样我们可以确定我们的拟合模型是否至少在该类中近似最优，或者决定该模型类是否不适合当前任务。 特别是在插值学习的背景下，即机器学习模型被训练以在训练数据上达到零误差时，我们可能会问，至少是否存在一个正的模型类风险下界——或者我们是否无法检测到“所有模型都是错误的”？ 在这项工作中，我们在无分布假设的情况下回答了这些问题，通过建立一个与模型无关的基本难度结果来探讨在模型类中构造最佳测试误差下界的难易程度，并研究其在特定模型类（如基于树的方法和线性回归）上的影响。
In statistics and machine learning, when we train a fitted model on available data, we typically want to ensure that we are searching within a model class that contains at least one accurate model -- that is, we would like to ensure an upper bound on the model class risk (the lowest possible risk that can be attained by any model in the class). However, it is also of interest to establish lower bounds on the model class risk, for instance so that we can determine whether our fitted model is at least approximately optimal within the class, or, so that we can decide whether the model class is unsuitable for the particular task at hand. Particularly in the setting of interpolation learning where machine learning models are trained to reach zero error on the training data, we might ask if, at the very least, a positive lower bound on the model class risk is possible -- or are we unable to detect that "all models are wrong"? In this work, we answer these questions in a distribution-free setting by establishing a model-agnostic, fundamental hardness result for the problem of constructing a lower bound on the best test error achievable over a model class, and examine its implications on specific model classes such as tree-based methods and linear regression.
- [13] arXiv:2212.02658 (替换) [中文pdf, pdf, html, 其他]
-
标题： 基于似然函数的无信息推断中的信息聚合标题： Pooling information in likelihood-free inference主题： 方法论 (stat.ME) ; 统计理论 (math.ST) ; 计算 (stat.CO)
似然无推断（LFI）方法，例如近似贝叶斯计算，已成为在复杂模型中进行推断的常用工具。许多方法基于从合成数据派生的摘要统计量或差异。然而，确定用于构建后验的哪些摘要统计量或差异仍然是一个具有实际和理论挑战性的问题。我们提出了一种新的池化后验方法，该方法最优地结合了来自多个LFI后验的推断，而不是依赖单一向量的摘要进行推断。这种方法消除了选择单一摘要向量甚至特定LFI算法的需求。我们的方法易于实现，并避免执行涉及所有摘要统计量的高维LFI分析。我们在理论上保证了池化后验均值在渐近频率风险方面的改进性能，并在多个基准示例中展示了该方法的有效性。
Likelihood-free inference (LFI) methods, such as approximate Bayesian computation, have become commonplace for conducting inference in complex models. Many approaches are based on summary statistics or discrepancies derived from synthetic data. However, determining which summary statistics or discrepancies to use for constructing the posterior remains a challenging question, both practically and theoretically. Instead of relying on a single vector of summaries for inference, we propose a new pooled posterior that optimally combines inferences from multiple LFI posteriors. This pooled approach eliminates the need to select a single vector of summaries or even a specific LFI algorithm. Our approach is straightforward to implement and avoids performing a high-dimensional LFI analysis involving all summary statistics. We give theoretical guarantees for the improved performance of the pooled posterior mean in terms of asymptotic frequentist risk and demonstrate the effectiveness of the approach in a number of benchmark examples.
- [14] arXiv:2212.09900 (替换) [中文pdf, pdf, 其他]
-
标题： 无重叠下的策略学习：悲观主义与广义的经验伯恩斯坦不等式标题： Policy learning "without" overlap: Pessimism and generalized empirical Bernstein's inequality评论： 发表于《统计年鉴》主题： 机器学习 (cs.LG) ; 统计理论 (math.ST) ; 方法论 (stat.ME) ; 机器学习 (stat.ML)
本文研究了离线策略学习问题，其目标是利用先验收集的观测数据（来自固定或自适应演化的行为策略），学习一个最优的个性化决策规则，以实现给定人群的最佳整体结果。 现有的策略学习方法依赖于均匀重叠假设，即所有个体特征探索所有行动的概率必须有下界。 由于无法控制数据收集过程，这一假设在许多情况下是不现实的，特别是当行为策略允许随着时间推移而演变，并且某些行动的倾向逐渐消失时。 本文提出了一种新的算法——悲观策略学习（PPL），该算法优化的是策略值的下置信边界（LCB），而不是点估计。 这些下置信边界通过了解用于收集离线数据的行为策略来构建。 在不假设任何均匀重叠条件的情况下，我们建立了我们算法次优性的数据依赖上界，该上界仅取决于（i）最优策略的重叠程度和（ii）我们优化的策略类的复杂性。 作为推论，在自适应收集的数据中，只要最优行动的倾向随时间有下界，而次优行动的倾向可以任意快速地消失，我们就能确保有效的策略学习。 在我们的理论分析中，我们开发了一种新的逆倾向加权估计量的自归一化类型集中不等式，将著名的经验伯恩斯坦不等式推广到非有界和非独立同分布的数据。 我们通过重大化最小化和策略树搜索提供了一个高效的优化算法，并进行了广泛的模拟研究和实际应用，以证明PPL的有效性。
This paper studies offline policy learning, which aims at utilizing observations collected a priori (from either fixed or adaptively evolving behavior policies) to learn an optimal individualized decision rule that achieves the best overall outcomes for a given population. Existing policy learning methods rely on a uniform overlap assumption, i.e., the propensities of exploring all actions for all individual characteristics must be lower bounded. As one has no control over the data collection process, this assumption can be unrealistic in many situations, especially when the behavior policies are allowed to evolve over time with diminishing propensities for certain actions. In this paper, we propose Pessimistic Policy Learning (PPL), a new algorithm that optimizes lower confidence bounds (LCBs) -- instead of point estimates -- of the policy values. The LCBs are constructed using knowledge of the behavior policies for collecting the offline data. Without assuming any uniform overlap condition, we establish a data-dependent upper bound for the suboptimality of our algorithm, which only depends on (i) the overlap for the optimal policy, and (ii) the complexity of the policy class we optimize over. As an implication, for adaptively collected data, we ensure efficient policy learning as long as the propensities for optimal actions are lower bounded over time, while those for suboptimal ones are allowed to diminish arbitrarily fast. In our theoretical analysis, we develop a new self-normalized type concentration inequality for inverse-propensity-weighting estimators, generalizing the well-known empirical Bernstein's inequality to unbounded and non-i.i.d. data. We complement our theory with an efficient optimization algorithm via Majorization-Minimization and policy tree search, as well as extensive simulation studies and real-world applications that demonstrate the efficacy of PPL.
- [15] arXiv:2501.09500 (替换) [中文pdf, pdf, html, 其他]
-
标题： 格规则与核立方积分相交标题： Lattice Rules Meet Kernel Cubature评论： 17页，2个图主题： 数值分析 (math.NA) ; 统计理论 (math.ST)
一维再生核Hilbert空间（RKHS）中具有平方可积的一阶混合偏导数的函数类，可以实现几乎线性收敛率。本文研究了用基于再生核得到的优化容积权重替换格子规则中的相等权重的影响。我们证明了一个理论结果，在一维情况下表明收敛率加倍，并且提供了高维情况下收敛率的数值研究。我们还展示了与带有随机系数的椭圆偏微分方程相关的不确定性量化问题的数值结果。
Rank-1 lattice rules are a class of equally weighted quasi-Monte Carlo methods that achieve essentially linear convergence rates for functions in a reproducing kernel Hilbert space (RKHS) characterized by square-integrable first-order mixed partial derivatives. In this work, we explore the impact of replacing the equal weights in lattice rules with optimized cubature weights derived using the reproducing kernel. We establish a theoretical result demonstrating a doubled convergence rate in the one-dimensional case and provide numerical investigations of convergence rates in higher dimensions. We also present numerical results for an uncertainty quantification problem involving an elliptic partial differential equation with a random coefficient.
- [16] arXiv:2506.03100 (替换) [中文pdf, pdf, html, 其他]
-
标题： 检索增强生成作为噪声上下文学习：一个统一的理论和风险界标题： Retrieval-Augmented Generation as Noisy In-Context Learning: A Unified Theory and Risk Bounds评论： 审阅中主题： 机器学习 (cs.LG) ; 人工智能 (cs.AI) ; 计算与语言 (cs.CL) ; 信息检索 (cs.IR) ; 统计理论 (math.ST)
基于检索的生成（RAG）近年来通过辅助大型语言模型（LLM）获取外部知识取得了许多经验上的成功。然而，其理论方面仍未得到充分探索。本文首次提出了针对上下文线性回归中的RAG的有限样本广义界，并推导出精确的偏差-方差权衡。我们的框架将检索到的文本视为与查询相关的噪声上下文示例，并将经典的上下文学习（ICL）和标准RAG作为极限情况恢复。我们的分析表明，与ICL相比，RAG存在一个内在的泛化误差上限。此外，通过引入均匀和非均匀RAG噪声，我们的框架能够建模从训练数据和外部语料库中进行检索。与我们的理论一致，我们通过在常见问答基准（如Natural Questions和TriviaQA）上的实验，实证展示了ICL和RAG的样本效率。
Retrieval-augmented generation (RAG) has seen many empirical successes in recent years by aiding the LLM with external knowledge. However, its theoretical aspect has remained mostly unexplored. In this paper, we propose the first finite-sample generalization bound for RAG in in-context linear regression and derive an exact bias-variance tradeoff. Our framework views the retrieved texts as query-dependent noisy in-context examples and recovers the classical in-context learning (ICL) and standard RAG as the limit cases. Our analysis suggests that an intrinsic ceiling on generalization error exists on RAG as opposed to the ICL. Furthermore, our framework is able to model retrieval both from the training data and from external corpora by introducing uniform and non-uniform RAG noise. In line with our theory, we show the sample efficiency of ICL and RAG empirically with experiments on common QA benchmarks, such as Natural Questions and TriviaQA.