https://arxiv.org/api/o3UGu9oBFttxfMo2f65LmFMo1bE 2026-03-20T10:47:03Z 9966 0 15 http://arxiv.org/abs/2603.19058v1 2026-03-19T15:50:38Z

Non-Gaussian statistics are a challenge for data assimilation. Linear methods oversimplify the problem, yet fully nonlinear methods are often too expensive to use in practice. The best solution usually lies between these extremes. Triangular measure transport offers a flexible framework for nonlinear data assimilation. Its success, however, depends on how the map is parametrized. Too much flexibility leads to overfitting; too little misses important structure. To address this balance, we develop an adaptation algorithm that selects a parsimonious parametrization automatically. Our method uses P-spline basis functions and an information criterion as a continuous measure of model complexity. This formulation enables gradient descent and allows efficient, fine-scale adaptation in high-dimensional settings. The resulting algorithm requires no hyperparameter tuning. It adjusts the transport map to the appropriate level of complexity based on the system statistics and ensemble size. We demonstrate its performance in nonlinear, non-Gaussian problems, including a high-dimensional distributed groundwater model.

2026-03-19T15:50:38Z 24 pages, 10 figures Berent Å. S. Lunde Maximilian Ramgraber http://arxiv.org/abs/2603.18846v1 2026-03-19T12:48:23Z

Foundation models are used to extract transferable representations from large amounts of unlabeled data, typically via self-supervised learning (SSL). However, many of these models rely on architectures that offer limited interpretability, which is a critical issue in high-stakes domains such as medical imaging. We propose Dual-IFM, a foundation model that is interpretable-by-design in two ways: First, it provides local interpretability for individual images through class evidence maps that are faithful to the decision-making process. Second, it provides global interpretability for entire datasets through a 2D projection layer that allows for direct visualization of the model's representation space. We trained our model on over 800,000 color fundus photography from various sources to learn generalizable, interpretable representations for different downstream tasks. Our results show that our model reaches a performance range similar to that of state-of-the-art foundation models with up to $16\times$ the number of parameters, while providing interpretable predictions on out-of-distribution data. Our results suggest that large-scale SSL pretraining paired with inherent interpretability can lead to robust representations for retinal imaging.

2026-03-19T12:48:23Z 11 pages, 3 figures, 2 tables, submitted to MICCAI 2026 Samuel Ofosu Mensah Maria Camila Roa Carvajal Kerol Djoumessi Philipp Berens http://arxiv.org/abs/2603.18845v1 2026-03-19T12:46:17Z

Although Hamiltonian Monte Carlo (HMC) scales as O(d^(1/4)) in dimension, there is a large constant factor determined by the curvature of the target density. This constant factor can be reduced in most cases through preconditioning, the state of the art for which uses diagonal or dense penalized maximum likelihood estimation of (co)variance based on a sample of warmup draws. These estimates converge slowly in the diagonal case and scale poorly when expanded to the dense case. We propose a more effective estimator based on minimizing the sample Fisher divergence from a linearly transformed density to a standard normal distribution. We present this estimator in three forms, (a) diagonal, (b) dense, and (c) low-rank plus diagonal. Using a collection of 114 models from posteriordb, we demonstrate that the diagonal minimizer of Fisher divergence outperforms the industry-standard variance-based diagonal estimators used by Stan and PyMC by a median factor of 1.3. The low-rank plus diagonal minimizer of the Fisher divergence outperforms Stan and PyMC's diagonal estimators by a median factor of 4.

2026-03-19T12:46:17Z Adrian Seyboldt Eliot L. Carlson Bob Carpenter http://arxiv.org/abs/2603.18833v1 2026-03-19T12:35:09Z

Sparse functional data arise when measurements are observed infrequently and at irregular time points for each subject, often in the presence of measurement error. These characteristics introduce additional challenges for functional principal component analysis. In this paper, we propose a new approach for extracting functional principal components from such data by combining basis expansion with maximum likelihood estimation. Orthogonality of the estimated eigenfunctions is preserved throughout the optimization using modified Gram-Schmidt orthonormalization. An information criterion is proposed to select both the optimal number of basis functions and the rank of the covariance structure. Principal component scores are subsequently estimated via conditional expectation, enabling accurate reconstruction of the underlying functional trajectories across the full domain despite sparse observations. Simulation studies demonstrate the effectiveness of the proposed method and show that it performs favorably compared with existing approaches. Its practical utility is illustrated through applications to CD4 cell count data from the Multicenter AIDS Cohort Study and somatic cell count data from Irish research dairy cattle. Supplementary materials, including technical details, additional simulation results, and the R package mGSFPCA, are available online.

2026-03-19T12:35:09Z Uche Mbaka University College Dublin Jiguo Cao Simon Fraser University Michelle Carey University College Dublin http://arxiv.org/abs/2602.18328v2 2026-03-19T10:28:23Z

We consider Bayesian inverse problems arising in data assimilation for dynamical systems governed by partial and stochastic partial differential equations. The space-time dependent field is inferred jointly with static parameters of the prior and likelihood densities. Particular emphasis is placed on the hyperparameter controlling the prior smoothness and regularity, which is critical in ensuring well-posedness, shaping posterior structure, and determining predictive uncertainty. Commonly it is assumed to be known and fixed a priori; however in this paper we will adopt a hierarchical Bayesian framework in which smoothness and other hyperparameters are treated as unknown and assigned hyperpriors. Posterior inference is performed using Metropolis-within-Gibbs sampling suitable to high dimensions, for which hyperparameter estimation involves little computational overhead. The methodology is demonstrated on inverse problems for the Navier-Stokes equations and the stochastic advection-diffusion equation, under sparse and dense observation regimes, using Gaussian priors with different covariance structure. Numerical results show that jointly estimating the smoothness substantially reduces the errors in uncertainty quantification and parameter estimation induced by smoothness misspecification, by achieving performance comparable to scenarios in which the true smoothness is known.

2026-02-20T16:33:22Z 28 pages, 11 figures Baptiste Simandoux Nikolas Kantas Dan Crisan http://arxiv.org/abs/2507.23240v2 2026-03-19T01:27:27Z

Designing efficient experiments under practical constraints is critical in both scientific research and industrial practice. Focusing on minimizing the average variance of the parameter estimates, A-optimal designs show advantages in screening factors and reducing prediction errors. Compared with other criteria, however, algorithms and software for generating A-optimal designs are scarce. In this paper, we characterize A-optimal designs under generalized linear models theoretically and develop efficient algorithms for identifying them. When a predetermined finite set of experimental settings is given, we derive analytic solutions or establish necessary and sufficient conditions for obtaining A-optimal approximate allocations. We show that a lift-one algorithm based on our formulae outperforms commonly used algorithms for finding A-optimal allocations. When continuous factors or design regions get involved, we develop a ForLion algorithm that is guaranteed to find A-optimal designs with mixed factors. Numerical studies show that our algorithms can find highly efficient designs with reduced numbers of distinct experimental settings, which may save both experimental time and cost significantly. Along with a rounding-off algorithm that converts approximate allocations to exact ones, we demonstrate that stratified samplers based on A-optimal allocations may provide more accurate parameter estimates than commonly used samplers.

2025-07-31T04:40:22Z 34 pages, 2 figure, 9 tables Yingying Yang Xiaotian Chen Jie Yang http://arxiv.org/abs/2603.18324v1 2026-03-18T22:14:55Z

Geostatistics is a branch of statistics concerned with stochastic processes over continuous domains, with Gaussian processes (GPs) providing a flexible and principled modelling framework. However, the high computational cost of simulating or computing likelihoods with GPs limits their scalability to large datasets. This paper introduces the piecewise continuous Gaussian process (PCGP), a new process that retains the rich probabilistic structure of traditional GPs while offering substantial computational efficiency. As will be shown and discussed, existing scalable approaches that define stochastic processes on continuous domains -- such as the nearest-neighbour GP (NNGP) and the radial-neighbour GP (RNGP) -- rely on conditional independence structures that effectively constrain the measurable space on which the processes are defined, which may induce undesirable probabilistic behaviour and compromise their practical applicability, particularly in complex latent GP models. The PCGP mitigates these limitations and provides a theoretically grounded and computationally efficient alternative, as demonstrated through numerical illustrations.

2026-03-18T22:14:55Z Flávio B. Gonçalves Marcos O. Prates Gareth O. Roberts http://arxiv.org/abs/2603.18201v1 2026-03-18T18:53:46Z

Artificial Intelligence (AI) systems are increasingly prominent in emerging smart cities, yet their reliability remains a critical concern. These systems typically operate through a sequence of interconnected functional stages, where upstream errors may propagate to downstream stages, ultimately affecting overall system reliability. Quantifying such error propagation is essential for accurate modeling of AI system reliability. However, this task is challenging due to: i) data availability: real-world AI system reliability data are often scarce and constrained by privacy concerns; ii) model validity: recurring error events across sequential stages are interdependent, violating the independence assumptions of statistical inference; and iii) computational complexity: AI systems process large volumes of high-speed data, resulting in frequent and complex recurrent error events that are difficult to track and analyze. To address these challenges, this paper leverages a physics-based autonomous vehicle simulation platform with a justifiable error injector to generate high-quality data for AI system reliability analysis. Building on this data, a new reliability modeling framework is developed to explicitly characterize error propagation across stages. Model parameters are estimated using a computationally efficient, theoretically guaranteed composite likelihood expectation - maximization algorithm. Its application to the reliability modeling for autonomous vehicle perception systems demonstrates its predictive accuracy and computational efficiency.

2026-03-18T18:53:46Z 42 pages, 11 figures Fenglian Pan Yinwei Zhang Yili Hong Larry Head Jian Liu http://arxiv.org/abs/2505.17300v2 2026-03-18T14:52:57Z

The construction of confidence intervals and hypothesis tests for functionals is a cornerstone of statistical inference. Traditionally, the most efficient procedures - such as the Wald interval or the Likelihood Ratio Test - require both a point estimator and a consistent estimate of its asymptotic variance. However, when estimators are derived from online or sequential algorithms, computational constraints often preclude multiple passes over the data, complicating variance estimation. In this article, we propose a computationally efficient, rate-optimal wrapper method (HulC) that wraps around any online algorithm to produce asymptotically valid confidence regions bypassing the need for explicit asymptotic variance estimation. The method is provably valid for any online algorithm that yields an asymptotically normal estimator. We evaluate the practical performance of the proposed method primarily using Stochastic Gradient Descent (SGD) with Polyak-Ruppert averaging. Furthermore, we provide extensive numerical simulations comparing the performance of our approach (HulC) when used with other online algorithms, including implicit-SGD and ROOT-SGD.

2025-05-22T21:31:49Z 1) Adding to ASGD simulations, we add 5 other SGD algorithms: averaged-implicit-SGD, last-iterate-implicit-SGD, ROOT-SGD, truncated-SGD, and noisy-truncated-SGD. 2) We modify links to the online viz/GitHub pages. 3) We qualify previous conclusions on ASGD: ex, we claim that logistic regression is sometimes more challenging "in terms of achieving the target coverage" than linear regression Selina Carter Arun K Kuchibhotla http://arxiv.org/abs/2507.00641v3 2026-03-18T08:55:11Z

Physical transport processes organize through local interactions that redistribute imbalance while preserving conservation. Classical solvers enforce this organization by applying fixed discrete operators on rigid grids. We introduce the Hebbian Physics Network (HPN), a computational framework that replaces this rigid scaffolding with a plastic transport geometry. An HPN is a coupled dynamical system of physical states on nodes and constitutive weights on edges in a graph. Residuals--local violations of continuity, momentum balance, or energy conservation--act as thermodynamic forces that drive the joint evolution of both the state and the operator (i.e. the adaptive weights). The weights adapt through a three-factor Hebbian rule, which we prove constitutes a strictly local gradient descent on the residual energy. This mechanism ensures thermodynamic stability: near equilibrium, the learned operator naturally converges to a symmetric, positive-definite form, rigorously reproducing Onsagerś reciprocal relations without explicit enforcement. Far from equilibrium, the system undergoes a self-organizing search for a transport topology that restores global coercivity. Unlike optimization-based approaches that impose physics through global loss functions, HPNs embed conservation intrinsically: transport is restored locally by the evolving operator itself, without a global Poisson solve or backpropagated objective. We demonstrate the framework on scalar diffusion and incompressible lid-driven cavity flow, showing that physically consistent transport geometries and flow structures emerge from random initial conditions solely through residual-driven local adaptation. HPNs thus reframe computation not as the solution of a fixed equation, but as a thermodynamic relaxation process where the constitutive geometry and physical state co-evolve.

2025-07-01T10:34:14Z 16 pages, 3 figures Gunjan Auti Hirofumi Daiguji Gouhei Tanaka 10.1103/tzgk-jqj4 http://arxiv.org/abs/2603.17466v1 2026-03-18T08:19:41Z

The goal of this study is to introduce a unified computational framework for simulating random iteration equations (RIE), understood as iteration equations containing random variables. The novelty of this work is that full probability densities of the state vectors are propagated stepwise through the iterations avoiding the need of repetitive pathwise Monte Carlo simulations of the iteration equation. The presentation of the methodology is conceptually efficient based on recent work on static random equations and intentionally accessible. The technical requirements on the RIE are minimal based on the previous work, allowing for potential nonlinearities, discontinuities and stochasticities in the transfer function, as well as nonstandard densities and diffusion processes. As results, illustrative applications of random and stochastic differential equation simulations, a novel full-density gradient descent method (FDGD) for global optimization under uncertainty and examples of chaotic mappings are presented in order to demonstrate the breadth of the utility of this framework. In total, the character of the presentation is explorative and encourages new applications and theoretical studies.

2026-03-18T08:19:41Z Wolfgang Hoegele http://arxiv.org/abs/2603.14801v2 2026-03-18T03:51:31Z

Many statistical problems involve optimization over a discrete parameter space having an unknown dimension. In such settings, gradient-based methods often fail due to the non-differentiability of the objective function or a non-convex or massive search space with an objective function having many local maxima/minima. This paper presents GAReg, a unified genetic algorithm package that handles discrete optimization regression problems, which works well when standard algorithms are unjustified. GAReg provides a compact chromosome representation supporting optimal knot placement for regression splines, best-subset regression variable selection, and related problems. The package allows for uniform initialization, constraint-preserving crossover and mutation, steady-state replacement, and an optional island-model parallelization. GAReg efficiently searches high-dimensional model spaces, providing near-optimal solutions in settings where exhaustive enumeration or integer or dynamic programming approaches are infeasible.

2026-03-16T03:59:21Z Mo Li QiQi Lu Robert Lund Xueheng Shi http://arxiv.org/abs/2404.12589v6 2026-03-18T02:46:36Z

This paper analyzes the factorizability and geometry of transition matrices of multivariate Markov chains. Specifically, we demonstrate that the induced chains on factors of a product space can be regarded as information projections with respect to the Kullback-Leibler divergence. This perspective yields Han-Shearer type inequalities and submodularity of the entropy rate of Markov chains, as well as applications in the context of large deviations and mixing time comparison. As concrete algorithmic applications in Markov chain Monte Carlo (MCMC) and approximate inference, we provide three illustrations based on lifted MCMC, swapping algorithm and factored filtering to demonstrate projection samplers improve mixing over the original samplers. The projection sampler based on the swapping algorithm resamples the highest-temperature coordinate at stationarity at each step, and we prove that such practice accelerates the mixing time by multiplicative factors related to the number of temperatures and the dimension of the underlying state space when compared with the original swapping algorithm. Through simple numerical experiments on a bimodal target distribution, we show that the projection samplers mix effectively, in contrast to lifted MCMC and the swapping algorithm, which mix less well. In filtering, our proposed factored filtering scheme is able to scale to high dimensions with linear-in-dimension computational cost per step at the price of an approximation error that can be tracked using the distance to independence, compared with the exponential-in-dimension cost per step of the exact filter.

2024-04-19T02:35:03Z 45 pages, 6 figures Michael C. H. Choi Youjia Wang Geoffrey Wolfer http://arxiv.org/abs/2603.16756v1 2026-03-17T16:36:24Z

In many scientific and engineering domains, physical experiments are often costly, non-replicable, or time-consuming. The Kennedy and O'Hagan (KOH) model framework has become a widely used approach for combining simulator runs with limited experimental observations. Under a Bayesian implementation, the simulator output, model discrepancy, and observation noise are jointly modeled by coupled Gaussian processes, followed by coherent posterior inference and uncertainty quantification. This work presents a genuinely sequential Bayesian experimental design (BED) framework explicitly aimed at improving the predictive performance of the KOH model. We employ a mutual information (MI)-based criterion and develop a hybrid variant that integrates it with measures of local model complexity, leading to significantly more efficient design decisions. We further show theoretically that the MI-based criterion is more comprehensive and robust than the classical integrated mean squared prediction error (IMSPE) minimization criterion, especially when the model is highly uncertain in the early stages of the experiment. To mitigate the computational burden of fully Bayesian inference and the ensuing BED process, we propose two acceleration strategies - Gaussian Mixture Compression and Schur complement and rank-one update - which together substantially reduce runtime. Finally, we demonstrate the effectiveness of the proposed methods through both a synthetic example and a real biochemical case study, and compare them against several classical design criteria under sequential (offline) and adaptive (online) BED settings.

2026-03-17T16:36:24Z Accepted for presentation at the SIAM Conference on Uncertainty Quantification (UQ26), March 22-25, 2026, Minneapolis, USA Hao Zhu Markus Hainy http://arxiv.org/abs/2512.07709v2 2026-03-17T11:35:03Z

We develop a unified nonparametric framework for sharp partial identification and inference on inequality indices when the data contain coarsened observations of the variable of interest. We characterize the extremal allocations for all Schur-convex inequality measures, and show that sharp bounds are attained by distributions with finite support. This reduces the computational problem to finite-dimensional optimization, and for indices admitting linear-fractional representations after suitable ordering of the data (including the Gini coefficient and quantile ratios), we express the bound problems as linear or quadratic programs. We then establish $\sqrt{n}$ inference for the upper and lower bounds using a directional delta method and bootstrap confidence intervals. In applications, we compute sharp Gini bounds from household wealth data with mixed point and interval observations and use historical U.S. grouped income tables to bound time series for the Gini and quantile ratios.

2025-12-08T16:55:38Z James Banks Thomas Glinnan Tatiana Komarova