Measuring Kaplan--Meier Overestimation Under Competing Risks: Event Ordering, Quantification, and Exact Inference

2026-11-10T18:26:33Z

Competing risks arise when individuals may experience multiple mutually exclusive event types. Competing events are frequently treated as censoring and analysed using Kaplan--Meier (KM) methods, which overestimate real-world event risks. While the existence of KM overestimation is well known, its determinants, magnitude, and inferential properties remain poorly characterised. We develop a nonparametric framework that treats competing risks overestimation as a statistical quantity in its own right. We show that competing event frequency alone is insufficient to characterise KM overestimation and develop an event ordering theory describing how temporal event allocation influences overestimation. Based on these results, we derive a closed-form Total Overestimation Diagnostic (TOD) that quantifies total KM overestimation directly from routine survival analysis output. We establish exact error equivalence results for both cause-specific and total overestimation. In particular, Product Limit Error Equivalence shows that inference for TOD reduces exactly to ordinary overall survival estimation. Simulation experiments validate the theoretical results and show that independent censoring primarily affects precision rather than overestimation magnitude. Across six benchmark competing risks datasets, cause-specific overestimation is characterised by both total overestimation and temporal event allocation. These results provide a practical framework for assessing competing risks relevance and determining when cumulative incidence methods should be preferred to naïve KM risks. The proposed framework transforms competing risks overestimation from a qualitative warning into a measurable quantity that can be routinely quantified, interpreted, and incorporated into reporting practice.

Cotangent-lift inference of stochastic microbial dynamics

2026-11-10T18:00:26Z

We study the parameter estimation for the stochastic logistic and Lotka--Volterra diffusions from discrete, irregularly spaced observations corrupted by measurement noise, a setting typical of microbiome time series. For the logistic equation, the growth rate acts as a time change and does not enter the stationary law, so it can be identified only from the temporal structure of the data. We prove that the maximum-likelihood estimator of the Euler scheme, with the observations taken as the states, coincides with the regression of log-increments on abundances used in the literature, and we characterise what this estimator cannot see under measurement noise: for the linearised dynamics, the law of consecutive pairs of observations is invariant along an explicit one-parameter family of relaxation rates and noise levels, and the regression converges to a limit set by the noise. Treating the true abundances as latent, we compute the likelihood with a Gaussian moment filter and show that measurement noise is precisely what couples the filter states across samples. The exact gradient is then the backward recursion of the discrete cotangent lift, whose cost, two to three likelihood evaluations, does not depend on the number of parameters, and we give the covector map of each filter step in closed form. In simulations, the latent estimator recovers the relaxation times, noise levels and interaction strengths that the regression biases, while larger communities require prior information on the interactions. In gut microbiome time series it attributes most of the variability of a dominant genus to measurement noise and estimates its relaxation time at about $64$ hours, with a large standard error.

A Bayesian approach for checking and estimating block designs in agricultural experiments

2026-11-10T17:24:41Z

Spatial blocking is a common technique used in designed agricultural experiments. Spatial blocking, however, requires important assumptions about the spatial dependence patterns in the field. Statistical models of blocked agricultural experiments often show spatial structure in the residuals, suggesting that the original model may benefit from additional assumptions regarding the spatial dependence patterns present in the field. We propose a Bayesian model for checking block membership and estimating spatial patterns missing in the original block design. Our model is particularly motivated by cases with mis-specified blocks. To accomplish this, we embed a regression tree within a Bayesian statistical model to estimate spatial effects with a block-like spatial structure. We illustrate the benefits of our model using simulated data mimicking four typical scenarios we have encountered in our applied experience in agricultural research and using two datasets generated from designed agricultural experiments. We found that our model may serve two purposes: validating the originally designed block layout, or mis-specified block-like spatial structures. Thus, our model provides researchers with a novel tool for analyzing data from designed agricultural experiments.

Probabilistic Upscaling of the Interfacial Transition Zone Behaviour in Concrete

2026-11-10T14:52:48Z

The stiffness and cracking behaviour of concrete are influenced by the interfacial transition zone (ITZ), the thin shell of paste that surrounds each aggregate. Yet the ITZ's mechanical properties are poorly characterized and are usually assigned to mesoscale models through ad hoc reduction factors. Because the ITZ is only tens of micrometres thick, its stiffness must be inferred indirectly from micromechanical tests, and the resulting estimates vary widely between studies and are rarely reported with a quantified uncertainty. We present a workflow that calibrates a homogenized ITZ model against the surface displacements of several micro-cantilever bending specimens and returns its stiffness descriptors as posterior distributions rather than single values. The calibration is posed as a Bayesian inverse problem with a model-discrepancy term, sampled with a surrogate-assisted Bouncy Particle sampler, the first application of this sampler to a real-world engineering problem. We compare a pooled model, which fits one descriptor vector to all specimens, with a hierarchical model that treats the specimens as draws from a common population and is sampled with a local extension of the sampler. The hierarchical model separates the estimation uncertainty from the aggregate-to-aggregate material variability, where only the former shrinks as more data are added. Propagated through a representative volume element of several aggregates, its predictive yields broader distributions of the mesoscale stress and strain and a wider spread variety of maximum von Mises stress locations. The calibration places the ITZ thickness above the 30 μm to 80 μm commonly assumed in the literature.

How much can a human being lift? A Bayesian latent capability model for Olympic weightlifting

2026-11-10T13:01:06Z

Estimates of the limits of human performance are usually built on record books, on the premise that a record is the best an athlete could do. In Olympic weightlifting the premise fails, because the competitor declares the weight before each of three attempts per lift and only the heaviest successful attempt is recorded, so a record is a lower bound, chosen by the athlete, on a maximal capability never observed. We treat the maximal lift of each competitor on each competition day as a latent variable, of which every attempt is a trial and every declared weight a measurement. Its distribution at a given bodyweight follows an extreme value mixture model with an estimated threshold, and an athlete-specific random effect links the competitions of each athlete. The model is fitted to the 117,892 attempts of 5,040 athletes in 141 international competitions between 2017 and 2026. The upper tail of capability is bounded, with posterior probability above 0.999 for men, 0.96 for the women's snatch and 0.94 for the women's clean and jerk. At 110 kilograms of bodyweight the men's ceiling lies one to three per cent above the heaviest lifts, against twelve and sixteen per cent for women at 86 kilograms. The record book alone places the men's ceiling six and twelve kilograms higher, because the best lift of a day ending on a success overstates capability.

Testing isotropy of random fields with values in abstract spaces

2026-11-10T12:50:49Z

We study the problem of testing isotropy, i.e., distributional invariance under orthogonal transformations, for random fields taking values in abstract spaces. We propose a novel approach based on computing local directional characteristics, which are uniformly distributed under the assumption of isotropy. Our work is motivated by an application in material science, where we are testing the isotropy of function-valued porosity profiles of paper sheets, whose morphology is resolved by $μ$-CT imaging. While considerably more general, the presented methodology is competitive with existing isotropy tests for real-valued random fields.

DynamAuction: A Reinforcement Learning Environment for Repeated Auctions with Dynamic Value

2026-11-10T12:43:58Z

A buyer who repeatedly participates in auctions for related items often faces a dynamic value problem: what they win today changes what tomorrow's items are worth. Despite the empirical importance of such effects, especially in display advertising, the algorithmic literature on repeated auctions has largely focused on the learning problem, by treating each opportunity as a contextual bandit, while sidestepping the planning problem induced by the dependence of current value on past wins. We introduce DynamAuction, a synthetic environment for single-buyer sequential bidding. The reward from each win depends on the buyer's history of previous wins, summarized through a carry-over state. The environment supports first- and second-price auctions, non-homogeneous Poisson arrivals, delayed feedback, saturation, and recency effects. We also introduce a new class of Model Predictive Control (MPC) algorithms. On our benchmark scenario, these MPC policies outperform off-the-shelf deep RL baselines (PPO, SAC, TD3) as well as a tuned shading baseline. We release DynamAuction to support RL research on repeated auctions.

Deceleration without identification: testing a power-law trend in Bitcoin prices under persistent noise

2026-11-10T11:56:26Z

Plotted against time since its first block on logarithmic axes, the price of Bitcoin lies close to a straight line, and a power law in time has been proposed as its long-run trend. A good fit is weak evidence: when deviations from a trend are as persistent as a random walk, regressions on time find trends and high $R^2$ where none exist. We first bound what fifteen years of weekly prices can reveal, then test only what is answerable. Using a few low-frequency averages, invariant to the noise scale and calibrated over a declared family of persistent deviations with a four-year cycle and declining volatility, we test constant drift in expected log price against an added log-age term. The procedure was fixed, validated and publicly hashed before it was applied once. Constant drift is rejected under this family at nominal 5%, and a log-age coefficient of 5.7, close to the published exponent, is not rejected. The rejection does not survive allowing persistence beyond a random walk, and a Gompertz curve with a ceiling fits about as well as the power law. Whether deviations revert to the trend cannot be decided. Registered forecasts for 2027-2030 will be scored by future prices.

Recursive memory for prediction and generation

2026-11-10T09:00:36Z

Recurrent and state-space language models compress the past into a state of fixed size. The state must answer questions about the future now, and it must remain useful as new tokens arrive, including tokens the model samples itself. We show that the second requirement can cost far more memory than the first. For a known hidden Markov source with positive transition probabilities, a state of $O(\log(1/\varepsilon))$ bits answers every question about the future to accuracy $\varepsilon$, however far ahead it looks, because the Bayesian filter forgets its rounding errors. Generating the future is different. When a model feeds each sampled token back into its state, the small errors in its next-token distributions add up, and $T$ tokens can require a state of order $\log(T/\varepsilon)$ bits. The lower bound comes from the KL chain rule: at every step, a model with $K$ states pays for approximating the true next-token distribution by one of only $K$ distributions. The argument applies to any source, including a language model continuing a prompt, and a small-ball estimate makes it tight for two-state sources. The extra $\log T$ disappears if the model keeps private randomness, or if error is budgeted per token, as in perplexity. Statistics of the recent past show the same gap between describing and maintaining. The number of ones among the last $n$ bits takes $\log n$ bits to write down. But if each bit is a noisy copy of the one $n$ steps earlier, tracking that count to a mean-square error $E^2$ well below its variance costs order $n/(1+E^2)+\log n$ bits. Information about each bit must be kept until the bit leaves the window, and a queue holding the counts of recent blocks keeps just enough of it.

Fractional-Order Hybrid SIR Forecasting of COVID-19 in Bangladesh and the United States: Memory Effects and Out-of-Sample Evaluation

2026-11-10T05:02:33Z

Fractional-order epidemic models are often justified by memory, yet whether memory improves forecasts is rarely tested out of sample. We extend a hybrid SIR framework, in which a daily transmission rate is reconstructed from reported cases and then forecast, to Caputo fractional order $α\in(0,1]$. We prove well-posedness, conservation, positivity and a linearized threshold at disease-free states, and derive a Grünwald-Letnikov reconstruction that reduces exactly to the integer-order one at $α=1$. Our main theoretical result shows that for $α<1$ the influence of the unknown initial infectious count vanishes, but no faster than $t^{-α}$; after 332 daily steps it is about $4.4\times10^{8}$ times larger at $α=0.9$ than at $α=1$. In out-of-sample forecasting over 27 weekly origins, the effect of memory depends on the setting. In Bangladesh the integer order gives the lowest error for linear and constant-rate forecasts in 47 of 48 settings. In the United States, where reported cases reached about 18% of the population, a fractional order lowers the linear forecaster's error in all 12 comparisons in 2021 but raises the constant-rate forecaster's, with no difference significant after Holm adjustment. Against standard baselines, the SIR-based linear forecaster was more accurate than the classical SIR model and a neural network without SIR structure, and competitive with ARIMA: in the United States the fractional linear forecaster had the lowest error of all methods (seven-day MAPE 7.54% against 9.14% for ARIMA), whereas in Bangladesh ARIMA had the lowest MAPE, with no significant difference from ARIMA in either country. The choice of a fractional order should rest on out-of-sample evidence rather than in-sample fit.

Sufficient Dimension Reduction with Multimodal Predictor Objects

2026-11-10T02:07:53Z

We study the regression of a scalar response $Y$ on a Euclidean predictor $\mathbf{X} \in \mathbb{R}^p$ and a random predictor object $\mathcal{Z}$ in a general metric space under arbitrary structural dependence. We introduce a joint sufficient dimension reduction $(\mathbf{B}^\top\mathbf{X}, \mathcal{G}_Z)$ constructed via a sequential conditional independence framework, which differs fundamentally from the ordinary central subspace of $Y \mid \mathbf{X}$. Methodologically, $\mathcal{Z}$ is embedded into a reproducing kernel Hilbert space, and a regularized kernel-response envelope model extracts a finite-dimensional reduction of the feature map. Partial adaptive estimation then recovers the Euclidean reduction conditional on the first-stage coordinates. We establish root-$n$ consistency rates for the first-stage envelope subspace under a shrinking regularizer, alongside large-sample consistency for the second-stage Euclidean subspace estimator. Simulations and two real-data applications illustrate the utility and flexibility of the proposed method.

Economics of Selection in Ranking & Selection: Neither Fixed-Precision nor Fixed-Budget

2026-11-10T01:52:43Z

Ranking and selection (R&S) has evolved tremendously over the last several decades, with applications in agriculture, discrete optimization with computer simulation, clinical trial design, and other areas. We challenge a widely accepted classification of R&S procedures into two categories: fixed-precision and fixed-budget. We discuss a third class, economics-of-selection (EoS) procedures, which is conceptually distinct and is found in the extant literature on Bayesian hypothesis testing and simulation optimization. Procedures in this third class seek to optimally balance two quantities: the cost of learning about the performance of each alternative and the quality of the alternative ultimately selected on the basis of that learning. Fixed-precision and fixed-budget procedures tend to optimize one of these quantities while constraining the other. We clarify these differences and show why they matter. We then present new frequentist R&S procedures in this third category, including both two-stage procedures and a sequential heuristic, assuming Gaussian observations with either known or unknown variances. We find that EoS is not a question of Bayesian versus frequentist analysis, asymptotic versus finite-sample analysis, finite-stage versus fully sequential procedures, or comparing one mean with a standard versus comparing multiple alternatives. We identify opportunities for further R&S research.

Bayesian Multinomial Logistic Normal Additive Regression for Microbiome Data Analysis

2026-11-10T01:16:40Z

Longitudinal microbiome studies often seek to separate background temporal variation from nonlinear perturbation-associated dynamics while accounting for the count-compositional nature of sequencing data. We introduce \textit{MultiAddGPs}, a semi-parametric Bayesian multinomial logistic-normal framework that combines linear effects with additive Gaussian process components. We first prove that MultiAddGPs remain within the Marginally Latent Matrix-T Process class (MLTP) and derive the collapsed and uncollapsed representations needed for scalable inference with the Collapse--Uncollapse (CU) sampler. We then develop a back-sampling procedure for efficient posterior inference on the individual additive components. We further develop approximate maximum marginal likelihood and penalized maximum marginal likelihood procedures for hyperparameter estimation in this broader class of models. In simulations, MultiAddGPs provides accurate posterior inference and calibrated uncertainty while scaling more effectively than existing GP-based additive alternatives. Applications to two longitudinal microbiome studies demonstrate how the model separates background temporal variation from intervention-associated dynamics, including travel-associated perturbations in a human gut time series and starvation effects in an artificial gut system. Our methods are implemented in the \textit{fido} package on CRAN.

Reconciling District Prediction Markets and Seat-Count Buckets in U.S. Midterm Elections

2026-11-10T01:13:53Z

The prediction market platform Kalshi offers individual markets for the winner in each U.S. congressional district and an aggregate market for the total number of Democratic House seats. We develop an applied statistical framework for reconciling these two market layers, since the aggregate seat count is determined by the collection of individual district outcomes. The framework combines a latent-Gaussian dependence structure with a log-odds adjustment to district probabilities and a power adjustment to aggregate seat-count probabilities, with parameters estimated by divergence minimization. Applied to a July 30, 2026 snapshot, the three-parameter primary model achieves close reconciliation, with forward and reverse KL divergences both approximately 0.0013, while yielding a positive latent national correlation, a negative district log-odds shift, and flattening of the aggregate target distribution. Daily analysis of 236 dates from January through August 2026 reveals two relatively stable periods separated by a spring transition period exhibiting substantial volatility in both parameter and divergence values. The framework provides a diagnostic approach for assessing cross-market consistency in hierarchically structured prediction or financial markets.

Orthographic depth at two scales, with English also matched for lexicon size

2026-11-10T00:28:23Z

A reader can struggle to read in only one of their languages, and one proposed reason is how predictably spelling maps to sound. We measure that predictability as conditional entropy, the pronunciation uncertainty left once the spelling is seen, at two grain sizes (unit sizes): the alignment unit, an aligned spelling-sound chunk, and the syllable, bounded in the pronunciation. We analyse word lists in twelve alphabetic orthographies and report five in full, with English pronunciations from a dictionary and the rest from the synthesizer eSpeak-ng, length marks removed. Among the five, English has the most uncertainty and Finnish the least at both grains. Across all twelve, Portuguese and Danish exceed English per alignment unit, and German and Italian order differently by grain and transcription. These orders hold when English is transcribed by eSpeak-ng with u allowed to spell /j/ plus a vowel. Uncertainty per unit has no shared direction as units grow: it falls in Spanish and, less securely, German, is level in Finnish, depends on transcription in Italian and on the word list in English, and rises in Norwegian and Swedish. Summed uncertainty per word falls in all twelve, but that score is handed the pronunciation's boundaries and mixes context with in-sample memorisation, so it is no evidence of what a reader computes. An exact identity states when discarded spelling boundaries could reverse that fall. A uniform 20,000-word English sample holds 35% of the dictionary's syllable spellings and has syllable entropy 0.098 bits lower, so lexicons must be matched in size. In an exploratory check over eleven languages, syllable-scale entropy tracked first-year reading accuracy (rho = -0.74, falling to -0.25 if English is given the lowest entropy instead), without separating the syllable scale from the alignment unit. These estimates describe sampled writing systems, not readers.