https://arxiv.org/api/0sItsfNY+OZ6Ii0JgxdByBrdpvk 2026-11-12T03:27:34Z 2841 0 15 http://arxiv.org/abs/2611.00301v2 2026-11-09T22:10:21Z

Floating point computing has dominated scientific computations since the 1950s because of its versatile ability to approximate real numbers with both sufficiently high precision and sufficiently great range. Since 1985, two specific formats have become standard across the industry, one with 32 bits which is adequate for many purposes and one with 64 bits which allows numbers with both much higher precision and much greater range for applications where these are needed. However, during the past decade, deep learning has come to dominate computation, and narrower floating point formats with lower precision and reduced range have steadily become more common. In this article we discuss this phenomenon, why it is happening, how various 16-bit, 8-bit and even 4-bit floating point formats have been developed, and efforts to standardize these formats.

2026-10-30T19:23:40Z Guy Lemieux Michael L. Overton http://arxiv.org/abs/2610.24074v2 2026-11-09T18:49:43Z

The Julia package QRecoupling evaluates classical and quantum recoupling coefficients, such as Clebsch-Gordan coefficients and $3j$ and $6j$ symbols, and user-defined sums of factorial products, with numerical, exact and symbolic targets. One compact factorial rule covers classical and deformed evaluations, while its cyclotomic exponents reveal common factors and determine which terms contribute at unitary levels. Adaptive arithmetic resolves strongly cancelling sums, while exact tests in quantum integers distinguish algebraic zeros from small nonzero values without a computer-algebra system. At spin cutoff 500, classical $3j$ and $6j$ symbols are evaluated about 7 and 15 times faster than with WIGXJPF. For the tested quantum deformations, relative errors remain below $4\times10^{-14}$ through spin cutoff 400. Recurrences compute fusion matrices with 401 channels about 100 times faster than loops over their entries. The resulting coefficients support recoupling identities, changes of coupling basis and tensor-network calculations.

2026-10-16T18:01:10Z 21 pages, 4 figures, 5 tables. Software: https://github.com/sethkasante/QRecoupling.jl (v0.6.4, doi:10.5281/zenodo.24027544) Seth K. Asante http://arxiv.org/abs/2610.38983v2 2026-11-09T09:22:26Z

Low-precision GPU kernels are commonly tested against a reference using hand-picked absolute and relative tolerances. Legitimate numerical error varies with the input, format and precision plan, so a fixed tolerance trades false alarms for missed faults. We present numoracle, which derives per-element acceptance regions from precision-annotated reference programs. Its calculus tracks shared rounding errors, provenance-ordered products, block-scaled quantizers and calibrated math-library errors. A rounding-interval check detects faulty stores that symmetric bounds can admit. Soundness is conditional on explicit rounding, hardware, domain and implementation assumptions; a fixed-input mode removes the zero-mean rounding assumption. We evaluate 13 Triton operators from Liger-Kernel, Unsloth and vLLM on two GPU generations. The main campaigns produce no false alarm on 33,960 correct-kernel and behavior-preserving-variant runs per GPU. Under input shift, the GPU campaigns record no false alarm while tuned tolerances flag up to 30% of correct runs; final-version reference checks identify inputs without complete finite-range certificates. On the RTX PRO 6000, numoracle kills 116 of 124 effective mutants, versus 59-83 for four fixed baselines. Among 400 tuned tolerance configurations, requiring at most 1% false alarms in every evaluated Liger input group permits at most 31 of 53 main-mutant kills, versus numoracle's 50. Historical replays expose accuracy faults in eight change pairs without rejecting their matched corrected controls; a ninth change concerns rounding parity.

2026-10-28T22:08:49Z 37 pages, 5 figures Xinyi Shi http://arxiv.org/abs/2611.12027v1 2026-11-09T08:04:08Z

Three-dimensional topology optimization solves linear systems with millions of unknowns at every design iteration. Consumer graphics cards run far faster in single than in double precision, and open codes for them mostly address minimum compliance. This educational article presents TOF3D, an open source Python code for density-based topology optimization in single precision on one consumer GPU. It solves four classes of problems: stress constrained minimum compliance, robust minimum compliance against manufacturing errors, compliant mechanisms and bone-like porous infill. The stiffness operator is applied without assembly, and the conjugate gradient method is preconditioned by geometric multigrid. Rounding the element matrix to single precision gave rigid body motions a spurious energy and changed compliances by up to 5.4%. Forming the element products from differences of nodal displacements reduced this error to $1.3\times 10^{-5}$ at no cost in time. Adjoint sensitivities agreed with finite differences to within $1.7\times 10^{-6}$, and the solutions agreed with Ansys Mechanical APDL to within $4.1\times 10^{-10}$ with the same element. On an RTX 3060 with 12 GB, one design iteration of a cantilever took 0.46 s with 0.5 million elements and 15 s with 33.5 million. On the same card and problem, an open GPU code that applies its operator in double precision took 3.7 and 21 s with 0.5 and 4.2 million elements and ran out of memory on larger meshes. Eight cases fixed in a protocol before the runs took 9 to 35 minutes each with 4.2 to 5.2 million elements.

2026-11-09T08:04:08Z 18 pages, 8 figures. Online Resource 1 is available as an ancillary file. Code and data: https://doi.org/10.5281/zenodo.24050327. Submitted to Structural and Multidisciplinary Optimization Leonilson Toffoli Pedrini Rodolfo G. M. Andrade Rafael M. Ferro http://arxiv.org/abs/2606.29129v2 2026-11-09T05:12:41Z

Ozaki scheme II emulates high-precision matrix multiplication using low-precision integer matrix operations based on the Chinese remainder theorem (CRT). It first scales the high-precision matrices to convert them into integer matrices. For this scaling step, Ozaki scheme II provides two modes: accurate mode, which uses INT8 matrix multiplication to estimate scaling factors, and fast mode, which applies the Cauchy--Schwarz inequality at lower computational cost. We show that the existing formula lacks scale invariance; multiplying the input matrices by a constant changes the effective bit width of the integer matrices in the scaling step, causing accuracy degradation or CRT recovery failure. To address this, we propose a revised scaling formula derived from the CRT uniqueness condition via the Cauchy--Schwarz inequality. The proposed formula is scale-invariant by construction, guarantees that the CRT uniqueness condition is always satisfied, and introduces no additional overhead over the original fast mode. Experiments on an NVIDIA GH200 GPU show that the proposed method achieves accuracy comparable to that of accurate mode while maintaining throughput comparable to that of fast mode. In the accuracy--throughput trade-off, the proposed method overcomes the accuracy limitation of fast mode and the throughput constraint of accurate mode, offering a superior accuracy and performance.

2026-06-28T00:37:25Z Corrected the equations on p. 7 and fixed minor typos. The proposed method and the experimental results are unchanged Shota Kawakami Daisuke Takahashi http://arxiv.org/abs/2610.35948v2 2026-11-08T13:57:45Z

Rectangular assignment with a uniform unmatched penalty is often implemented by expanding the cost matrix. We integrate established rectangular Jonker-Volgenant principles and implicit rejection with float64 working arithmetic and reusable workspaces in Rust (JVRUST), hybrid Rust/C++ (JVHYBRID) and C++ (JVCPP). A campaign evaluates laprust/lapcpp 1.2.0 and LAPX 0.13.0 in four Windows/Linux and Python environments. Each environment contains 302 cases: 3,286 complete-call timing conditions with five inputs and three rounds, plus 91 memory conditions with five fresh-process trials. Against the matching compact SciPy reference, all three native implementations exceed a 3% advantage on all 102 required cases; JVRUST and JVCPP do so on 163-165 and 163-166 of 166 optional cases, respectively. Controlled comparisons qualify these gains by representation, initialization, sparsity, batching and orientation. Assignment-replay gains attenuate in tracker updates, and equal replay objectives do not imply identical tracking outputs. A separate 67-case Windows LSAPE study examines related transformations within Hungarian code. The contribution is software integration and reproducible performance characterization. Adverse results are retained; no algorithmic priority, statistical significance or universal solver ranking is claimed. The initialization policy was developed using related workload families, so the campaign is not a held-out test of heuristic optimality.

2026-10-24T09:42:04Z 32 pages, 14 figures. v2: updated benchmark methodology and results, redesigned figures, and updated author list Ratha Siv Raphael Reme Matei Mancas Bernard Gosselin http://arxiv.org/abs/2611.11248v1 2026-11-08T09:04:23Z

Halving the floating-point precision of an ocean model can make it substantially faster, but whether the rounding errors stay negligible over climate time scales has rarely been tested. We implement single precision in the global ocean-sea-ice model FESOM2 and compare 60-year hindcasts with double-precision controls. A first implementation lost salt steadily, accumulated a heat-content deficit of 129 ZJ, and shifted major ocean fronts. Controlled experiments trace the drift to the tracer update: small increments added to a large salinity value are rounded away, and the increments themselves are biased. Accumulating tracer tendencies and layer thicknesses in 64 bits, and carrying the rounding error of each tracer update into the next, removes the drift for at most 8% additional runtime. Single- and double-precision simulations then differ in heat content and mean state about as much as two double-precision runs with slightly perturbed initial conditions, sea ice differs by at most 0.2%, and he salinity difference is four orders of magnitude smaller than before. In the coupled model AWI-ESM3, single precision adds no detectable heat source or sink over 60 years. Single precision speeds up the 60-year hindcasts by a factor of 1.35, and the model by up to 1.66 on meshes of up to 5 km resolution; on GPUs, a C++/Kokkos version runs up to 1.27 times faster and needs 43% less memory. Whether reduced precision is safe for climate integrations is decided by long-term budgets, not by short tests.

2026-11-08T09:04:23Z Suvarchal Kumar Cheedela Nikolay V. Koldunov Sergey Danilov Jan Streffing Dmitry Sidorenko Thomas Jung http://arxiv.org/abs/2611.11100v1 2026-11-08T03:30:30Z

In High-performance Computing the holy grail has been to translate numerical algorithms expressed in mathematical notation into code that performs well across a range of platforms. A variety of portability frameworks have abstracted data layout, parallelization and orchestration from application's view, however, the resulting format is still code that conforms to the idioms of the framework. Additionally, substantial refactoring is typically needed to achieve performance goals. Increasing sophistication of Large Language Models (LLMs) naturally raises the question whether they can be used to make application codes and/or solvers both portable and performant. The obstacle is the lack of correctness guarantee in LLM produced code. In this paper we pose the question differently -- can we start with expressing numerical intent of the code into an immutable core arithmetic component isolated from the surrounding code such that the surrounding code can be safely generated by the LLMs? We answer this question by using Spark, a mature compressible hydrodynamics solver in Flash-X, a multiphysics software system. We use LLM assistance in devising the abstractions and embedding them in the solver. Once properly verified, the resulting solver is lowered to concrete realization deterministically with tools also built with LLM assistance. The driver of the numerics and the code for interfacing with the application are generated by the LLMs. The resulting solver is exercised within Flash-X and with a separate environment we built for this experiment. Our results show that this is a viable approach for producing correct and performance portable code.

2026-11-08T03:30:30Z Elena Aschmann Anshu Dubey Mohamed Wahib Jens Domke http://arxiv.org/abs/2611.12972v1 2026-11-07T18:55:15Z

We present a software implementation (as a Python package) of approximation schemes for verifying frequency inequalities arising from the quadratic Bendixson criterion for systems of delay equations (with a single delay in the linear part). In applications to dissipative nonlinear autonomous systems, such inequalities ensure global stability. The theoretical foundations of this approach were developed in our related work. In particular, it was shown that the transfer operators involved into the frequency inequalities can be represented as integral operators with bounded kernels of the Hardy--Volterra type. Here, we use this representation to construct approximation schemes based on finite-dimensional truncations of the transfer operators. Examples of using the package are provided.

2026-11-07T18:55:15Z Mikhail Anikushin Andrey Romanov http://arxiv.org/abs/2611.10855v1 2026-11-07T17:09:25Z

We propose a new mathematical formalism in which all kinds of tensors and of operations on those tensors are represented as simple function compositions. Tensors are functions from indices to values. An element-wise operation (e.g. an activation function) is a composition with a function applied to values; re-indexing (e.g. reshape, transposition) is a composition with a function applied to indices; and a reduction is a composition with both an index function, mapping each index to several indices, and a reduction operation (e.g. product, sum, max). Combined with tensor stacking, these compositions can represent most interesting kinds of tensors (e.g. dense, jagged, all kinds of sparsity patterns, matrix factorizations), as well as lazy versions of almost every operation available in modern tensor computation frameworks, including einsums between tensors of any kind. Identities between compositions can then be used to rewrite such expressions into equivalent ones, without evaluating them. For instance, the product of two sparse matrices can be rewritten so that only products between non-zero values are computed, turning a sparse problem into a dense one. Using this algebra as the intermediate representation of a tensor computation engine would shift the focus from defining operations on tensors (e.g. einsum between sparse tensors) to defining an algebra on index functions. This should reduce the codebase complexity from $\mathcal{O}(mn^2)$, with $m$ the number of binary operations between tensors and $n$ the number of tensor types, to $\mathcal{O}(m + k^2)$, with $k$ the number of types of index functions. A draft implementation of a tensor computation framework based on this algebra is publicly available.

2026-11-07T17:09:25Z 15 pages, 6 figures Valérian Rey Praneth Paruchuri http://arxiv.org/abs/2610.08299v3 2026-11-07T12:54:04Z

Pauli decomposition of an $n$-qubit operator is expensive if each of the $4^n$ Pauli strings is treated as an unstructured object. Labeling those strings by the configuration space $(\mathbb{Z}_2)^n$ makes them displacement operators $X^{\otimes x}Z^{\otimes z}$ on the computational space: shifts by~$x$ and multiplications by characters $Z^{\otimes z}$ in the computational basis. That labeling organizes the operator into $2^n$ fibers of length $2^n$, one per shift~$x$, each collecting the computational-basis amplitudes that hop by~$x$. Expanding each fiber function in the character basis of $(\mathbb{Z}_2)^n$ --- a length-$2^n$ Walsh--Hadamard transform --- recovers the Pauli coefficients at total cost $O(n\cdot4^n)$. The same fiber / shift--character picture extends to $(\mathbb{Z}_p)^n$ (Heisenberg--Weyl qudits). \texttt{paulikit} is a memory-bounded realization for qubits: for matrix-sparse inputs, peak memory need not densify to a full $2^n\times2^n$ operator. Performance is checked on a stated low-end GNU/Linux platform for a coupled-oscillator sweep beyond a billion Pauli terms.

2026-10-06T13:03:45Z 23 pages, 4 figures. Corrected oscillator Hamiltonian (paulikit 0.1.4); remesured tables. Zenodo 10.5281/zenodo.23972284 (software), 10.5281/zenodo.24030142 (measurements) Mohammadreza Khellat Mohammad Masoumi Saman Nasoori Soroush Nasoori http://arxiv.org/abs/2611.10487v1 2026-11-07T05:29:20Z

We present a novel GPU-accelerated global solver for constrained nonlinear programming (NLP) inspired by the efficient linear bound propagation framework in neural network (NN) verification. Existing work on NN verification, a problem that can be cast as feasibility detection for a nonlinear objective, demonstrated orders-of-magnitude speedups with the linear bound propagation-based framework when compared to traditional solvers. Extending this paradigm to global nonlinear programming requires addressing many challenges: a global constrained NLP solver must effectively exploit all, possibly unstructured, nonlinear constraints to improve the objective, whereas NN verification typically handles simple input box constraints over a well-structured NN and does not directly optimize an objective function. We address these challenges by fully exploiting the linear bounds produced by bound propagation: these linear bounds can be viewed as relaxations of nonlinear constraints and used to provide lower bounds, check infeasibility, and guide the search for a primal solution. In addition, we propose tighter linear relaxations for bilinear, quotient, and other commonly used nonlinear functions in NLP problems, and a projected augmented Lagrangian method tightly coupled to our dual solving procedure to produce primal solutions. Our method solves 421 of 505 bounded and continuous NLP problems in the GAMS and MINLPLib benchmarks under a 180-second limit, more than each of the six global solvers we compare with, including Gurobi and BARON (commercial), and SCIP (open-source). Sixteen of these problems are certified by none of the six global solvers; all of them contain exponential, logarithmic, or trigonometric terms, which are often challenging for traditional approaches. Project webpage is at https://verified-intelligence.github.io/GEAR.

2026-11-07T05:29:20Z Accepted by NeurIPS 2026 Duo Zhou Hesun Chen Xiangru Zhong Grani A. Hanasusanto Huan Zhang http://arxiv.org/abs/2603.01915v3 2026-11-06T11:22:55Z

We present a novel, practical approach to speed up sparse matrix-vector multiplication (SpMVM) on GPUs. The novel key idea is to apply lossless entropy coding to further compress the sparse matrix when stored in one of the commonly supported formats. Our method is based on dtANS, our new lossless compression method that improves the entropy coding technique of asymmetric numeral systems (ANS) specifically for fast parallel GPU decoding when used in tandem with SpMVM. We apply dtANS on the widely used CSR format and present extensive benchmarks on the SuiteSparse collection of matrices against the state-of-the-art cuSPARSE library. On matrices with at least 2^(15) entries and at least 10 entries per row on average, our compression reduces the matrix size over the smallest cuSPARSE format (CSR, COO and SELL) in almost all cases and up to 11.77 times. Further, we achieve an SpMVM speedup for the majority of matrices with at least 2^(25) nonzero entries. The best speedup is 3.48x. We also show that we can improve over the AI-based multi-format AlphaSparse in an experiment that is limited due to its extreme computation overhead. We provide our code as an open source C++/CUDA header library, which includes both compression and multiplication kernels.

2026-03-02T14:28:48Z Published in the Proceedings of the 40th IEEE International Parallel & Distributed Processing Symposium (IPDPS), 2026. Artifact: https://doi.org/10.5281/zenodo.18694064 2026 IEEE International Parallel and Distributed Processing Symposium (IPDPS), pp. 416-432 Emil Schätzle Tommaso Pegolotti Markus Püschel 10.1109/IPDPS65963.2026.00044 http://arxiv.org/abs/2611.08637v1 2026-11-05T19:35:01Z

We present branch-extended automatic differentiation (BEAD), a software package that provides information about inactive branches of piecewise-differentiable functions, intended for use in nonsmooth optimization algorithms. BEAD is implemented in Python, building on the popular JAX framework. For supported nonsmooth operations such as abs and max, BEAD identifies alternative branches within tolerance from the given input, and computes gradients of the resulting branches extended to the given input. Such information has been previously used for robust optimization methods, but has required error-prone and tedious manual differentiation, unlike BEAD, which is fully automated for functions written in NumPy-like JAX code. Our experiments show that our approach is as accurate as manually written derivative code, and offers more flexibility in modifying the function and choosing between branch-discovery rules. While we were originally motivated by the manifold sampling algorithm for nonsmooth composite objectives, we also integrate BEAD branch-extension gradients into a nonsmooth BFGS-SQP solver. On two small nonsmooth test problems, the resulting path-aware search resolves failures or substantially reduces the number of evaluations required by the BFGS-SQP solver.

2026-11-05T19:35:01Z Jan Hückelheim Jeffrey Larson Matt Menickelly http://arxiv.org/abs/2611.08371v1 2026-11-05T17:50:34Z

Datacenter GPUs increasingly favour low-precision floating-point formats, offering higher throughput than FP64. Double-word (DW) arithmetic represents numbers as unevaluated sums of two floating-point values, providing higher accuracy. We present the first error analysis of complex DW multiplication and division. Both operations reduce to sums of two DW products, combined using one of three DW addition algorithms: accurate, sloppy, or Madd. Normalising DW products ensures nonoverlapping components, but omitting this step reduces the number of arithmetic operations. We bound the resulting overlap and prove that Madd retains a relative error bound of $3u^2+O(u^3)$ for overlapping inputs, where $u$ is the unit roundoff. For all variants, we establish relative error bounds of the form $(ακ+β)u^2+O(u^3)$, where $κ$ is the condition number of the sum of products. Our analysis shows that the less expensive sloppy addition suffices. We implement these variants in CompleXDW, an open-source C++ library for CPUs and GPUs. In the high-energy physics code MadGraph7, FP32-based DW arithmetic achieves up to 2.9x and 2.1x higher throughput than FP64 on NVIDIA L40S and AMD Radeon Pro W7900 GPUs, respectively, while losing approximately 1.4 correct significant digits.

2026-11-05T17:50:34Z 46 pages, 8 figures, 3 tables František Stloukal Fabienne Jézéquel