Systematic Bernoulli Generator Matrix Codes

2025-04-18T14:58:50Z

This paper is concerned with the systematic Bernoulli generator matrix~(BGM) codes, which have been proved to be capacity-achieving over binary-input output-symmetric~(BIOS) channels in terms of bit-error rate~(BER). We prove that the systematic BGM codes are also capacity-achieving over BIOS channels in terms of frame-error rate (FER). To this end, we present a new framework to prove the coding theorems for binary linear codes. Different from the widely-accepted approach via ensemble enlargement, the proof directly applies to the systematic binary linear codes. The new proof indicates that the pair-wise independence condition is not necessary for proving the binary linear code ensemble to achieve the capacity of the BIOS channel. The Bernoulli parity-check~(BPC) codes, which fall within the framework of the systematic BGM codes with parity-check bits known at the decoder can also be proved to achieve the capacity. The presented framework also reveals a new mechanism pertained to the systematic linear codes that the systematic bits and the corresponding parity-check bits play different roles. Precisely, the noisy systematic bits are used to limit the list size of candidate codewords, while the noisy parity-check bits are used to select from the list the maximum likelihood codeword. For systematic BGM codes with finite length, we derive the lower bounds on the BER and FER, which can be used to predict the error floors. Numerical results show that the systematic BGM codes match well with the derived error floors. The performance in water-fall region can be improved with approaches in statistical physics and the error floors can be significantly improved by implementing the concatenated codes with the systematic BGM codes as the inner codes.

Capacity-achieving sparse superposition codes with spatially coupled VAMP decoder

2025-04-18T10:07:56Z

Sparse superposition (SS) codes provide an efficient communication scheme over the Gaussian channel, utilizing the vector approximate message passing (VAMP) decoder for rotational invariant design matrices. Previous work has established that the VAMP decoder for SS achieves Shannon capacity when the design matrix satisfies a specific spectral criterion and exponential decay power allocation is used. In this work, we propose a spatially coupled VAMP (SC-VAMP) decoder for SS with spatially coupled design matrices. Based on state evolution (SE) analysis, we demonstrate that the SC-VAMP decoder is capacity-achieving when the design matrices satisfy the spectra criterion. Empirically, we show that the SC-VAMP decoder outperforms the VAMP decoder with exponential decay power allocation, achieving a lower section error rate. All codes are available on https://github.com/yztfu/SC-VAMP-for-Superposition-Code.git.

Irregular Sampling of High-Dimensional Functions in Reproducing Kernel Hilbert Spaces

2025-04-18T08:18:15Z

We develop sampling formulas for high-dimensional functions in reproducing kernel Hilbert spaces, where we rely on irregular samples that are taken at determining sequences of data points. We place particular emphasis on sampling formulas for tensor product kernels, where we show that determining irregular samples in lower dimensions can be composed to obtain a tensor of determining irregular samples in higher dimensions. This in turn reduces the computational complexity of sampling formulas for high-dimensional functions quite significantly.

Joint Power and Bit Allocation for Precoded Massive MIMO Channels

2025-04-18T07:22:09Z

This work addresses the joint optimization of power and bit allocation in precoded large-scale n x n MIMO systems with discrete input alphabets, specifically QAM constellations. We propose an adaptive QAM scheme that maintains a fixed gap to the Gaussian-input capacity for a given n. A key finding is that, under the proposed scheme, the mercury/waterfilling (MWF) solution reduces analytically to the classical water-filling (WF) policy. Furthermore, the adaptive QAM configuration can be precomputed under the large-system assumption, enabling the replacement of full SVD with truncated SVD and yielding substantial computational savings. To support practical deployment, we develop a bit-allocation algorithm that meets a target transmission data rate while minimizing the overall decoding error rate and preserving computational complexity at O(n log n). Simulation results confirm that the proposed truncated SVD precoding, paired with the joint power and bit allocation, achieves superior decoding performance relative to conventional approaches, while operating at significantly lower complexity.

Algebras of Information. An Axiomatic Foundation

2025-04-18T07:18:15Z

The basic idea behind information algebras is that information comes in pieces, each referring to a certain question, that these pieces can be combined or aggregated and that the part relating to a given question can be extracted. This algebraic structure can be given different forms. Questions were originally represented by subsets of variables. Pieces of information were then represented by valuations associated with the domains of variables. This leads to an algebraic structure called valuation algebras. The basic axiomatics of this algebraic structure was in essence proposed by Shenoy and Shafer. Here a much more general view of systems of questions is proposed and pieces of information are related to the elements of this system of questions. This leads to a new and extended system of axioms for information algebras. Classical valuation algebras are essentially a special case of this new system. A full discussion of the algebraic theory of this new information algebras is given, including local computation, duality between labeled and domain-free versions of the algebras, order of information, finiteness of information and approximation, compact and continuous information algebras. Finally a rather complete discussion of uncertain information, based on random maps into information algebras is presented. This is shown to represent a generalisation of classical Dempster-Shafer theory.

Mixed Fractional Information: Consistency of Dissipation Measures for Stable Laws

2025-04-18T02:49:24Z

Symmetric alpha-stable (S alpha S) distributions with alpha<2 lack finite classical Fisher information. Building on Johnson's framework, we define Mixed Fractional Information (MFI) via the initial rate of relative entropy dissipation during interpolation between S alpha S laws with differing scales, v and s. We demonstrate two equivalent formulations for MFI in this specific S alpha S-to-S alpha S setting. The first involves the derivative D'(v) of the relative entropy between the two S alpha S densities. The second uses an integral expectation E_gv[u(x,0) (pF_v(x) - pF_s(x))] involving the difference between Fisher scores (pF_v, pF_s) and a specific MMSE-related score function u(x,0) derived from the interpolation dynamics. Our central contribution is a rigorous proof of the consistency identity: D'(v) = (1/(alpha v)) E_gv[X (pF_v(X) - pF_s(X))]. This identity mathematically validates the equivalence of the two MFI formulations for S alpha S inputs, establishing MFI's internal coherence and directly linking entropy dissipation rates to score function differences. We further establish MFI's non-negativity (zero if and only if v=s), derive its closed-form expression for the Cauchy case (alpha=1), and numerically validate the consistency identity. MFI provides a finite, coherent, and computable information-theoretic measure for comparing S alpha S distributions where classical Fisher information fails, connecting entropy dynamics to score functions and estimation concepts. This work lays a foundation for exploring potential fractional I-MMSE relations and new functional inequalities tailored to heavy-tailed systems.

How Breakable Is Privacy: Probing and Resisting Model Inversion Attacks in Collaborative Inference

2025-04-18T02:06:06Z

Collaborative inference (CI) improves computational efficiency for edge devices by transmitting intermediate features to cloud models. However, this process inevitably exposes feature representations to model inversion attacks (MIAs), enabling unauthorized data reconstruction. Despite extensive research, there is no established criterion for assessing the difficulty of MIA implementation, leaving a fundamental question unanswered: \textit{What factors truly and verifiably determine the attack's success in CI?} Moreover, existing defenses lack the theoretical foundation described above, making it challenging to regulate feature information effectively while ensuring privacy and minimizing computational overhead. These shortcomings introduce three key challenges: theoretical gap, methodological limitation, and practical constraint. To overcome these challenges, we propose the first theoretical criterion to assess MIA difficulty in CI, identifying mutual information, entropy, and effective information volume as key influencing factors. The validity of this criterion is demonstrated by using the mutual information neural estimator. Building on this insight, we propose SiftFunnel, a privacy-preserving framework to resist MIA while maintaining usability. Specifically, we incorporate linear and non-linear correlation constraints alongside label smoothing to suppress redundant information transmission, effectively balancing privacy and usability. To enhance deployability, the edge model adopts a funnel-shaped structure with attention mechanisms, strengthening privacy while reducing computational and storage burdens. Experiments show that, compared to state-of-the-art defense, SiftFunnel increases reconstruction error by $\sim$30\%, lowers mutual and effective information metrics by $\geq$50\%, and reduces edge burdens by almost $20\times$, while maintaining comparable usability.

Quantum error correction for long chains of trapped ions

2025-04-18T01:54:05Z

We propose a model for quantum computing with long chains of trapped ions and we design quantum error correction schemes for this model. The main components of a quantum error correction scheme are the quantum code and a quantum circuit called the syndrome extraction circuit, which is executed to perform error correction with this code. In this work, we design syndrome extraction circuits tailored to our ion chain model, a syndrome extraction tuning protocol to optimize these circuits, and we construct new quantum codes that outperform the state-of-the-art for chains of about $50$ qubits. To establish a baseline under the ion chain model, we simulate the performance of surface codes and bivariate bicycle (BB) codes equipped with our optimized syndrome extraction circuits. Then, we propose a new variant of BB codes defined by weight-five measurements, that we refer to as BB5 codes and we identify BB5 codes that achieve a better minimum distance than any BB codes with the same number of logical qubits and data qubits, such as a $[[48, 4, 7]]$ BB5 code. For a physical error rate of $10^{-3}$, the $[[48, 4, 7]]$ BB5 code achieves a logical error rate per logical qubit of $5 \cdot 10^{-5}$, which is four times smaller than the best BB code in our baseline family. It also achieves the same logical error rate per logical qubit as the distance-7 surface code but using four times fewer physical qubits per logical qubit.

Foundational theories of hesitant fuzzy sets and families of hesitant fuzzy sets

2025-04-18T01:10:32Z

Hesitant fuzzy sets find extensive application in specific scenarios involving uncertainty and hesitation. In the context of set theory, the concept of inclusion relationship holds significant importance as a fundamental definition. Consequently, as a type of sets, hesitant fuzzy sets necessitate a clear and explicit definition of the inclusion relationship. Based on the discrete form of hesitant fuzzy membership degrees, this study proposes multiple types of inclusion relationships for hesitant fuzzy sets. Subsequently, this paper introduces foundational propositions related to hesitant fuzzy sets, as well as propositions concerning families of hesitant fuzzy sets.

Improved Decoding Algorithm of BD-LRPC Codes

2025-04-18T00:00:55Z

A Bounded-Degree Low-Rank Parity-Check (BD-LRPC) code is a rank-metric code that admits a parity-check matrix whose support is generated by a set of powers of an element. This specific structure of the parity-check matrix was employed to enhance the first phase of the decoding algorithm through the expansion of the syndrome support. However, this expansion decreases the probability of recovering the error support in the second phase of the decoding algorithm. This paper introduces a novel method based on successive intersections to recover the error support. This method offers two key advantages: it increases the probability of successful decoding and enables the decoding of a greater number of errors.

AI-Empowered Integrated Sensing and Communications

2025-04-17T22:03:43Z

Integrating sensing and communication (ISAC) can help overcome the challenges of limited spectrum and expensive hardware, leading to improved energy and cost efficiency. While full cooperation between sensing and communication can result in significant performance gains, achieving optimal performance requires efficient designs of unified waveforms and beamformers for joint sensing and communication. Sophisticated statistical signal processing and multi-objective optimization techniques are necessary to balance the competing design requirements of joint sensing and communication tasks. Since model-based analytical approaches may be suboptimal or overly complex, deep learning emerges as a powerful tool for developing data-driven signal processing algorithms, particularly when optimal algorithms are unknown or when known algorithms are too complex for real-time implementation. Unified waveform and beamformer design problems for ISAC fall into this category, where fundamental design trade-offs exist between sensing and communication performance metrics, and the underlying models may be inadequate or incomplete. This article explores the application of artificial intelligence (AI) in ISAC designs to enhance efficiency and reduce complexity. We emphasize the integration benefits through AI-driven ISAC designs, prioritizing the development of unified waveforms, constellations, and beamforming strategies for both sensing and communication. To illustrate the practical potential of AI-driven ISAC, we present two case studies on waveform and beamforming design, demonstrating how unsupervised learning and neural network-based optimization can effectively balance performance, complexity, and implementation constraints.

Levenshtein's Sequence Reconstruction Problem and Results for Larger Alphabet Sizes

2025-04-17T21:23:42Z

The problem of storing large amounts of information safely for a long period of time has become essential. One of the most promising new data storage mediums are the polymer-based data storage systems, like the DNA-storage system. These storage systems are highly durable and they consume very little energy to store the data. When information is retrieved from a storage, however, several different types of errors may occur in the process. It is known that the Levenshtein's sequence reconstruction framework is well-suited to overcome such errors and to retrieve the original information. Many of the previous results regarding Levenshtein's sequence reconstruction method are so far given only for the binary alphabet. However, larger alphabets are natural for the polymer-based data storage. For example, the quaternary alphabet is suitable for DNA-storage due to the four amino-acids in DNA. The results for larger alphabets often require, as we will see in this work, different and more complicated techniques compared to the binary case. Moreover, we show that an increase in the alphabet size makes some error types behave rather surprisingly.

From Bayesian Asymptotics to General Large-Scale MIMO Capacity

2025-04-17T20:24:46Z

We present a unifying framework that bridges Bayesian asymptotics and information theory to analyze the asymptotic Shannon capacity of general large-scale MIMO channels including ones with non-linearities or imperfect hardware. We derive both an analytic capacity formula and an asymptotically optimal input distribution in the large-antenna regime, each of which depends solely on the single-output channel's Fisher information through a term we call the (tilted) Jeffreys' factor. We demonstrate how our method applies broadly to scenarios with clipping, coarse quantization (including 1-bit ADCs), phase noise, fading with imperfect CSI, and even optical Poisson channels. Our asymptotic analysis motivates a practical approach to constellation design via a compander-like transformation. Furthermore, we introduce a low-complexity receiver structure that approximates the log-likelihood by quantizing the channel outputs into finitely many bins, enabling near-capacity performance with computational complexity independent of the output dimension. Numerical results confirm that the proposed method unifies and simplifies many previously intractable MIMO capacity problems and reveals how the Fisher information alone governs the channel's asymptotic behavior.

Space-bounded online Kolmogorov complexity is additive

2025-04-17T20:11:03Z

The even online Kolmogorov complexity of a string $x = x_1 x_2 \cdots x_{n}$ is the minimal length of a program that for all $i\le n/2$, on input $x_1x_3 \cdots x_{2i-1}$ outputs $x_{2i}$. The odd complexity is defined similarly. The sum of the odd and even complexities is called the dialogue complexity. In [Bauwens, 2014] it is proven that for all $n$, there exist $n$-bit $x$ for which the dialogue complexity exceeds the Kolmogorov complexity by $n\log \frac 4 3 + O(\log n)$. Let $\mathrm C^s(x)$ denote the Kolmogorov complexity with space bound~$s$. Here, we prove that the space-bounded dialogue complexity with bound $s + 6n + O(1)$ is at most $\mathrm C^{s}(x) + O(\log (sn))$, where $n=|x|$.

Kähler information manifolds of signal processing filters in weighted Hardy spaces

2025-04-17T17:00:03Z

We extend the framework of K\"ahler information manifolds for complex-valued signal processing filters by introducing weighted Hardy spaces and smooth transformations of transfer functions. We demonstrate that the Riemannian geometry induced from weighted Hardy norms for the smooth transformations of its transfer function is a K\"ahler manifold. In this setting, the K\"ahler potential of the linear system geometry corresponds to the squared weighted Hardy norm of the composite transfer function. With the inherent structure of K\"ahler manifolds, geometric quantities on the manifold of linear systems in weighted Hardy spaces can be computed more efficiently and elegantly. Moreover, this generalized framework unifies a variety of well-known information manifolds within the structure of K\"ahler information manifolds for signal filters. Several illustrative examples from time series models are provided, wherein the metric tensor, Levi-Civita connection, and K\"ahler potentials are explicitly expressed in terms of polylogarithmic functions of the poles and zeros of transfer functions parameterized by weight vectors.