https://arxiv.org/api/zp1ThNKP/Wk79IxpBOGf0bVW9oU 2026-09-11T20:43:28Z 9019 45 15 http://arxiv.org/abs/2609.07175v1 2026-09-07T08:07:14Z

Generative recommendation (GR) systems increasingly leverage user-level KV cache reuse to avoid recomputing long user histories. However, the growing KV cache capacity and bandwidth requirements introduce new challenges for memory system. High-Bandwidth Flash (HBF) provides a promising solution by offering substantially higher capacity than HBM while approaching HBM-class read bandwidth, enabling larger scale KV cache retention and improved serving throughput. Yet conventional Least-Recently-Used (LRU) KV cache management tightly couples KV cache writes with cache misses, generating excessive write traffic that rapidly exhausts flash endurance. In this work, we evaluate a write-aware KV cache policy based on admission-controlled LRU-K for HBF-based GR serving. By filtering low-reuse users before cache admission, LRU-K decouples KV cache writes from misses and significantly reduces unnecessary writes. We develop an analytical model to characterize GR serving performance, KV cache write traffic, and HBF lifetime, and evaluate performance across diverse memory systems and GR workloads. Our results show that HBF-based systems achieve 3.8 to 4.7 times higher throughput than HBM-only systems. Moreover, LRU-K extends HBF lifetime from about one year under conventional LRU to over six years with a moderate K=10, while maintaining comparable or even slightly improved throughput. These results highlight the importance of write aware KV cache policy for sustainable HBF-based GR serving.

2026-09-07T08:07:14Z 5 pages, 7 figures Danni Peng Kai Wu Tianyu Zuo Pengfei Xia Hui Zang http://arxiv.org/abs/2609.07035v1 2026-09-07T04:44:19Z

A quantum error correction decoder is a library other people's results depend on, judged in one dominant way. Sample errors, decode, and count wrong logical observables. We ask what else can be checked there. Our conformance contract needs no oracle. One check asks that a returned correction explain the syndrome in the caller's index space. The other hands a decoder one instance under two presentations differing only in bookkeeping, where two feasible corrections of different weight prove the heavier is not minimum-weight. Verdicts are gated on what each library declares, so a firing contradicts a published guarantee. Nine configurations from five public libraries give three results. Documentation answers 4 of 54 capability questions. Bounded-distance correctness, the property callers most depend on, has a direct declaration yield of 0.0%, though its hypotheses hold in 62.1% of cases. Presentation sensitivity is real but shallow. One solver moved to a 26% heavier correction under a different numbering, which reaches the logical class at most once in twenty thousand shots. Established evaluation misses corruptions that preserve logical parity, while one summation over the caller's weights catches every one we injected. All 639 certificates ship as bundles a standalone verifier re-derives from first principles.

2026-09-07T04:44:19Z Jiachen Shen Hui Zhong http://arxiv.org/abs/2609.06796v1 2026-09-06T19:35:19Z

Multi-chiplet photonic neural network accelerators (MCPNAs) combine photonic computation, photonic communica-tion, and heterogeneous chiplet integration to enable scalable and energy-efficient AI acceleration. However, their distributed archi-tecture and reliance on third-party chiplets introduce significant hardware security risks. This paper examines Hardware Trojan (HT) threats to MCPNAs across three dimensions: confidentiality, integrity, and availability.

2026-09-06T19:35:19Z Sudeep Pasricha http://arxiv.org/abs/2609.06781v1 2026-09-06T18:56:16Z

In safety-critical deployments, AI hardware must remain reliable against a broad spectrum of threats such as aging, soft errors, hard faults, and adversarial attacks (e.g. progressive bit flip attack (PBFA)). All of these corrupt stored weights while the chip keeps producing confident but inaccurate predictions. Detecting and mitigating such weight perturbations is crucial for safety-critical platforms. To that end, we propose BLINK, an on-chip batch normalization (BN)-based on-the-fly detection and mitigation approach, which is based on continual sensing of the shift in the activation statistics, and targets a wide variety of weight corruptions (random and localized faults as well as adversarial bit flips). BLINK operates in two phases: (1) off-line pre-characterization of the relationship of the activation shifts with inference accuracy drop, and (2) on-chip runtime detection and mitigation of weight corruptions. Upon detection, the flagged layer is re-centered to bring it closer to its stored clean reference within the same forward pass. BLINK is fully autonomous, eliminating the need for host communication, operation halts, or access to fine-tuning data. If the residual shift after mitigation indicates that accuracy has fallen below a user-set floor, a held-out watcher aborts the inference. Evaluated on ResNet-20/50 and MobileNetV2 for CIFAR-10/100, BLINK detects harmful corruptions with >99% precision across all fault types. Further, it recovers accuracy from 10% to 85.88% under 0.5% random bit flips (Resnet-50/CIFAR-10), up to 84% for localized faults (MobileNetV2/CIFAR-10), and from random-guess accuracy to 80%-83% under PBFA (ResNet-20/CIFAR-10). Hardware overhead estimates indicate that BLINK incurs negligible costs, with less than a 2% increase in latency and only a 0.53% increase in computation overhead.

2026-09-06T18:56:16Z Marzia Khan Akul Malhotra Sumeet Kumar Gupta http://arxiv.org/abs/2609.06691v1 2026-09-06T16:04:42Z

Latency-critical cloud services place growing pressure on memory while requiring isolation, fairness, and predictable QoS. Near-data processing (NDP) reduces data movement by executing requests close to memory, and prior systems further improve locality through caching and replication. However, writes make replica maintenance expensive, while uneven compute and memory traffic can overload a few NDP units and increase tail latency. Existing throughput-oriented schedulers do not fully address these challenges for co-located cloud services. We present Gutenberg, a CPU+NDP for mutable, latency-critical cloud services. Gutenberg stages subpage updates in a CPU-resident delta buffer, allowing hot writable pages to remain replicated without eager full-page synchronization. It also adopts CPU helper cores to assist request execution when NDP execution or replica maintenance becomes costly. An online controller jointly decides page placement, replication, CPU/NDP execution, and routing using access patterns, queue pressure, and feedback from prior decisions. The system further enforces isolation and fair resource allocation across services. We also model-check CPU--NDP coordination protocol for correctness. We evaluate on TailBench using ZSim with Ramulator-calibrated memory timing. Across evaluated services, Gutenberg outperforms prior systems, reducing average and p99 latency by up to 80.4% and 85.8%. It also improves isolation and fairness while adapting to changing workload behaviors.

2026-09-06T16:04:42Z Qiushi Lin Phillip B. Gibbons Jovan Stojkovic Yiwei Zhao http://arxiv.org/abs/2609.06551v1 2026-09-06T11:52:49Z

Mobile vendors and application developers increasingly deploy LLMs on smartphones for diverse prefill-only services. Yet current systems rely mainly on dense models whose regular computation maps efficiently to mobile NPUs, leaving more capable MoEs underused. MoE prefill does not fit mobile NPUs: NPU graphs are fixed at compile time, yet MoE picks experts at runtime; and one request touches most experts, more than a phone can hold in memory. We present EStream, which resolves both by separating what the NPU must fix from what MoE decides at runtime. A single compiled expert graph serves every expert, with each expert's routed tokens and weight address bound at call time, so dynamic MoE execution runs entirely on the NPU without padding or CPU/GPU fallback. Expert virtualization keeps the expert pool in UFS flash storage and pages it through a fixed-size NPU-addressable arena, group by group, with loading hidden behind computation, so memory is bounded by the arena rather than by the model. It further introduces a hardware-aware configuration algorithm that automatically configures the UFS--NPU pipeline and maximizes loading--computation overlap. Across 18 comparative settings covering three 7B--16B MoEs and 256--4,096-token prompts, we evaluate EStream on a commercial Snapdragon smartphone. Compared to the fastest baseline at each setting, EStream achieves a 2.25--27.57X pure-prefill TTFT speedup and reduces peak physical memory by 1.19--12.29X. EStream further scales to MoE models with up to 46.7B parameters.

2026-09-06T11:52:49Z Junming Zhang Zhenzhe Zheng Fan Wu Xiaoyao Huang Jie Wu http://arxiv.org/abs/2609.06365v1 2026-09-06T03:48:09Z

Ising Machines offer vast potential to solve NP-hard optimization problems efficiently that are intractable to solve using conventional computing architecture. A lot of these optimization problems fall into statistical learnability and involve finding an optimal solution among many possible, near-analogous configurations, by searching in a non-convex energy landscape. In this context, the probabilistic Boltzmann machine architecture especially PASS (Parallel Asynchronous Stochastic Sampler), explores and models the complex probability landscape pertaining to all possible configurations and excels in finding the ground-state energy solution of these intractable problems. Additionally, the noise-based neuron architecture addresses the limitation of conventional annealing methods, which may get stuck around local minima. Here, we demonstrate a stochastic sampling approach based on Block Gibbs Sampling to integrate multiple asynchronous PASS chips (four in this work), enabling improved scalability. Further, we demonstrate the scaling by mapping 784 nodes Max-Cut problem integrating 256 nodes PASS accelerator manufactured in 14 nm CMOS FinFET technology. PASS-enabled system with Block Gibbs Sampling protocol shows approximately 1000 times speedup for Max-Cut optimization compared to state-of-the-art methods implemented on CPUs and GPUs. The general applicability of this approach is further illustrated by solving a quantum spin chain Transverse Ising system and accurately representing complex probability landscapes. Moreover, our results demonstrate the change in the scaling law to constant in the scaled-PASS accelerator as compared to exponential on GPUs enabling at least 4 orders of magnitude improvement in time-to-solution. Hence, the presented methodology enables the pathway for scaling of asynchronous brain-like dynamics systems that do not follow any clock for its operation.

2026-09-06T03:48:09Z 19 pages, 6 figures Chirag Garg Pratik Brahma Saavan Patel Sayeef Salahuddin http://arxiv.org/abs/2609.06270v1 2026-09-05T21:43:50Z

Approximate Nearest-Neighbor Search (ANNS) in high dimensional vector datasets is an application of significant prevalence across different AI applications. However, such an operation is significantly bandwidth limited at large workingset sizes owing to the curse of dimensionality. Traditional indices used to accelerate ANNS rely on search-space pruning as a preprocessing step to alleviate such bandwidth requirement, but such optimization occurs either at the cost of increased bandwidth-inefficiency and/or degradation of search quality. This paper proposes a data-parallel hardware/software mechanism for performing large-scale similarity search in-memory. We propose a novel algorithm to simplify the computation requirement for similarity search across various distance metrics through lightweight primitives to perform a fast and approximate data-parallel brute-force search on the entire vector space. We further build a memory system capable of executing the required operations to generate a distance metric per datapoints, which is then used to enable pruning as a post-processing step. We offer adequate software support for user control over the proposed system. By enabling such search-space pruning as a post-processing step, we achieve near-perfect recall across representative workloads while achieving orders of magnitude performance and energy improvement over state-of-the-art algorithmic approaches on million and billion-scale workloads.

2026-09-05T21:43:50Z 14 pages, 10 figures, 3 Tavles Ananth Krishna Prasad Rajeev Balasubramonian Mahdi Nazm Bojnordi http://arxiv.org/abs/2609.09208v1 2026-09-05T21:14:28Z

Power-of-two (PoT) quantisation turns a multiplication into a bit shift, so far only for the post-softmax attention--value product. The earlier and larger product, S=QK^T, has not been reformulated the same way. We quantise the key cache to a signed power-of-two fixed-point code, so that every scalar multiplication in QK^T becomes a sign flip, a bit shift and an integer accumulation. A fixed-point head-room F>= e_max turns every shift index r=F-e into a non-negative left shift, making the accumulation exact in integer arithmetic. We add a sub-power mantissa extension that halves the score error for one extra shift-add, a shift-exact online softmax whose running maximum lives in an integer log2 domain so that every rescaling is itself an exact shift, and a log-quantised AV product. Fused CUDA kernels in a 1.1B Llama decoder on an RTX 4090: at B=8, T=32k the 4-bit kernel is 4.60x faster than FP16 scaled dot-product attention with a 2.5x smaller KV cache, and batched decoding crosses over at B=64 to 1.22x FP16 throughput at 11.02 against 13.77 GB peak memory. A matched INT8 multiply--accumulate kernel on the same tiling reaches 5.29x: on a GPU with a hardware 4-way INT8 dot product, shift-accumulate is not faster than the MAC it replaces. An iso-storage control--an 8-bit shift code of exactly one byte per key, run through the identical kernel--prices the arithmetic substitution alone at 2.02x at comparable accuracy. The advantage of the PoT representation is therefore its density and the removal of the multiplier, not raw GPU throughput. A sweep over exponent width, mantissa levels, granularity and rounding gives the design lesson: nearest-PoT relative error is scale-free, so extra exponent bits buy nothing (eps_S=0.1302 at 3, 4 and 5 bits) and accuracy must come from mantissa terms.

2026-09-05T21:14:28Z Khubaib Ahmed Amna Noor Ahsan Ul haq http://arxiv.org/abs/2609.06157v1 2026-09-05T15:57:26Z

3D Gaussian splatting (3DGS) represents scenes with explicit primitives and supports real-time novel-view synthesis, yet its system efficiency varies substantially across scenes, viewpoints, rendering paths, and platform constraints. Existing studies pursue efficiency through representation and algorithm design, GPU runtime optimization, and architectural support, but their reported gains correspond to different points along the rendering and update paths. Connecting these indicators to end-to-end system benefit requires tracing how each optimization changes Gaussian selection, screen-space work, data movement, and stage or frame time. We therefore use a workload-centric framework to connect representation and algorithm research, GPU runtimes, and hardware architectures and to identify recurring workload patterns. We complement literature analysis with reproduced measurements and controlled GPU profiling of selected implementations, relating workload counts to stage time and memory traffic. Together, these comparisons show that system gains depend on workload reductions reaching downstream execution, granularity matching each stage, and the cost of data transfers, synchronization, and cached results, gradients, and optimizer data. Building on these findings, we discuss more consistent evaluation under rendering-quality constraints and identify key directions for future system design.

2026-09-05T15:57:26Z Minnan Pei Qiwei Dong Yihan Zhou Gang Li Yuchen Zhu Wenju Zhao Zhongtian Long Siting Wang Peisong Wang Jian Cheng http://arxiv.org/abs/2609.06125v1 2026-09-05T14:47:20Z

Wafer-scale GPUs (WSGs) provide sufficient on-wafer bandwidth to make near-lossless Unified Memory feasible. However, existing designs still rely on a CPU-IOMMU to translate remote virtual-address accesses. This centralized mechanism scales poorly to tens of GPU dies: translation requests must traverse costly off-wafer hierarchies and contend for limited CPU-side resources, making address translation a critical bottleneck. We propose WaferTrans, an IOMMU-free distributed virtual-address translation design for WSGs. WaferTrans introduces PTE Presence Consistency (PTE-PC), a lightweight consistency model that tracks PTE insertions and removals, and equips each GPU with a PTE Presence Directory (PPD) that locates the GPU holding a requested PTE. It further employs a distributed PTE-PC mapping and a cooperative query mechanism to localize PTE-PC maintenance while preserving complete lookup coverage. Together, these mechanisms enable the GPU array to resolve remote translations within the wafer, eliminating its dependence on the CPU-IOMMU. Compared with the SOTA Trans-FW design, WaferTrans improves performance by 2.5x on average.

2026-09-05T14:47:20Z Xinru Tang Jingxiang Hou Guanghong Wu Yang Hu Shouyi Yin http://arxiv.org/abs/2609.05982v1 2026-09-05T08:55:07Z

Coarse-Grained Reconfigurable Arrays (CGRAs) are a promising solution for achieving high energy efficiency and reconfigurability across various application domains, but their performance is often crippled by rigid memory architectures that limit the number and location of tiles that can access data memory. This creates a significant bottleneck for kernels with intensive memory accesses. To address this, we propose CMD, an integrated CGRA framework featuring cluster-based distributed memory design with a co-designed compilation toolchain. The compiler includes a novel memory-aware mapper and a design space exploration (DSE) mechanism that identifies the optimal memory architecture design for specific kernels. Experimental results show that our post-DSE CMD CGRAs achieve an average speedup of $1.39\times$ over a conventional CGRA while simultaneously reducing the total area to an average of $0.912\times$ of the conventional CGRA.

2026-09-05T08:55:07Z Accepted by ICCD 2026 Shangkun Li Cheng Tan Zeyu Li Jinming Ge Jiawei Liang Hao Yang Linfeng Du Jiang Xu Wei Zhang http://arxiv.org/abs/2608.01536v2 2026-09-05T08:42:22Z

Large Language Models (LLMs) increasingly rely on sparsity to cut inference cost, but most prior work exploits a single sparsity source and targets batched multi-user inference. Dual-sparsity, which pairs unstructured weight pruning with runtime activation sparsity, offers a compelling size-accuracy-latency tradeoff for single-user decoding, but forms a Sparse Matrix-Sparse Vector (SpMSpV) workload that existing GPU kernels handle poorly. We propose Celty, a co-designed sparse format, GPU kernel, and SIMT microarchitecture for SpMSpV in LLM inference. Celty's Run-Length Compressed CSC (RLC-CSC) format enables vectorized loading of compressed weight columns and exploits both sparsity sources to skip memory accesses, accumulating partial products in shared memory. The Celty Sparse SIMT Core then adds a pipelined RLC decoder that removes software index reconstruction and repurposes local register files for conflict-free accumulation, operating on the same compressed representation. The kernel alone achieves up to 2.8x over cuBLAS; with the Sparse SIMT Core, speedup reaches 5.3x over cuBLAS at 70% dual-sparsity.

2026-08-02T23:10:04Z ICCAD 2026. The code is available on Github at https://github.com/RuokaiYin/Celty Ruokai Yin Priyadarshini Panda http://arxiv.org/abs/2609.05960v1 2026-09-05T07:53:52Z

Power analysis is crucial in modern chip design flow. Particularly, time-based power analysis can provide fine-grained power consumption information to facilitate the diagnosis of power issues and guide power optimization accordingly. However, it may take tens of hours to conduct time-based power analysis on modern large-scale circuits, which greatly slows down the power optimization flow. In this paper, we present the first GPU-accelerated gate-level time-based power analysis framework. We propose a novel data structure to enable efficient state-dependent power retrieval. To accommodate the imbalanced event distribution across gates, we propose an event-density-aware partitioning strategy that allocates GPU threads based on gate event density. Finally, we fuse the power computation into a single kernel invocation to reduce redundant work in separate kernels. Experimental results show that our proposed framework achieves high accuracy while delivering up to 37.63x end-to-end speedup compared to multi-threaded Synopsys PrimeTime PX.

2026-09-05T07:53:52Z Accepted for publication in ACM Transactions on Architecture and Code Optimization (TACO) Weihao Wang Yikang Ouyang Hongyuan Liu Yuzhe Ma http://arxiv.org/abs/2609.05764v1 2026-09-04T22:58:21Z

The key-value (KV) cache is the dominant memory bottleneck in long-context large language model (LLM) decoding: every step reads it entirely, so decoding is memory-bandwidth bound. Holding a quantized KV cache in dense on-chip non-volatile memory (NVM) removes the off-chip transfer. Existing KV quantization methods, however, were designed for GPU-style memory systems: KIVI attaches per-group metadata, adding about 25% to the stored KV cache; KVQuant keeps sparse full-precision outliers that a dense array cannot hold in place. This paper examines what these structures cost when the KV cache resides in NVM behind fixed-range converters, and designs a quantization scheme matched to that interface. The architecture stores the quantized KV cache in dense on-chip NVM, uses a small static analog crossbar only for the fixed rotation, and keeps attention in on-chip digital logic. A randomized rotation and per-vector normalization give every coordinate the same range, so one fixed codebook for keys and one for values, each shared across all tokens of the corresponding tensor type, serve the entire KV cache. The codebook thresholds are programmed once as the read converter's reference levels, enabling fixed-range digitization with no per-token converter reconfiguration. Dequantization is a sixteen-entry lookup and one norm multiply; the only per-vector metadata is one scalar, about 3%. Across models from 3B to 14B and contexts to 32k tokens, the four-bit KV cache maintains accuracy under storage and crossbar noise simulated at realistic device levels. KIVI and KVQuant remain more accurate in software; the advantage of our format lies at the memory interface: 3.1-3.6x lower KV read energy than both mapped to the same NVM, and 8x lower metadata overhead than KIVI. The contribution is a KV quantization co-designed with the NVM memory interface rather than a new accuracy record.

2026-09-04T22:58:21Z IEEE/ACM International Conference on Computer-Aided Design (ICCAD 2026) Jiahao Zheng Yifan Qin Xiaobo Sharon Hu Yiyu Shi 10.1145/3831252.3845860