https://arxiv.org/api/XjrapaQd3gNR5s8Eq3apbfPdBo0 2026-07-17T19:55:51Z 8608 0 15 http://arxiv.org/abs/2607.15225v1 2026-07-16T17:28:39Z

We present campaign diagrams, a visualization technique for phase-level analysis of resource utilization and bottlenecks in modern workloads. Existing tools have a trade-off: rooflines aggregate a workload into a single point and lose all notion of time, while profilers and traces expose fine-grained events but obscure what bounds performance. Instead, a campaign diagram depicts compute throughput and memory bandwidth utilization, compute and memory traffic volume, and latency in a single figure. Since they can be generated from analytical models, simulations, or profiling data, campaign diagrams capture both ideal bounds and a kernel's achieved performance. We demonstrate them on two case studies: a low-rank GEMM, where they reveal the counterintuitive result that reducing operational intensity can improve end-to-end performance, and Mamba, where they expose fusion and pipelining opportunities across phases. In both cases, our visualization technique reveals optimization opportunities that are difficult to identify with rooflines or profilers alone.

2026-07-16T17:28:39Z 12 pages, 13 figures Toluwanimi O. Odemuyiwa John D. Owens Michael Pellauer Joel S. Emer http://arxiv.org/abs/2607.15123v1 2026-07-16T15:32:44Z

Recent FPGAs have improved deep learning (DL) inference efficiency through dedicated tensor blocks and in-BRAM computation. ReRAM-based analog in-memory computing (IMC) pushes efficiency further, offering an order-of-magnitude improvement in compute density and energy efficiency over conventional digital logic by performing vector-matrix multiplication (VMM) directly within the ReRAM crossbar; prior work has integrated such IMC blocks into FPGAs for DL inference. However, conventional IMC designs support only static-weight VMM, leaving nonlinear operations and dynamic matrix-matrix multiplication (DIMM) to the FPGA fabric. As a result, the benefits of IMC are largely confined to static-weight models, whereas Transformer-based models, which rely on frequent nonlinear and DIMM operations, gain only limited improvement. Moreover, the ADCs within each IMC block consume more than 70% of its area and power, further limiting system efficiency and scalability. To address these limitations, we propose a novel FPGA architecture that integrates an ADC-free IMC block, replacing the conventional ADC with analog content-addressable memories (ACAMs) that natively perform nonlinear operations inside the block. To fully exploit this block, we conduct an FPGA-aware design-space exploration that determines optimal crossbar dimensions while balancing FPGA area, flexibility, and DL performance, and we develop an efficient mapping that leverages ACAMs to carry out DIMM operations, extending the applicability of IMC to attention computation. On CNN and Transformer-based benchmarks, the proposed architecture achieves up to 40x and 1.9x higher energy efficiency and 4.1x and 2.5x higher area efficiency, respectively. Overall, it significantly improves FPGA DL inference efficiency and sustains robust gains on Transformer-based workloads across long input sequences, advancing domain-specialized FPGA design.

2026-07-16T15:32:44Z Jiajun Hu Ruthwik Reddy Sunketa Lei Zhao Archit Gajjar Luca Buonanno Aman Arora http://arxiv.org/abs/2507.05876v2 2026-07-16T14:46:24Z

Asynchronous Distributed Reinforcement Learning (DRL) can suffer from degraded convergence when model updates become stale, often the result of network congestion and packet loss during large-scale training. This paper presents \textit{Olaf}, an in-network DRL accelerator engine, which \textit{opportunistically aggregates} asynchronously generated model updates \textit{on the fly} while they traverse the data plane queue. This aggregation operation motivates an alternative queue design, which we prototype and envision for future Top-of-Rack switches. We further present corresponding host-side transmission control in the face of possible congestion, taking advantage of in-network accelerator feedback. A quantification of model staleness, denoted Age-of-Model (AoM), together with a formal verifier, allows us to reason on system-wide AoM objectives in multi DRL-cluster scenarios. Olaf shows significant reductions in model staleness and queue congestion, improving overall convergence rate for asynchronous DRL workloads.

2025-07-08T10:59:56Z 17 pages, 11 figures Nehal Baganal Krishna Anam Tahir Firas Khamis Mina Tahmasbi Arashloo Michael Zink Amr Rizk http://arxiv.org/abs/2607.15068v1 2026-07-16T14:41:17Z

High-level synthesis (HLS) raises the abstraction level of FPGA accelerator design from hardware description languages to C/C++, but high-quality results still depend on schedule decisions such as pipelining, unrolling, tiling, reordering, and buffering. These decisions create a combinatorial design space, while many numerical kernels exhibit recurring computation patterns that suggest different optimization strategies. This paper presents PATTERNDSE, a lightweight pattern-guided design space exploration (DSE) framework for FPGA kernels written in Allo, a scheduling-oriented HLS programming system. PATTERNDSE maps recurring computation patterns, including elementwise maps, reductions, matrix-vector operations, matrix-matrix operations, and stencil-like updates, to compact schedule spaces. It then applies candidate schedules, validates functional correctness through LLVM execution, checks HLS C code generation, and uses a simple pattern-aware estimator to rank candidates before Vitis HLS synthesis. We evaluate PATTERNDSE on six representative kernels: vecadd, axpy, dot, matvec, gemm, and jacobi2d. Compared with an exhaustive-lite baseline, pattern-guided DSE reduces the number of HLS-evaluated candidates from 140 to 29, achieving a 4.83x overall search reduction and up to 12.0x reduction for individual kernels. Across all evaluated kernels, PATTERNDSE recovers the same best valid Vitis HLS latency as the exhaustive-lite baseline, demonstrating that computation-pattern information can prune unproductive schedule combinations while preserving high-quality HLS outcomes.

2026-07-16T14:41:17Z 6 pages, 4 figures, IEEE ICECCME conference Jialiang Zhang Weiman Yan Yuelin Zou http://arxiv.org/abs/2607.15005v1 2026-07-16T13:52:30Z

As advanced packaging technology evolves, increasing interconnect density in redistribution layers (RDLs) makes routability critical to package floorplanning. Meanwhile, power integrity requirements often reserve fan-in regions for the power delivery network (PDN), forcing signal nets through fan-out regions and complicating routability estimation. Existing uniform grid-based congestion models cannot accurately characterize fan-out congestion, while previous pin assignment methods struggle to evaluate net crossings. We propose a differentiable routability-driven floorplanning and pin assignment algorithm for advanced packaging with fan-out routing. First, a differentiable wirelength minimization method directly models discrete chip orientations and back-propagates wirelength gradients to chip locations and orientations. It reduces wirelength under fixed pin selection while avoiding the bias of continuous-angle modeling. Second, a crossing-aware pin assignment method incorporates net-crossing cost into a multi-strategy DPSO algorithm and uses GPU-parallel cost evaluation to reduce wirelength efficiently. Finally, a differentiable routability maximization method constructs a congestion model tailored to fan-out routing and establishes a back-propagation path from congestion information to chip locations, thereby guiding routability optimization. Experimental results show that our method achieves 100% routability on all benchmarks. For cases successfully routed by the baselines, it reduces wirelength by up to approximately 23% compared with a leading floorplanning method equipped with our pin assignment flow.

2026-07-16T13:52:30Z Yiqi Huang Zepeng Li Zhen Zhuang Kehao Chen Genggeng Liu Tsung-Yi Ho http://arxiv.org/abs/2607.14908v1 2026-07-16T12:28:01Z

Deploying Video Diffusion Models (VDMs) on edge devices is appealing for localized and privacy-preserving generation, but their iterative Transformer-based denoising remains too slow for practical local inference. Cross-Timestep Caching (CTC) has emerged as a promising direction for reducing redundant computation, reusing activations across adjacent denoising steps rather than modifying model weights, while largely preserving generation fidelity. However, on memory-constrained edge GPUs, CTC requires a massive cache footprint that quickly exceeds on-device VRAM and forces the cache into host memory. More fundamentally, cache operators remain tightly interleaved and chain-dependent with native compute operators, so naive near-memory offloading still incurs repeated PCIe exchanges for residual and fusion computations, turning cache reuse into a communication- and serialization-bound execution flow. We therefore propose CODA, an algorithm-hardware co-designed architecture centered on Compute-Cache Operator Disaggregation. CODA separates dense compute paths and memory-bound cache paths across the xPU and a lightweight DIMM-side near-memory engine, reorganizes fragmented cache activity into hardware-friendly coalesced segments, and exploits Classifier-Free Guidance (CFG) branch independence to overlap xPU compute with cache-side execution. Experiments show that CODA achieves up to 1.80x end-to-end speedup and 1.74x higher energy efficiency, while preserving competitive generation quality compared with a state-of-the-art caching algorithm.

2026-07-16T12:28:01Z 15 pages, 14 figures, accepted to the 59th IEEE/ACM International Symposium on Microarchitecture (MICRO 2026) Yuanpeng Zhang Peking University YuXuan Wu Peking University Yitong Xiao Peking University Chenhao Xue Peking University Yi Ren Peking University Cong Li Peking University Yihan Yin Peking University Dimin Niu Alibaba Group Inc. Guangyu Sun Peking University http://arxiv.org/abs/2604.26979v2 2026-07-16T12:20:14Z

In-memory computing (IMC) is a paradigm that enables neural network inference by computing analog matrix-vector multiplications (MVM) directly in memory crossbar arrays, with the potential for energy efficiency gains over conventional von Neumann architectures. In this work we present a simulation framework for N-ary crossbar architectures that retrieves MVM results with minimal implementation assumptions. The XOR and MNIST classification tasks were successfully inferred using a simulated crossbar array of (4x4) 4-states magnetic tunnel junctions (MTJ). MNIST accuracy reached 93.56% (vs. 97.56% software baseline). PCA dimensionality reduction was shown to drastically lower the number of required operations and improve the software baseline, for only a modest reduction in crossbar inference accuracy. We identified weight quantization as the primary error source, and studied its impact alongside systematic non-idealities and random noise. We find that cell-specific random noise is less detrimental than systematic errors due to averaging across the array. Finally, we demonstrate an optimal number of states per cell that balances quantization error against resistance state resolution to minimize total MVM error.

2026-04-28T13:29:51Z 13 pages, 7 figures, 3 tables Anatole Moureaux Anthony Lopes Temporao Flavio Abreu Araujo http://arxiv.org/abs/2607.14789v1 2026-07-16T10:08:26Z

Physical memory allocation establishes virtual-to-physical mappings on demand. In current systems, each minor page fault traps into the kernel and triggers pipeline flushes, stalls, and a long sequence of allocation steps that can cost tens of thousands of cycles. These overheads are increasingly significant for short-lived workloads such as serverless functions and microservices, where minor faults can account for up to 54% of runtime and up to 40% of system energy. Prior hardware allocation proposals avoid traps and context switches, but either sacrifice useful placement optimizations or rely on fixed-function logic that cannot adapt to new policies or changing hardware conditions. We present Valinor, a hardware-OS cooperative memory allocation substrate that combines software flexibility with hardware-class performance. Valinor introduces a programmable hardware allocation engine that executes compact OS-supplied allocation libraries at close to fixed-hardware speed. It supports diverse policies, including short-lived object allocators, integrity mechanisms, and hardware-telemetry-guided placement. We implement Valinor on a BOOM RISC-V soft core running Linux and in a full-system simulator. On real hardware, Valinor accelerates allocation by 17x, improves end-to-end performance by 16%, and reduces energy consumption by up to 8%. Full-system simulation further evaluates the programmable allocation engine and six allocation libraries, showing that Valinor provides hardware-class performance without sacrificing programmability.

2026-07-16T10:08:26Z Konstantinos Kanellopoulos Spiros Galanopoulos Konstantinos Sgouras Vlad-Petru Nitu Ilias Papalamprou Andreas Kosmas Kakolyris Rahul Bera Dimosthenis Masouros Dimitrios Soudris Onur Mutlu http://arxiv.org/abs/2607.14747v1 2026-07-16T09:31:31Z

Cardiovascular diseases are the leading cause of death worldwide, and conditions such as arrhythmia often require long-term monitoring for effective detection and diagnosis. However, current wearable monitoring devices are bulky, uncomfortable, and typically rely on clinicians to manually evaluate electrocardiograms (ECGs). While Deep Learning (DL) algorithms have shown superior performance in arrhythmia detection and classification, their computational complexity coupled with high power consumption limit deployment in wearable devices. To address this challenge, this paper investigates the use of approximation techniques to reduce the power and energy consumption of DL architectures while maintaining acceptable classification performance. Specifically, techniques such as data precision reduction and approximate multiplication are investigated in a state-of-the-art DL model and its corresponding hardware architecture. The model is trained and validated using the MIT-BIH Arrhythmia Database, and hardware implementations employing various approximate multipliers are synthesized and evaluated. Compared with the state-of-the-art 8.75 μW (and 2.08 μJ) reference architecture, our proposed architecture consumes 3.07 μW (and 2.17 μJ) at 12 kHz, showing 64.9% reduction in power consumption while providing an acceptable output quality, i.e., 93.7% classification accuracy and 92.1% sensitivity. At 100 MHz, our proposed architecture consumes 9.45 mW (and 0.8 μJ), showing 61.5% reduction in energy consumption as compared to the state-of-the-art architecture. These results demonstrate that our proposed approximations significantly extend wearable device battery life while preserving the required arrhythmia classification performance.

2026-07-16T09:31:31Z Floriaan Bulten Yawar Rasheed Arlene John Vincenzo Stoico Ghayoor Gillani http://arxiv.org/abs/2607.14622v1 2026-07-16T06:38:49Z

Low-bit GEMM is increasingly central to efficient ML inference, yet very-low-bit execution remains a poor fit for conventional CPUs. Practical deployment spans fragmented regimes-from 1/2/4-bit weights to varying activation precision-whose feasibility, reuse opportunity, and support cost differ under fixed SIMD and register-file budgets, making lightweight CPU support selection a first-class design problem. We present ExaGEMM, a workload-aware codesign and exploration framework for CPU-native low-bit GEMM via register-resident LUT execution. The key insight is that existing SIMD datapaths already cover table generation and accumulation; the only new hardware is an in-register select/feed mechanism with explicitly modeled cost. ExaGEMM co-explores parameterized kernels and lightweight SIMD ISA support using analytical models of register feasibility, compute cost, memory traffic, and hardware overhead, pruning the candidate space by 99.2% before simulation. It then identifies non-dominated support points and generates ISA specs, gem5 patches, and GEMM kernels for validation. Across representative ML models and CPU targets, ExaGEMM improves latency by 13.29x over software-only baselines, while showing that workload-aware frontier selection is especially important for mixed-precision LLM workloads.

2026-07-16T06:38:49Z Accepted to ICCAD 2026 Hyunwoo Oh Suyeon Jang Hanning Chen Sanggeon Yun Ryozo Masukawa Mohsen Imani http://arxiv.org/abs/2607.14618v1 2026-07-16T06:31:39Z

CPUs are the most universal target for on-device LLM inference, but existing low-bit quantization methods offer either coarse operating points or fine-grained mixed precision that is difficult to execute efficiently on CPUs. We present PolyQ, a CPU-oriented compiler/quantization co-design for activation-aware channel-wise bit allocation under a user-specified average-bit budget. PolyQ assigns per-channel bit-widths from $\{2,3,4,8,16\}$, then uses a compile-time model compiler to permute and cluster channels into bit-homogeneous blocks, generate SIMD- and LUT-compatible kernels, and merge compatible permutations across operators to keep layout regularization off the runtime path. This turns fine-grained budget fitting into a practical fractional-bit deployment method for CPU-only inference. Across Falcon-H1-3B, Llama2-13B, and Qwen3-32B on WikiText-2, PolyQ provides stable quality scaling from 3--6\,b and improves perplexity by 2.4--32.1\% over prior methods at a 3\,b target. End-to-end measurements on three representative CPUs -- workstation, laptop, and mobile -- show that compiler layout regularization reduces activation reorder traffic by up to 70.8\%, prefill latency and decode throughput scale nearly proportionally with the configured bit budget, and energy/token overhead stays below 2\% relative to an optimized LUT-based back-end. These results show that fractional-bit CPU deployment is practical, predictable, and energy-efficient across diverse edge targets.

2026-07-16T06:31:39Z Accepted to ICCAD 2026 Hyunwoo Oh Suyeon Jang Hanning Chen KyungIn Nam Sanggeon Yun Ryozo Masukawa Mohsen Imani http://arxiv.org/abs/2607.12839v3 2026-07-16T01:46:28Z

Modern edge system-on-chips (SoCs) combine CPUs, integrated GPUs (iGPUs), and neural processing units (NPUs), yet existing LLM runtimes typically make coarse device-level decisions or optimize operators in isolation. As a result, they underutilize heterogeneous resources, particularly on unified-memory platforms where performance depends on both device placement and task-graph coordination. We present HeteroMosaic, a heterogeneity-first scheduling framework for edge LLM inference. HeteroMosaic first uses a heterogeneous roofline model to identify when combining iGPU and NPU execution is beneficial. It then decomposes inference into dependency-preserving micro-batches that expose cross-accelerator overlap and applies trace-guided co-optimization of scheduling and device allocation under practical effects such as memory contention, DVFS, device variation, and NPU runtime overheads. We implement HeteroMosaic in PyTorch C++ and evaluate it on three AMD Ryzen AI platforms spanning NPU-heavy, balanced, and iGPU-heavy designs. On the balanced platform, HeteroMosaic achieves up to 1.73X speedup over an iGPU baseline, 1.78X over an NPU baseline, and 2.05X over frameworks such as llama dot cpp, while reducing energy by up to 45.3%. It also improves performance over prior heterogeneous edge AI solutions by up to 2.35X.

2026-07-14T14:56:14Z Accepted at the 59th IEEE/ACM International Symposium on Microarchitecture (MICRO 2026) Gregory Hyegang Jun Wesley Pang Eddie Richter Mehdi Saeedi Aporva Amarnath Pallavi Ferrao Deming Chen http://arxiv.org/abs/2607.14008v1 2026-07-15T16:37:57Z

In this paper, we introduce Lighthouse RL, a sample-efficient reinforcement learning (RL) approach for analog circuit sizing. Traditional methods lack generalization across different performance targets, while standard RL approaches waste resources exploring unpromising regions. Our method addresses these inefficiencies through a strategic reset strategy that initializes episodes from high-performing configurations discovered during training, called "lighthouses". These states, which are closer to the target objectives, guide exploration toward promising regions. When compared to RL and Bayesian optimization methods from the literature, we demonstrate the effectiveness of our approach on a 2D benchmark problem and on two analog circuits, showing significant improvements in sample efficiency (up to 1.72x faster), optimization performance (100% vs. 0-87% success rate), generalization (75% vs. 0-50% extrapolation success), and objective maximization. This efficiency is particularly valuable for computationally expensive black-box optimization problems, and our reset strategy can be used as a plug-and-play enhancement for any RL-based optimization approach.

2026-07-15T16:37:57Z Mustafa Emre Gürsoy Stefan Uhlich Ryoga Matsuo Yağız Gençer Arun Venkitaraman Chia-Yu Hsieh Andrea Bonetti Eisaku Ohbuchi Lorenzo Servadei http://arxiv.org/abs/2607.13898v1 2026-07-15T14:40:59Z

Modern deep learning workloads increasingly rely on narrow numerical formats to improve efficiency and reduce memory footprint. The recently standardized microscaling floating-point (MXFP) family of formats, including MXFP8, MXFP6, and MXFP4, offers a practical approach to low-precision inference, yet the digital signal processing (DSP) blocks in current FPGA architectures offer limited native support for these formats. In this work, we first present a comprehensive characterization of MXFP dot product implementations on Altera Agilex-5 FPGAs, exploring a range of strategies spanning pure soft logic, DSP blocks in fixed-point, floating-point, and tensor modes. Our results show that while the tensor mode delivers the highest arithmetic density for MXFP4 (E2M1) and MXFP6 (E2M3), it cannot implement MXFP6 (E3M2) or any MXFP8 precisions, forcing designers to fall back to lower-density alternatives. Motivated by this gap, we propose targeted modifications to the DSP block's internal tensor-mode architecture that enable native support for all MXFP precisions while retaining backward compatibility. We estimate the area cost of these modifications using a simplified version of the Agilex-5 DSP block core implemented using the open-source ASAP7 PDK. We evaluate a variety of modified DSP block designs that present a tradeoff between format coverage, arithmetic density, and area overhead. Our preferred design point increases the DSP tile area by 36%, corresponding to only 1.8\% of the total FPGA die area. We evaluate the device-level impact of our enhanced DSP block by comparing systolic array matrix multiplier implementations across all MXFP precisions, contrasting the best-available strategies on the existing architecture against designs leveraging our modified DSP block. Our results demonstrate an average throughput improvement of 4.2x across all supported MXFP formats.

2026-07-15T14:40:59Z Marwan Mekhemer Ahmed Elsousy Balaji Venkatesh Raphael Rowley Vaughn Betz Nachiket Kapre Andrew Boutros http://arxiv.org/abs/2607.13770v1 2026-07-15T12:31:19Z

Video diffusion transformers (vDiTs) generate high quality video but introduce extremely high compute cost due to the long diffusion timesteps and self attention computation. As diffusion timesteps are reduced, the computation cost of self attention becomes the dominant bottleneck. Existing acceleration approaches largely inherit sparse attention techniques from large language models, which fail to consider the unique spatiotemporal correlation of video data. This paper presents Kaleido, an algorithm hardware codesign that accelerates all operations in vDiTs by exploiting channel-wise spatiotemporal correlations in latent space. Based on this insight, we propose a lightweight channelwise reuse algorithm that skips redundant computations by reusing partial results while preserving higher generative quality than prior methods (>17 dB). To efficiently support this algorithm, we design a systolic array like accelerator with reconfigurable processing elements and a lightweight data dispatcher to mitigate irregular sparsity and data access patterns introduced by our reuse algorithm. Evaluations across three mainstream vDiT models show that Kaleido achieves up to 5.9x speedup and 16.0x energy savings over state of the art accelerators.

2026-07-15T12:31:19Z Wenxuan Miao Haosong Liu Weiming Hu Zihan Liu Aiyue Chen Jianlin Yu Yiwu Yao Yiming Gan Jieru Zhao Jingwen Leng Minyi Guo Yu Feng