https://arxiv.org/api/L79nIRgc1aftYhQNKaQcPMeCSZo 2026-11-12T03:27:29Z 1575 0 15 http://arxiv.org/abs/2611.14356v1 2026-11-10T17:16:47Z

Rapidly evolving LLM architectures and heterogeneous mobile hardware make supporting new models in on-device inference engines a recurring and costly system challenge. Each model may require coordinated changes across conversion, graph construction, operators, and hardware backends. Existing coding agents often miss these cross-layer dependencies and struggle to diagnose failures across host--device execution paths. We present EngineAgent, an agentic system for automating on-device LLM model enablement. Its key insight is to separate reusable engine and hardware knowledge from model-specific implementation. Before enablement, EngineAgent constructs a hierarchy-aware engine index that captures engine abstraction layers and cross-layer dependencies, and pre-validates backend-specific build, execution, and debugging workflows using already supported models. For each new model, a builder uses this knowledge to coordinate cross-layer changes, while an auditor checks implementation completeness and drives iterative debugging through device-side validation. Across 12 tasks spanning three mobile LLMs and four engine--backend pairs on CPUs, GPUs, and NPUs, EngineAgent achieves 75% end-to-end success with either of two underlying LLMs, versus 33.3% for the strongest coding-agent baseline. Success requires compilation, on-device execution, and output-logit agreement with reference implementations. EngineAgent also reduces average per-task time and model-API cost relative to baselines using the same underlying LLMs.

2026-11-10T17:16:47Z Zhengxiang Huang Yujie Sun Shengheng Chen Zhaode Wang Chaoyue Niu Zeyu Zhao Chengfei Lv Fan Wu Guihai Chen http://arxiv.org/abs/2611.13036v1 2026-11-09T10:52:13Z

Capability-based resource-controlling microkernels securely multiplex hardware by exposing low-level resource abstractions guarded by capability systems. Such a design helps provide an application-specific resource management policy to achieve high performance without sacrificing security. However, the misuse of capability-protected secure abstractions can introduce performance overhead in user-level resource management, affecting the runtime performance of dynamic applications in emerging scenarios. In this paper, we focus on the secure abstraction of physical memory in microkernels with capability-based resource management delegation to investigate an appropriate design for dynamic memory management. We propose CapBuddy, an approach to building an efficient buddy system on capability-protected secure abstractions of memory, to bypass the performance overhead of the capability system by offloading functionalities that can run offline from the critical path of serving a memory request. We implemented a prototype system on the seL4 library OS, and the evaluation results show that our approach improves the performance of the seL4 user-level buddy system by 20 to 54.5 times and achieves the same order of magnitude as Linux. Further burst experiments evaluate CapBuddy's scalability and show that it maintains low median allocation latency as the number of concurrent clients increases.

2026-11-09T10:52:13Z Accepted by ACM Transactions on Architecture and Code Optimization (TACO) Zhuang Lu Guangtao Zhu Shijun Zhao Qianying Zhang Lixin Chen Zekun Xu Wu Luo Rui Hou http://arxiv.org/abs/2611.11739v1 2026-11-09T01:27:50Z

Lazy container-image pulling makes cold starts of autoscaled model-serving fleets on Kubernetes nearly independent of model size, but it moves the model's bytes into a node-level cache that the platform neither bounds nor accounts for. This article establishes, by experiment, two impossibility results for that cache. A monitoring-side remedy cannot catch its failure: a sentinel readiness probe stays green while 272 of 280 files are unreadable. A platform-side remedy cannot contain it: kubelet, given full disk-pressure visibility, frees 0 bytes and evicts 26 bystander pods, because its reclamation actuator is keyed to an accounting that does not know the cache exists. The article then eliminates the failure class inside the component that owns the bytes, with no change to kubelet or the container runtime interface, by two design rules: a byte budget whose pressure signal is keyed to filesystem truth, so that a never-blocking, lossy accounting still enforces a hard bound (held against about 301,000 dropped updates), and pass-through semantics under which a failed cache write never becomes a failed read, with the coupling traced to a single io.MultiWriter. Across six pre-specified campaigns on AWS, upstream stargz-snapshotter fails 273 of 280 model files at 90% cache pre-fill; the proposed system reads all 150.3 GB with zero errors at +18.5% completion cost and holds the budget under a live read, co-tenancy, and a three-node fleet. A source audit shows the missing bound generalizes to SOCI and Nydus. The implementation is open source and has been proposed upstream.

2026-11-09T01:27:50Z 10 pages, 2 figures. Under review at IEEE Access. Data, protocols, and harness: https://github.com/kliukovkin/lazy-model-delivery-study Georgii Kliukovkin http://arxiv.org/abs/2611.10453v1 2026-11-07T04:05:17Z

Agentic reinforcement learning (RL) rollouts interleave model reasoning with tool calls in persistent workspaces. Scaling these rollouts incurs substantial memory costs for concurrent environments, while slow tool calls delay entire rollout batches. We present BunshinRL, an agentic RL rollout system that exploits shared environment state and recurring tool calls through three techniques: memory sharing across environments, tool-prefix caching, and speculative tool execution. These techniques build on BunshinFork, a novel live whole-workspace forking mechanism for containers. BunshinFork reconstructs process trees, shares memory copy-on-write, and preserves file, communication, and GUI state, enabling cached-state reuse and rollback after incorrect speculation. We implement BunshinRL and BunshinFork through Linux kernel module and CRIU modifications. BunshinRL reduces peak memory by up to 43%, total memory consumption by up to 51%, and end-to-end rollout latency by 23%.

2026-11-07T04:05:17Z Yutong Huang Vikranth Srivatsa Alex Asch Yiying Zhang http://arxiv.org/abs/2611.10312v1 2026-11-06T23:24:55Z

A phone that shows calls, messages and photographs from years ago may have been set up recently and carried them over from an earlier handset, so its screen dates do not date the device's own life. Which traces fix where that life begins, how well they agree and how much of what is visible predates it is unmeasured. We read 14 Android full file system extractions (12 Samsung; Android 10 to 16), seven bug reports and 33 iOS extractions (a probe). Every Android device recorded a first boot, but the first evidence of user setup came a median of 98 days later (0 to 563). Of the 13 devices with messages, 7 held some dated before the first boot, a majority of them on 5; 24% of the apps outside the system image were installed as restored. Against the creator field that a migration tool leaves on messages, a boundary at user setup recalled all but 1 of 46,968 such messages, the first-boot boundary 94 to 100%. On synthetic fixtures, the labelling error followed the boundary error. A date on a record dates the device only once the record is shown to have been created on it.

2026-11-06T23:24:55Z 9 pages, 4 figures. Lab Technical Report LAB-TR-0028, SectraTEK Lab; preprint, not peer reviewed. Corresponding author: Daniel Kwaku Ntiamoah Addai Daniel Kwaku Ntiamoah Addai Selorm Adjabeng Kingsley Albert Egyir Sanders Larweh Tetteh http://arxiv.org/abs/2611.10309v1 2026-11-06T23:15:31Z

Whether an extraction can answer a question about a given day depends on how far back each store reaches, yet public statements rest on single test devices. We measured the retention horizon, the time from a store's oldest retained record to acquisition, for eight iOS and seven Android stores on 48 full file system extractions and seven bug reports, keeping devices whose clock agreed with the acquisition time (26 Apple devices, 13 Android phones). Horizons ranged from minutes to years and followed the mechanism that bounds each store. The unified log Persist stream reached a median of 2.5 days; interactionC reached 166 to 170 days on 18 of 25 devices. Android stores stayed within their source-code limits (usage statistics 10.2 days, DropBox 3.1, notifications 1.5); bug-report logs reached 3.1 hours at most. The oldest record misled three times: one knowledgeC stream reached back two years, the others a median of 29 days; interactionC read without its write-ahead log showed older records on 18 of 25 devices; and four of seven bug reports began with stray lines up to 16.6 days old above a body of 3.1 hours or less. A horizon belongs to a store, a release and a device.

2026-11-06T23:15:31Z 17 pages, 3 figures. Lab Technical Report LAB-TR-0021, SectraTEK Lab; preprint, not peer reviewed. Corresponding author: Daniel Kwaku Ntiamoah Addai. Also archived at Zenodo, doi:10.5281/zenodo.24023417 Daniel Kwaku Ntiamoah Addai Sanders Larweh Tetteh Selorm Adjabeng Kingsley Albert Egyir http://arxiv.org/abs/2611.10178v1 2026-11-06T19:58:58Z

Modern LLM serving increasingly extends KV caches beyond GPU memory into host DRAM, SSDs, and remote storage, yet these stores remain largely governed by workload-agnostic policies such as LRU and fixed TTLs. We ask what execution context is needed by storage to make retention, reclamation, and placement decisions across heterogeneous workloads. We characterize KV-cache reuse, lifetime, and footprint at block granularity across 11 production and public traces spanning chat, API serving, RAG, coding agents, tool-using agents, and multi-lineage workflows. We examine reuse sources, recurrence timing and frequency, useful lifetimes, and storage consumption. Across workloads, KV state takes two recurring forms. Single-session state is reused within one execution: its footprint grows as sessions deepen and accumulates across concurrent sessions; it can remain useful through long idle periods but loses value at session completion. Multi-session state recurs across sessions of the same workload, outlives individual sessions, and spans hot prefixes and a large infrequent tail. While prior systems often exploit application-, agent-, or workflow-specific knowledge, our results suggest that three generic signals (a workload identifier, a session identifier, and session completion) expose much of the structure needed for KV-storage management. Trace-driven comparisons show that workload-specific recurrence estimates retain up to 22% more within-session hits and reduce dead state by up to 40%; a session completion signal eliminates the fundamental idle-time reclamation tradeoff for single-session state; and session-aware reuse classification identifies shared state 4-700x more selectively than request-level access counts. These results motivate a small cross-layer interface for workload-aware KV storage without application-specific semantics.

2026-11-06T19:58:58Z Talia Q. Chen Daniel Waddington Narangerelt Batsoyol Vijay Chidambaram http://arxiv.org/abs/2512.24637v3 2026-11-06T06:41:52Z

Limited HBM capacity has become the primary bottleneck for hosting a growing number of increasingly larger-scale GPU tasks. While demand paging extends capacity via host DRAM, it can incur order-of-magnitude slowdowns due to the massive working sets of GPU workloads. However, we observe that GPU memory accesses are inherently predictable from kernel launch arguments, prior executions, and the asynchronous execution nature. Leveraging this opportunity, we propose MSched, an OS-level scheduler that extends GPU context switching to include proactive working set preparation. This approach coalesces fragmented, eventual, and expensive page faults into a single efficient migration. MSched combines parameterized templates with page fault feedback to accurately predict working sets, and co-designs the task scheduler and memory manager to resolve global access order and enforce near-optimal page replacement. Across scientific, deep learning, and LLM workloads, MSched outperforms demand paging by up to 186.69x and whole-process swapping by up to 8.63x. MSched is open-source and has been integrated into https://github.com/XpuOS/nvxsched.

2025-12-31T05:18:52Z Weihang Shen Yinqiu Chen Rong Chen Haibo Chen http://arxiv.org/abs/2611.08638v1 2026-11-05T19:35:20Z

Existing agent serving systems divide each prompt into a reusable prefix and an opaque suffix, managing the KV-cache at this coarse granularity without distinguishing the components within each part. We observe that prompt components differ in which LLM calls share their values and how long those values remain reusable, a property we term intra-prompt heterogeneity. Our key insight is that, in frameworks that generate prompts from typed LLM function calls, these components correspond to program variables, allowing compiler analysis to recover their sharing patterns and lifetimes. We present MORTIS, a compiler-serving co-design that combines this analysis with runtime observations to improve KV-cache management. MORTIS reorders prompt components to expose additional prefix reuse and uses their predicted reuse and lifetimes to guide cache retention, eviction, and prefetching. Our evaluation shows that MORTIS achieves up to 1.99x job completion time (JCT) speedup and 1.97x throughput improvement over SGLang, with geometric-mean speedups of 1.45x and 1.49x, respectively.

2026-11-05T19:35:20Z 10 pages Xiaoyu Ma Jayanaka L. Dantanarayana Savini Kashmira Baichuan Li Lingjia Tang Jason Mars http://arxiv.org/abs/2611.07506v1 2026-11-05T06:10:55Z

Server main memory is increasingly heterogeneous, incorporating technologies such as zswap, CXL memory expanders, and emerging phase-change memories (PCM). To decide where to place pages, operating systems have to choose between efficient but coarse-grained access bits, out-of-band profiling, or costly runtime instrumentation. This paper proposes memory fit, a hardware abstraction that provides precise and accurate page placement information to an operating system. Memory fit measures how well a page "fits" each type of memory in the system. Memory fit separates the mechanism of measuring affinity between pages and memories from the OS policy of choosing which pages to move when. The paper describes soft trace matching (STM), a computationally and space efficient algorithm to compute memory fit. The key technical insight behind STM is that it can estimate memory fit by comparing accesses to a page with a learned model of "good" traces for a memory. We have implemented memory fit and STM in a BOOMv3 RISC-V processor running Linux 6.6. On extended GUPS, memory fit placements recover 2.2-3.8 times as much of the performance gap to optimal placement as Linux TPP. For Redis with equal DRAM and PCM capacities, placements selected by memory fit recover 80% of the performance gap, versus 19% for TPP, in a placement replay. Memory fit requires only 64 bits of state per 4 KiB page and can be updated on every main-memory access with only a 1.2% overhead of OS kernel notification processing.

2026-11-05T06:10:55Z Chun Deng Hui Sub Shim Evan Laufer Anna Eaton David Tennenhouse Tsachy Weissman Philip Levis http://arxiv.org/abs/2611.06461v1 2026-11-04T12:57:17Z

NVMe Flexible Data Placement (FDP) reduces SSD write amplification by separating data with different lifetimes to minimize garbage collection (GC) copying. However, on shared servers, multiple tenants generate more lifetime classes than the available placement handles. Existing approaches rely on static mappings or application-specific knowledge, failing to capture runtime lifetimes and adapt to changing workloads. The challenge is to identify which data die together and determine which classes should be separated to maximize GC savings under a limited handle budget. We present eFDP, an eBPF-based system that automatically learns data lifetimes and manages FDP placement for unmodified applications on a stock Linux kernel. By tracking writes and invalidations, eFDP identifies lifetime classes and estimates the GC copying saved by separating them. It first assigns each tenant a handle, then allocates the remaining handles to the separations offering the greatest savings. It also re-plans placement when active tenants change. With four database tenants on an emulated FDP SSD, eFDP eliminates twice as much excess write amplification as tenant-level isolation and improves every writing tenant's throughput by 19-23%. On an enterprise FDP SSD, given RocksDB's levels, it matches a manually optimized placement using three handles instead of five. After a tenant switch, eFDP re-plans within seven minutes and maintains 25% lower write amplification than the original layout even after an hour.

2026-11-04T12:57:17Z Hajin Moon Taewan Noh Chaeeun Yoon Jeonghun Lee Junyoung Han Joo-Young Hwang Youngjae Kim http://arxiv.org/abs/2611.04253v2 2026-11-04T11:08:20Z

On-demand model loading dominates response latency for interactive on-device AI, yet unified memory architectures (UMAs), now common on edge devices, should make it cheap by placing weights in memory the GPU already sees. However, frameworks still use the loading flow built for discrete GPUs, leaving significant inefficiencies. This paper presents UMLoad, a system-level model-loading accelerator that decouples a framework's logical allocations from their physical page backing. UMLoad uses on-demand reconstruction to eliminate the weight copy while preserving copy independence, and GPU-assisted page zeroing to accelerate runtime allocation while preserving its security guarantee. Implemented on Intel, AMD, and NVIDIA UMA platforms and evaluated with llama$.$cpp, PyTorch, and vLLM across six models, UMLoad reduces average loading time by 36.6%-71.9% over vanilla and 19.9%-62.7% over pipelined loaders, and cuts mean LLM-routing time to first token (TTFT) by 51.3%-59.3% and model-backed tool completion latency by 10.2%-31.6%.

2026-11-03T03:44:38Z 17 pages, 12 figures, 2 tables Zimeng Wang Jinrong Yang Rong Chen http://arxiv.org/abs/2611.06071v1 2026-11-04T07:17:33Z

Emerging I/O-centric architectures empower peripheral devices to access storage directly while bypassing the host CPU and kernel. However, SSDs expose only a block interface and rely on the host filesystem for file metadata and access control, making file-level access for applications and peripheral devices inherently difficult in I/O-centric architectures. This tight coupling between the host filesystem and storage device introduces unavoidable software overhead and limits the potential of fully hostless I/O paths. In this paper, we present File-Semantic SSD (FS-SSD), a new SSD architecture that provides a native file abstraction within SSDs, enabling direct file access by peripheral and remote devices without host intervention. FS-SSD adopts the lightweight, stateless NFS interface and introduces NFS-over-NVMe, a protocol extension that embeds NFS operations into the standard NVMe command structure. This allows FS-SSD to support local and remote file I/O through existing PCIe and NVMe-oF transports. Our evaluation shows that FS-SSD provides on-device file abstraction while fully utilizing raw-device bandwidth in I/O-centric architectures.

2026-11-04T07:17:33Z Kyu-Jin Cho Injae Kang Keunsan Park Jaehoon Shim Munsung Bae Jiho Jeon Eyee Hyun Nam Jin-Soo Kim http://arxiv.org/abs/2611.05971v1 2026-11-04T05:05:43Z

Advances in CPU and GPU confidential computing enable confidential AI, allowing LLMs to process user prompts without exposing them to service providers or infrastructure operators. However, AI agents often need to send prompt-derived information to external services, and providers may also require limited disclosure for safety monitoring and failure diagnosis. We propose CORDE, a system for confidential virtual machines (CVMs) designed to satisfy the principle of Confine or Declare: user-derived data is confined by default, but when necessary, it may leave the CVM only after being "declared." To "declare" data is to record its exact contents, recipient, and stated reason in a user-accessible, tamper-evident declaration log. To put this principle into practice, we present an integrated system design that combines per-user information-flow control in Rust with a TEE-based ledger and remote attestation. Our evaluation on an NVIDIA H100 GPU shows that declaration before disclosure adds little overhead in task latency, about 0.5% on the AppWorld agent benchmark. CORDE provides a basis for accountable disclosure of user data, supporting responsible and transparent data management in AI services.

2026-11-04T05:05:43Z Shumpei Shiina Haruka Takahashi Kento Sasaki Takuya Hayashi Tsubasa Takahashi http://arxiv.org/abs/2611.05269v1 2026-11-03T17:55:21Z

Local LLM agents often work on shared hosts. Unlike multi-tenant cloud clusters, these hosts often have tightly bounded memory, CPU, and I/O resources. Concurrent agents frequently suffer from resource contention, leading to failures such as OOM and reduced resource efficiency. Considering that agents' resource demands are bursty and shell command delay is imperceptible to agents, there is a huge scheduling opportunity to arrange when and in what order commands are actually executed. Thus, we present ToolSched, a Shell MCP Server for shared local hosts that serves multiple agents and schedules their operations. ToolSched embeds a resource model into the MCP interface: agents estimate resource demands along with command requests. These hints, together with real-time host resource conditions, determine when commands are actually executed. Internally, ToolSched applies a scheduling algorithm to stagger commands, reducing contention and improving resource utilization. Prediction correction and checkpoint-based recovery mitigate estimation bias and resource errors. Our evaluation on Terminal Bench shows that, compared with unscheduled execution, ToolSched improves resource utilization by 148.4% and end-to-end command throughput by 29.4%, demonstrating its effectiveness and practicality.

2026-11-03T17:55:21Z Shucheng Zou Yuhao Gu Long Chen Zhongchun Zheng BinZe Xie Xianwei Zhang